diff --git a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.i18n.yaml b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.i18n.yaml index 3cc2da0976..966be9e743 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md -2026-07-23-acp-automation-only-protocol.md: 56dcaf8b4327a008f26b884264958cac02d6541a -2026-07-23-acp-automation-only-protocol.zh.md: a34e179a1d38c1867ea8165b149a9ec7716c1c0e +2026-07-23-acp-automation-only-protocol.md: 3d13e3fb51819ef4f892f33f9c86554988576e36 +2026-07-23-acp-automation-only-protocol.zh.md: 224c1bd611aae23937f5610665c4bd316e15c425 diff --git a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md index 56dcaf8b43..3d13e3fb51 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md +++ b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md @@ -8,15 +8,17 @@ English | [中文](2026-07-23-acp-automation-only-protocol.zh.md) The ACP bridge had become a second interactive product UI. It translated durable events into editor cards, terminal metadata, diffs, plans, titles, reasoning, commands, modes, model and permission pickers, session navigation, and human elicitation. Those responsibilities duplicated the TUI and the Web client while coupling an automation transport to UI services, persistence queries, presentation policy, and editor-specific conventions. -ACP still has one useful role: another agent or automated controller can start a harness process, create an isolated session, send text, receive the committed answer, cancel work, and answer a permission request. The out-of-process ACP subagent backend depends on that standard protocol boundary. +ACP still has one useful role: another agent or automated controller can start a harness process, create an isolated session, send text or a narrowly supported inline image, receive the committed text/image answer, cancel work, and answer a permission request. The out-of-process ACP subagent backend depends on that standard protocol boundary. The snapshot suite complicates removal. Most ACP scenarios exercise the assembled agent backend rather than ACP presentation, so deleting the suite with the editor bridge would discard broad keyless behavioral coverage. ## Decision -`@deepseek-ai/dsh-acp` is an automation transport under [`packages/acp/acp`](../../../../packages/acp/acp/README.md), outside the `ui` package group. Its public protocol is intentionally small: version negotiation, fresh text sessions with one in-flight prompt each, committed assistant text updates, per-session cancellation, concurrent sessions, and connection-owned teardown. Prompts carry the spec-required baseline only — text plus resource links flattened to bracketed textual references; the bridge rejects additional directories, MCP servers, beyond-baseline prompt content (image, audio, embedded resources), empty prompts, unknown sessions, and overlapping prompts. +`@deepseek-ai/dsh-acp` is an automation transport under [`packages/acp/acp`](../../../../packages/acp/acp/README.md), outside the `ui` package group. Its public protocol is intentionally small: version negotiation, fresh sessions with one in-flight prompt each, committed assistant text/image updates, per-session cancellation, concurrent sessions, and connection-owned teardown. Prompts preserve text and supported raster images in wire order, while resource links flatten to bracketed textual references; the bridge rejects additional directories, MCP servers, audio, embedded resources, malformed or empty prompts, unknown sessions, and overlapping prompts. -The bridge emits only committed `assistant/message` text. Reasoning, raw chunks, tool activity, todos, plans, titles, retry markers, terminal metadata, diffs, locations, and resource links remain in the durable session log or in UI-specific transports. It does not provide session load/list/delete, commands, modes, configuration selectors, model switching, plan review, or human elicitation. +Image capability is truthful rather than structural: `initialize` advertises it only when a durable attachment store exists and the configured exact provider/model resolves with explicit image input. Each image prompt rechecks the session's latest exact route, strictly decodes every block, and delegates the complete batch to `AttachmentStore.saveImages()` before publishing the user event. Cancellation reserves and aborts the admission slot before any asynchronous work, waits for already-started writes to quiesce before the prompt settles, and never publishes a late message; a completed content-addressed write may remain unreachable because destructive rollback is not valid for a deduplicated store. + +The bridge emits only committed `assistant/message` text and images. A per-session promise chain preserves block and message order while assistant image references are asynchronously re-read and integrity-verified for ACP base64 delivery; a missing or corrupt object fails prompt delivery instead of becoming a placeholder. Reasoning, raw chunks, tool activity, todos, plans, titles, retry markers, terminal metadata, diffs, locations, and resource links remain in the durable session log or in UI-specific transports. It does not provide session load/list/delete, commands, modes, configuration selectors, model switching, plan review, or human elicitation. One-shot `session/request_permission` remains. It is a machine policy channel for bridge-owned agents, not a human approval UI: the answerer accepts only an exact agent object in the bridge's live session map, delegates foreign or call-less requests, and maps failed RPCs to the fail-closed unavailable outcome. The client chooses allow once, reject once, or cancel, and the bridge never turns that response into a durable grant. Asking policy stays in the approval seam and its producers; [`dsh-subagent-acp`](../../../../packages/subagent/subagent-acp/README.md) uses this channel programmatically. @@ -24,13 +26,13 @@ The app composition contains the agent spine, persistence, checkpoint policy, an The transport programs interface-level agent, session, and approval services rather than the concrete agent loop. Tool execution stays inside the harness; ACP never delegates shell execution to an editor. stdout carries framed JSON-RPC only, so the app mounts no stdout logger and the bridge does not monkey-patch process output. -Disconnect and plugin disposal share one memoized quiescence boundary. Both successful and failed transport closure settle pending prompts as cancelled, dispose every bridge-owned agent, and await loop and session cleanup. A create that loses the close race disposes its unpublished handle. +Disconnect and plugin disposal share one memoized quiescence boundary. Both successful and failed transport closure cancel prompt admission and agents, drain ordered output, settle pending prompts as cancelled, dispose every bridge-owned agent, and await loop and session cleanup. A create that loses the close race disposes its unpublished handle. ## Snapshot boundary The ACP snapshot suite still boots the assembled ACP example and retains scenarios that pin backend behavior. Only scenarios driven through deleted UI methods leave the suite; semantic-checkpoint recovery runs through the headless `stream-json` example because ACP no longer loads sessions. -Protocol and lifecycle tests pin stop-reason and prompt codecs, version negotiation, fresh-session creation, text and resource-link flattening, rejection of empty or unsupported prompts, exact-agent permission ownership, multi-session isolation, prompt settlement, per-session cancellation, failed transport closure, ACP-only reload cleanup, and teardown quiescence. Built and real-stdio smokes reject stray stdout. The `session/new` branch that loses a real stdio close race remains coverage-exempt because the in-memory transport cannot reproduce that ordering; it disposes the unpublished handle, while the surrounding disposal tests pin the no-orphan invariant. +Protocol and lifecycle tests pin stop-reason codecs, version negotiation, truthful image capability, fresh-session creation, ordered text/image admission, resource-link flattening, all-member validation before writes, absence of inline base64 in durable events, rejection of empty or unsupported prompts, exact-agent permission ownership, multi-session isolation, prompt settlement after ordered output, verified assistant-image delivery, cancellation during admission without a late followup, failed transport closure, ACP-only reload cleanup, and teardown quiescence. An assembled keyless snapshot sends a real inline PNG through the runnable ACP example and pins only its durable reference in the session log. Built and real-stdio smokes reject stray stdout. The `session/new` branch that loses a real stdio close race remains coverage-exempt because the in-memory transport cannot reproduce that ordering; it disposes the unpublished handle, while the surrounding disposal tests pin the no-orphan invariant. ## Alternatives considered @@ -44,10 +46,16 @@ Protocol and lifecycle tests pin stop-reason and prompt codecs, version negotiat **Delete the ACP snapshot suite or migrate every scenario in this change.** Rejected because most scenarios test the backend and remain valuable, while a full harness migration is an independent testing change. Only scenarios whose driver was a deleted UI method leave this suite. +**Advertise image support whenever the ACP SDK has an image block.** Rejected because protocol vocabulary does not prove this deployment can persist bytes or that the configured exact route accepts visual input. Unknown capability is false at initialization; prompt admission rechecks the live route. + +**Flatten inline and assistant images to markers or persist ACP base64 in session events.** Rejected because markers silently lose model/user intent and base64 makes durable logs the binary store. ACP translates between its wire block and the existing durable `ImageBlock` reference at the transport boundary. + +**Create a generic RichContent service for ACP, MCP, and Web.** Rejected because core `ContentBlock` plus the attachment seam already own the shared contract. Each front door keeps only protocol parsing, capability proof, and lifecycle orchestration; shared batch limits and image validation stay in `AttachmentStore.saveImages()`. + ## Consequences ACP has a narrow contract suitable for agents and automation, while TUI and Web own human interaction and presentation. The package has fewer injected services, dependencies, protocol branches, and lifecycle states, and it no longer claims compatibility as a general editor entry point. -Automation clients receive complete committed text rather than token deltas or structured tool UI. They inspect durable logs or another API when they need reasoning, tool traces, titles, or richer state. Fresh-session-only operation also means callers that need durable browsing or resume use a host API rather than ACP. +Automation clients receive complete committed text/images rather than token deltas or structured tool UI. They inspect durable logs or another API when they need reasoning, tool traces, titles, or richer state. Fresh-session-only operation also means callers that need durable browsing or resume use a host API rather than ACP. Backend snapshot coverage therefore remains transport-coupled to ACP even though that transport is incidental to the behavior under test. diff --git a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.zh.md b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.zh.md index a34e179a1d..224c1bd611 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.zh.md +++ b/.agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.zh.md @@ -8,15 +8,17 @@ Status: implemented ACP(Agent Client Protocol)桥接层已经变成第二套交互式产品 UI。它将持久事件转换为编辑器卡片、终端元数据、diff、计划、标题、推理(reasoning)、命令、模式、模型和权限选择器、会话导航以及面向人类的询问。这些职责与 TUI 和 Web 客户端重复,同时将自动化传输层与 UI 服务、持久化查询、展示策略和编辑器特定约定耦合在一起。 -ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控制器可以启动 harness 进程、创建隔离会话、发送文本、接收已提交的回答、取消工作并回答权限请求。跨进程 ACP subagent 后端依赖这个标准协议边界。 +ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控制器可以启动 harness 进程、创建隔离会话、发送文本或范围狭窄的受支持内联图片、接收已提交的文本/图片回答、取消工作并回答权限请求。跨进程 ACP subagent 后端依赖这个标准协议边界。 快照套件使移除工作更复杂。大多数 ACP 场景测试的是组装后的 agent 后端,而不是 ACP 展示层;如果随编辑器桥接层一起删除整个套件,就会丢失大量无密钥行为测试。 ## 决策 -`@deepseek-ai/dsh-acp` 是位于 [`packages/acp/acp`](../../../../packages/acp/acp/README.md) 下、独立于 `ui` 包组的自动化传输层。其公开协议特意保持精简:版本协商、全新文本会话(每个会话最多允许一个进行中的提示词)、已提交的助手文本更新、按会话取消、并发会话,以及由连接负责的资源清理。提示词只承载规范要求的基线内容——文本,加上被展平为方括号文本引用的资源链接;桥接层会拒绝附加目录、MCP 服务器、超出基线的提示词内容(图片、音频、内嵌资源)、空提示词、未知会话和重叠提示词。 +`@deepseek-ai/dsh-acp` 是位于 [`packages/acp/acp`](../../../../packages/acp/acp/README.md) 下、独立于 `ui` 包组的自动化传输层。其公开协议特意保持精简:版本协商、全新会话(每个会话最多允许一个进行中的提示词)、已提交的助手文本/图片更新、按会话取消、并发会话,以及由连接负责的资源清理。提示词按协议顺序保留文本与受支持光栅图片,资源链接则展平为方括号文本引用;桥接层会拒绝附加目录、MCP 服务器、音频、嵌入资源、格式错误或空提示词、未知会话和重叠提示词。 -桥接层只发出已提交的 `assistant/message` 文本。推理、原始分片、工具活动、待办事项、计划、标题、重试标记、终端元数据、diff、位置和资源链接仍保留在持久会话日志或 UI 专用传输层中。它不提供会话加载、列出与删除、命令、模式、配置选择器、模型切换、plan 评审或面向人类的询问。 +图片能力必须真实,而不能只看结构:只有持久附件存储存在,且配置的确切提供方/模型解析后明确支持图片输入时,`initialize` 才会公布该能力。每个图片提示词都会重新检查会话的最新确切路由、严格解码全部块,并在发布用户事件前把完整批次委托给 `AttachmentStore.saveImages()`。取消会在任何异步工作前预留并中止准入槽位,使提示词在已经启动的写入停稳后才结算,而且绝不发布迟到消息;已经完成的内容寻址写入可能保持不可达,因为对去重存储执行破坏性回滚并不正确。 + +桥接层只发出已提交的 `assistant/message` 文本与图片。每个会话使用一条 Promise 链,在异步重新读取并校验助手图片引用、将其转换为 ACP base64 交付时保持块与消息顺序;对象缺失或损坏会使提示词交付失败,而不是变成占位符。推理、原始分片、工具活动、待办事项、计划、标题、重试标记、终端元数据、diff、位置和资源链接仍保留在持久会话日志或 UI 专用传输层中。它不提供会话加载、列出与删除、命令、模式、配置选择器、模型切换、plan 评审或面向人类的询问。 保留一次性 `session/request_permission`。它是为桥接层拥有的 agent 提供的机器策略通道,而不是面向人类的审批 UI:应答者只接受桥接层当前会话映射中登记的同一 agent 对象;外部请求或缺少调用标识的请求会继续委派;RPC 失败则映射为拒绝请求的 `unavailable` 结果。客户端可选择允许一次、拒绝一次或取消,桥接层绝不会将该响应转换为持久授权。询问策略仍归审批 seam 及其生产者所有;[`dsh-subagent-acp`](../../../../packages/subagent/subagent-acp/README.md) 会以程序化方式使用该通道。 @@ -24,13 +26,13 @@ ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控 传输层调用 agent、会话和审批的接口服务,而不依赖具体的 agent loop。工具执行仍留在 harness 内;ACP 绝不会把 shell 执行委派给编辑器。stdout 只承载分帧 JSON-RPC,因此 app 不挂载 stdout logger,桥接层也不会 monkey-patch 进程输出。 -断开连接与插件 dispose(资源释放)共享同一个经记忆化的完全停稳边界。传输关闭无论成功还是失败,都会将待处理提示词以已取消状态结算,dispose 每个由桥接层拥有的 agent,并等待循环和会话清理完成。创建流程如果在与关闭的竞态中落败,就会 dispose 其尚未发布的 handle。 +断开连接与插件 dispose(资源释放)共享同一个经记忆化的完全停稳边界。传输关闭无论成功还是失败,都会取消提示词准入和 agent、排空有序输出、将待处理提示词以已取消状态结算、dispose 每个由桥接层拥有的 agent,并等待循环和会话清理完成。创建流程如果在与关闭的竞态中落败,就会 dispose 其尚未发布的 handle。 ## 快照边界 ACP 快照套件仍会启动组装后的 ACP 示例,并保留用于锁定后端行为的场景。从该套件移出的只有通过已删除的 UI 方法驱动的场景;由于 ACP 不再加载会话,语义检查点恢复通过 headless `stream-json` 示例执行。 -协议与生命周期测试会锁定停止原因编解码器和提示词编解码器、版本协商、新会话创建、文本与资源链接展平、拒绝空提示词或不受支持的提示词、基于同一 agent 对象的权限归属、多会话隔离、提示词结算、按会话取消、传输关闭失败、ACP 专属重载清理,以及拆卸完全停稳。构建产物冒烟测试与真实 stdio 冒烟测试会拒绝混入 stdout 的额外输出。`session/new` 中在真实 stdio 关闭竞态中落败的分支仍豁免覆盖率要求,因为内存传输层无法复现这一顺序;该分支会 dispose 尚未发布的 handle,而周边 dispose 测试会锁定无遗留资源不变式。 +协议与生命周期测试会锁定停止原因编解码器、版本协商、真实图片能力、新会话创建、有序文本/图片准入、资源链接展平、写入前校验全部成员、持久事件中不含内联 base64、拒绝空提示词或不受支持的提示词、基于同一 agent 对象的权限归属、多会话隔离、在有序输出后结算提示词、经过校验的助手图片交付、准入期间取消且不产生迟到 followup、传输关闭失败、ACP 专属重载清理,以及拆卸完全停稳。组装后的无密钥快照通过可运行 ACP 示例发送一张真实内联 PNG,并在会话日志中只固定其持久引用。构建产物冒烟测试与真实 stdio 冒烟测试会拒绝混入 stdout 的额外输出。`session/new` 中在真实 stdio 关闭竞态中落败的分支仍豁免覆盖率要求,因为内存传输层无法复现这一顺序;该分支会 dispose 尚未发布的 handle,而周边 dispose 测试会锁定无遗留资源不变式。 ## 考虑过的替代方案 @@ -44,10 +46,16 @@ ACP 快照套件仍会启动组装后的 ACP 示例,并保留用于锁定后 **删除 ACP 快照套件,或在本次变更中迁移每个场景。** 不予采用,因为大多数场景测试后端且仍有价值,而完整的 harness 迁移是一项独立的测试变更。只有通过已删除的 UI 方法驱动的场景才离开该套件。 +**只要 ACP SDK 具有图片块就公布图片支持。** 不予采用,因为协议词汇不能证明当前部署可以持久化字节,也不能证明配置的确切路由接受视觉输入。初始化时能力未知即为 false;提示词准入会重新检查实时路由。 + +**把内联图片和助手图片展平为标记,或把 ACP base64 持久化进会话事件。** 不予采用,因为标记会静默丢失模型/用户意图,base64 则会让持久日志变成二进制存储。ACP 在传输边界把自身协议块与现有持久 `ImageBlock` 引用相互转换。 + +**为 ACP、MCP 和 Web 创建通用 RichContent 服务。** 不予采用,因为核心 `ContentBlock` 与附件 seam 已经拥有共享契约。每个入口只保留协议解析、能力证明与生命周期编排;共享批次限制和图片校验留在 `AttachmentStore.saveImages()` 中。 + ## 结果 ACP 具有适合 agent 与自动化的精简约定,而 TUI 和 Web 拥有面向人类的交互与展示。该包注入的服务、依赖、协议分支和生命周期状态更少,也不再将自身定位为通用编辑器入口。 -自动化客户端收到完整的已提交文本,而不是 token 增量或结构化工具 UI。当它们需要推理、工具跟踪信息、标题或更丰富的状态时,需要查看持久日志或其他 API。只支持全新会话也意味着,需要浏览持久会话或恢复会话的调用方必须使用 host API,而不是 ACP。 +自动化客户端收到完整的已提交文本/图片,而不是 token 增量或结构化工具 UI。当它们需要推理、工具跟踪信息、标题或更丰富的状态时,需要查看持久日志或其他 API。只支持全新会话也意味着,需要浏览持久会话或恢复会话的调用方必须使用 host API,而不是 ACP。 因此,后端快照测试仍与 ACP 传输层耦合,尽管对于受测行为而言,该传输层只是附带因素。 diff --git a/docs/config-catalog.i18n.yaml b/docs/config-catalog.i18n.yaml index ceba64b2c0..38167f6f59 100644 --- a/docs/config-catalog.i18n.yaml +++ b/docs/config-catalog.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/config-catalog.md -config-catalog.md: 369490ec41480b8c46355f0edc3eb97f2f0c76cb -config-catalog.zh.md: a735d7021d44dda3cffa49c31430249f70431720 +config-catalog.md: 27ed96659fd23bd33495a632aa0fd552f44e0163 +config-catalog.zh.md: 01a74bd8f06dcce5b5281f12796b151246b73f35 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index 369490ec41..27ed96659f 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -27,7 +27,7 @@ export interface AcpConfig { Depends on: `Stream` (`@agentclientprotocol/sdk`) -Source: [`packages/acp/acp/src/index.ts:70`](../packages/acp/acp/src/index.ts) +Source: [`packages/acp/acp/src/index.ts:71`](../packages/acp/acp/src/index.ts) ## `@deepseek-ai/dsh-acp-demo` diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index a735d7021d..01a74bd8f0 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -29,7 +29,7 @@ export interface AcpConfig { 依赖:`Stream`(`@agentclientprotocol/sdk`) -来源:[`packages/acp/acp/src/index.ts:70`](../packages/acp/acp/src/index.ts) +来源:[`packages/acp/acp/src/index.ts:71`](../packages/acp/acp/src/index.ts) ## `@deepseek-ai/dsh-acp-demo` diff --git a/examples/acp-agent/code-mode-image.cordis.snapshot.yml b/examples/acp-agent/code-mode-image.cordis.snapshot.yml new file mode 100644 index 0000000000..42dda231f7 --- /dev/null +++ b/examples/acp-agent/code-mode-image.cordis.snapshot.yml @@ -0,0 +1,42 @@ +# Keyless replay combines Code Mode with the durable image store and an exact +# image-capable replay route. The scenario generates its tiny PNG inside the +# run_code program, then exercises read_image as a nested dispatch. +- id: base + name: '@deepseek-ai/cordis-plugin-include' + config: + path: ./cordis.yml + patches: + - id: llm-deepseek + name: '@deepseek-ai/dsh-llm-deepseek' + disabled: true + - id: acp-agent + name: '@deepseek-ai/dsh-acp-demo' + config: + provider: deepseek-official + model: deepseek-v4-flash + persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' + persistenceCompression: none + workspaceContext: + maxBytes: 65536 + tools: + mode: code + persona: | + You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. + + Verify your work by running the code or tests. Keep answers brief and factual. + - insert: + - id: attachment-local + name: '@deepseek-ai/dsh-attachment-local' + - id: code-runtime + name: '@deepseek-ai/dsh-code-runtime-worker' + - id: llm-replay + name: '@deepseek-ai/dsh-llm-replay' + config: + providers: + - id: deepseek-official + name: DeepSeek + models: + - id: deepseek-v4-flash + inputModalities: [text, image] + - id: deepseek-v4-pro + inputModalities: [text] diff --git a/examples/acp-agent/code-mode-image.cordis.yml b/examples/acp-agent/code-mode-image.cordis.yml new file mode 100644 index 0000000000..e51984b354 --- /dev/null +++ b/examples/acp-agent/code-mode-image.cordis.yml @@ -0,0 +1,29 @@ +# Code Mode image overlay: mounts the worker runtime and durable attachment +# store so a nested read_image result can cross the generic rich-result bridge. +# The authored snapshot is replay-only; the live config retains the ordinary +# exact provider route for manual use. +- id: base + name: '@deepseek-ai/cordis-plugin-include' + config: + path: ./cordis.yml + patches: + - id: acp-agent + name: '@deepseek-ai/dsh-acp-demo' + config: + provider: deepseek-official + model: deepseek-v4-pro + persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions' + persistenceCompression: !!js "process.env.DSH_SNAPSHOT === undefined ? 'zstd' : 'none'" + workspaceContext: + maxBytes: 65536 + tools: + mode: code + persona: | + You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. + + Verify your work by running the code or tests. Keep answers brief and factual. + - insert: + - id: attachment-local + name: '@deepseek-ai/dsh-attachment-local' + - id: code-runtime + name: '@deepseek-ai/dsh-code-runtime-worker' diff --git a/examples/acp-agent/tests/acp.snapshot.ts b/examples/acp-agent/tests/acp.snapshot.ts index 736b17ed04..357472dd77 100644 --- a/examples/acp-agent/tests/acp.snapshot.ts +++ b/examples/acp-agent/tests/acp.snapshot.ts @@ -32,6 +32,7 @@ const AGENT = { // The Code Mode overlay configs (include-patched variants of cordis.yml; the // replay swap resolves each one's sibling `*cordis.snapshot.yml`). const CODE_MODE_CONFIG = fileURLToPath(new URL('../code-mode.cordis.yml', import.meta.url)) +const CODE_MODE_IMAGE_CONFIG = fileURLToPath(new URL('../code-mode-image.cordis.yml', import.meta.url)) const CODE_MODE_WORKSPACE_CONTEXT_CONFIG = fileURLToPath(new URL('../code-mode-workspace-context.cordis.yml', import.meta.url)) const BOTH_MODE_CONFIG = fileURLToPath(new URL('../both-mode.cordis.yml', import.meta.url)) const WORKSPACE_CONTEXT_CONFIG = fileURLToPath(new URL('../workspace-context.cordis.yml', import.meta.url)) @@ -212,6 +213,13 @@ const SCENARIOS: Scenario[] = [ headerClass: 'image', configPath: IMAGE_TEXT_ROUTE_CONFIG, }, + { + name: 'inline-image-prompt', + hasModelTurn: true, + recorded: false, + headerClass: 'image', + configPath: IMAGE_CONFIG, + }, { name: 'pty-tools', hasModelTurn: true, @@ -539,6 +547,16 @@ const SCENARIOS: Scenario[] = [ // tools:sdk section rides in the prompt, and the program's tool calls land as // tool/code-dispatch events. Each overlay composes and pins its own header class. { name: 'code-mode-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'code', configPath: CODE_MODE_CONFIG }, + { + name: 'code-mode-read-image', + hasModelTurn: true, + recorded: false, + pinsHeader: true, + headerClass: 'code-image', + toolSchemasSource: 'code-mode-turn', + configPath: CODE_MODE_IMAGE_CONFIG, + posixOnly: true, + }, // A nested fs dispatch inside run_code discovers workspace instructions. The // projection enters the inbox after the outer result and becomes model-visible // on the following step, retaining workspace provenance end to end. diff --git a/examples/acp-agent/tests/snapshots/code-mode-read-image/input.json b/examples/acp-agent/tests/snapshots/code-mode-read-image/input.json new file mode 100644 index 0000000000..f04f56a70e --- /dev/null +++ b/examples/acp-agent/tests/snapshots/code-mode-read-image/input.json @@ -0,0 +1,14 @@ +{ + "steps": [ + { + "op": "initialize" + }, + { + "op": "newSession" + }, + { + "op": "prompt", + "text": "Using ONE run_code program, create a one-pixel PNG with Node.js, call read_image on it, then reply with exactly the single word DONE." + } + ] +} diff --git a/examples/acp-agent/tests/snapshots/code-mode-read-image/session.jsonl b/examples/acp-agent/tests/snapshots/code-mode-read-image/session.jsonl new file mode 100644 index 0000000000..cb63f29c34 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/code-mode-read-image/session.jsonl @@ -0,0 +1,33 @@ +{"type":"session","version":0,"id":"44444444-4444-4444-8444-444444444444","createdAt":1783952000000,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"agent/inbox/spliced","seq":0,"time":1786431644501,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Using ONE run_code program, create a one-pixel PNG with Node.js, call read_image on it, then reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"08e67dbb-9432-4fe4-b7da-4483998c0a31"}]}} +{"type":"turn/start","seq":1,"time":1786431644502,"data":{"turn":1}} +{"type":"agent/inbox/spliced","seq":2,"time":1786431644502,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","seq":3,"time":1786431644557,"data":{"turn":1,"step":1}} +{"type":"user/message","seq":4,"time":1786431644558,"data":{"content":[{"type":"text","text":"Using ONE run_code program, create a one-pixel PNG with Node.js, call read_image on it, then reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"08e67dbb-9432-4fe4-b7da-4483998c0a31"},"surfaceOp":"append"} +{"type":"user/message","seq":5,"time":1786431644558,"data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"99b9db8d-e4ec-4ea9-b5e2-1e4c0ff6354b"},"surfaceOp":"append"} +{"type":"session/title","seq":6,"time":1786431644558,"data":{"title":"Using ONE run_code program, create","messageSeqs":[4],"source":{"kind":"fallback"}}} +{"type":"request/header","seq":7,"time":1786431644559,"data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/context","seq":8,"time":1786431644560,"data":{"provider":"deepseek-official","model":"deepseek-v4-flash"}} +{"type":"assistant/chunk","seq":9,"time":1783952000009,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"tool-call"}}} +{"type":"assistant/chunk","seq":10,"time":1786431644571,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"tool-call","id":"code-image-call","name":"run_code","arguments":"{\"code\":\"const bytes = [137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130];\\nawait tools.bash({ command: \\\"node -e \\\\\\\"require('node:fs').writeFileSync('red.png',Buffer.from([137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130]));\\\\\\\"\\\", description: \\\"Create a one pixel PNG\\\" });\\nconst image = await tools.read_image({ file_path: \\\"red.png\\\" });\\nreturn image.path;\",\"description\":\"Create and inspect one image\"}"}}}} +{"type":"assistant/chunk","seq":11,"time":1786431644572,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":12,"time":1786431644572,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}} +{"type":"assistant/message","seq":13,"time":1786431644572,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"code-image-call","name":"run_code","arguments":"{\"code\":\"const bytes = [137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130];\\nawait tools.bash({ command: \\\"node -e \\\\\\\"require('node:fs').writeFileSync('red.png',Buffer.from([137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130]));\\\\\\\"\\\", description: \\\"Create a one pixel PNG\\\" });\\nconst image = await tools.read_image({ file_path: \\\"red.png\\\" });\\nreturn image.path;\",\"description\":\"Create and inspect one image\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"644382c5-5a05-4bda-b8dc-b9195d6a7d8b"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[9,10,11,12],"surfaceOp":"append"} +{"type":"tool/call","seq":14,"time":1786431644573,"data":{"turn":1,"step":1,"callId":"code-image-call","name":"run_code","arguments":"{\"code\":\"const bytes = [137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130];\\nawait tools.bash({ command: \\\"node -e \\\\\\\"require('node:fs').writeFileSync('red.png',Buffer.from([137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130]));\\\\\\\"\\\", description: \\\"Create a one pixel PNG\\\" });\\nconst image = await tools.read_image({ file_path: \\\"red.png\\\" });\\nreturn image.path;\",\"description\":\"Create and inspect one image\"}"}} +{"type":"tool/code-dispatch-start","seq":15,"time":1786431644697,"data":{"rootCallId":"code-image-call","parentCallId":"code-image-call","subCallId":"code-image-call:code:1","name":"bash","arguments":{"command":"node -e \"require('node:fs').writeFileSync('red.png',Buffer.from([137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130]));\"","description":"Create a one pixel PNG"}}} +{"type":"tool/code-dispatch","seq":16,"time":1786431644828,"data":{"rootCallId":"code-image-call","parentCallId":"code-image-call","subCallId":"code-image-call:code:1","name":"bash","arguments":{"command":"node -e \"require('node:fs').writeFileSync('red.png',Buffer.from([137,80,78,71,13,10,26,10,0,0,0,13,73,72,68,82,0,0,0,1,0,0,0,1,8,2,0,0,0,144,119,83,222,0,0,0,12,73,68,65,84,120,156,99,248,207,192,0,0,3,1,1,0,201,254,146,239,0,0,0,0,73,69,78,68,174,66,96,130]));\"","description":"Create a one pixel PNG"},"isError":false,"content":[{"type":"text","text":"(no output)"}]}} +{"type":"tool/code-dispatch-start","seq":17,"time":1786431644829,"data":{"rootCallId":"code-image-call","parentCallId":"code-image-call","subCallId":"code-image-call:code:2","name":"read_image","arguments":{"file_path":"red.png"}}} +{"type":"tool/code-dispatch","seq":18,"time":1786431644871,"data":{"rootCallId":"code-image-call","parentCallId":"code-image-call","subCallId":"code-image-call:code:2","name":"read_image","arguments":{"file_path":"red.png"},"isError":false,"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}]}} +{"type":"tool/result","seq":19,"time":1786431644874,"data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"code-image-call"},"content":[{"type":"tool-result","toolCallId":"code-image-call","content":[{"type":"text","text":"{{cwd}}/red.png"}],"isError":false}],"role":"user","id":"73e999fa-4aab-4609-970d-4c675e3557f1"}},"sourceEventSeqs":[14],"surfaceOp":"append"} +{"type":"agent/inbox/spliced","seq":20,"time":1786431644874,"data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"99bca54a-c323-4df8-8695-7ef17d02dd65"}]}} +{"type":"step/end","seq":21,"time":1786431644874,"data":{"turn":1,"step":1}} +{"type":"agent/inbox/spliced","seq":22,"time":1786431644874,"data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","seq":23,"time":1786431644884,"data":{"turn":1,"step":2}} +{"type":"user/message","seq":24,"time":1786431644885,"data":{"content":[{"type":"text","text":"{{cwd}}/red.png\nimage\n\nimage/png image, 1x1 px, 69 bytes\n"},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","bytes":69,"width":1,"height":1,"name":"red.png"}}],"source":{"kind":"plugin","plugin":"tools-code-mode"},"role":"user","id":"99bca54a-c323-4df8-8695-7ef17d02dd65"},"surfaceOp":"append"} +{"type":"assistant/chunk","seq":25,"time":1786431644889,"data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","seq":26,"time":1786431644889,"data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","seq":27,"time":1786431644890,"data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":28,"time":1786431644890,"data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":29,"time":1786431644890,"data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"a6da60ea-d420-432b-ba00-9b99af045110"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[25,26,27,28],"surfaceOp":"append"} +{"type":"step/end","seq":30,"time":1786431644890,"data":{"turn":1,"step":2}} +{"type":"turn/end","seq":31,"time":1786431644890,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/code-mode-read-image/stdout.expected.jsonl b/examples/acp-agent/tests/snapshots/code-mode-read-image/stdout.expected.jsonl new file mode 100644 index 0000000000..4f0fb2e442 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/code-mode-read-image/stdout.expected.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"DONE"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/code-mode-read-image/system-prompt.expected.md b/examples/acp-agent/tests/snapshots/code-mode-read-image/system-prompt.expected.md new file mode 100644 index 0000000000..3dde6f9f77 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/code-mode-read-image/system-prompt.expected.md @@ -0,0 +1,457 @@ +You are an AI agent powered by the DeepSeek Harness SDK. + +You are a coding assistant powered by the deepseek-v4-flash model. Your working directory is {{cwd}}. + +Verify your work by running the code or tests. Keep answers brief and factual. + + +Use the read tool — not shell commands like cat — to inspect text files. Results include line numbers. Use offset and limit to continue reading large files. + +Use the write tool to create files or completely replace file contents. Existing files are overwritten, so read an existing file first (the default fs-policy requires it) and prefer edit for targeted changes. + +Use the edit tool for targeted changes to existing UTF-8 text files. It replaces literal old_string with new_string; by default old_string must appear exactly once. If old_string appears multiple times, provide a more specific old_string or set replace_all to true. Read the file first (the default fs-policy requires it), unless you just created or edited it in this session. + +Check the [exit code: N] marker on every bash result; investigate failures before moving on. + +Track every background task id you start. You are notified in-session when a task finishes — do not busy-poll or sleep on one; keep working on independent steps and do not duplicate a running task's work. Before giving a final answer, collect every still-relevant task with task_output (set wait: true only when you are genuinely blocked on it), and task_kill tasks that stopped mattering. + +Use goal tools for one long-running completion objective in the current session. create_goal may infer goal intent from a direct human request in any language; do not create a goal for routine single-turn work. Call get_goal before update_goal and copy its exact goal_id and revision. After session resume or fork, an active goal is disarmed: when a human asks to continue or resume in any wording or language, use update_goal action resume to rearm it. Mark complete only when the objective is actually achieved. Mark blocked only after the same blocking condition persists for at least 3 consecutive goal rounds, and report that concrete condition in blocked_reason; difficulty, uncertainty, or useful remaining work is not blocked. + +Use the workflow tool ONLY when the user explicitly asks for a workflow or for large multi-agent orchestration: you write a JavaScript script (the tool description documents the exact format) that fans work out across many subagents with phases and structured results. For one or two delegations, prefer plain subagent calls. + +Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Completion and blockers are worker reports, not independent evaluation. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out. + +## Writing code for run_code + +Pass `run_code` the body of an async TypeScript function (erasable syntax only — no `enum` or namespaces; type annotations are advisory, the code runs type-stripped). Inside the program: + +- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON. +- A FAILED tool call rejects with `ToolCallError`, whose `toolName` identifies the failed tool and whose `message` is human-readable — `try/catch` it to handle and continue. +- Independent read-only calls MAY overlap under `Promise.all` (safe calls run concurrently; mutating calls run alone, in submission order). Sequence dependent work with `await`. +- Emit results with `return` and/or `console.log(...)`. Only what you print or return is program output. A successful tool result containing an image is attached after the run so you can inspect it on the next step; every other intermediate result stays out of the conversation, so extract just what you need. + +The available tools: + +```ts +type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue } + +interface ToolArgsMap { + /** Execute a bash command (`bash -c`) and return its stdout/stderr. Each call runs in a fresh shell: no state (cwd, variables, functions) persists between calls — pass `workdir` instead of using `cd`. Non-zero exits are reported as `[exit code: N]`. Current harness environment facts are exposed through managed `$DSH_*` variables; inspect them when needed. Commands may run under a file sandbox; a blocked file operation is reported as `[sandbox: file access denied under mode]` — a policy denial, not a bug in the command; do not retry another way. Long output is truncated to its tail; the full output is saved to a file whose path is reported when available. Set `run_in_background: true` for long-running commands: the call returns a task id immediately; read its output with `task_output` and stop it with `task_kill`. Attempting a command the sandbox may deny is safe and expected: run it and read the marker rather than assuming the denial. When a command is denied and a wider mode would let it succeed, escalate immediately in the same turn — the one sanctioned exception to a denial: retry the exact same command once with `sandbox_permissions` (the narrowest wider mode that suffices) plus a one-sentence `justification`. Do not detour through chat to ask permission first — the approval prompt raised by that retry is how the user consents. If the session states approval prompts are disabled, there is no exception: a denial is final — do not set `sandbox_permissions`. Never escalate speculatively: ground the request in a real denial — normally the one this command just hit; escalating up front is fine only when this session already denied the same access. A rejected escalation is final for that command — stop and explain, never work around it — but it does not forbid attempting or escalating other commands later. */ + bash: { + /** The bash command to execute. */ + command: string; + /** Clear, concise description of what this command does in active voice, 5-10 words (shown in the UI). Examples: "ls" → "List files in current directory"; "git status" → "Show working tree status"; "npm install" → "Install package dependencies". */ + description: string; + /** Timeout in milliseconds. The executor applies its configured default and cap, and kills the command on expiry. */ + timeoutMs?: number; + /** Working directory for this command. Defaults to the session workspace; a relative path is resolved against it. */ + workdir?: string; + /** Run in the background and return a task id immediately (collect with task_output, stop with task_kill). No timeout applies. */ + run_in_background?: boolean; + /** The wider sandbox mode this command needs. Only valid as a one-shot retry of a command the sandbox just denied; requires justification and user approval. */ + sandbox_permissions?: "workspace-write" | "danger-full-access"; + /** Required with sandbox_permissions: one sentence for the user explaining why this exact command needs the wider access. */ + justification?: string; + } & Record; + /** Create one persisted same-session completion goal when the current direct human request is a long-running objective that should continue across autonomous goal rounds. You may infer that intent without requiring the user to say "create a goal". Do not use this for trivial single-turn work. Execution rejects non-human and subagent authority. */ + create_goal: { + /** The concrete completion objective inferred from the direct human request. */ + objective: string; + /** Optional positive safe-integer limit on automatic continuation rounds. */ + max_goal_rounds?: number; + } & Record; + /** Edit an existing UTF-8 text file by replacing literal text. */ + edit: { + /** Path to edit, resolved by the filesystem backend. */ + file_path: string; + /** Literal text to replace. Must match exactly. */ + old_string: string; + /** Literal replacement text. Use an empty string to delete the match. */ + new_string: string; + /** Replace all matches. Defaults to false; when false, old_string must appear exactly once. */ + replace_all?: boolean; + /** The wider sandbox mode this file operation needs. Only valid as a one-shot retry of an operation the sandbox just denied; requires justification and user approval. */ + sandbox_permissions?: "workspace-write" | "danger-full-access"; + /** Required with sandbox_permissions: one sentence for the user explaining why this exact file operation needs the wider access. */ + justification?: string; + } & Record; + /** Read the current same-session goal, including its exact id/revision, objective, phase, completed continuation rounds, round limit, blocker reason when present, and whether another continuation is armed. Call this before updating a goal. */ + get_goal: Record; + /** Request cancellation of a background agent's current turn by its agent id. The target may be your direct child or a deeper agent created under you. Only the current turn stops: messages already queued for the agent stay parked until a later send_message, agents it started keep running, and the agent itself stays available for follow-ups. This call returns as soon as the stop request is accepted, so the target may keep running briefly; interrupting an agent that already finished is an accepted no-op. */ + interrupt_agent: { + /** The agent id of the running agent to interrupt. */ + agent_id: string; + } & Record; + /** List your continuable background subagents by durable id and label. Status comes from the live registry: running means the agent is working right now, idle means it is loaded but between turns (it may be waiting on agents it started), and complete means it exists only in storage — a direct child remains a `send_message` candidate in every status. The snapshot is not a delivery promise — `send_message` performs the authoritative check and may still fail. Children that could not be read are reported as diagnostics instead of being silently dropped. Scope `descendants` walks the whole tree below you in stable pre-order, annotating each entry with its durable direct-parent session id and depth. You may use `send_message` only for depth-1 entries; deeper entries are candidates for `interrupt_agent` only. */ + list_agents: { + /** children (default) lists direct children only; descendants walks the complete tree below you. */ + scope?: "children" | "descendants"; + } & Record; + /** Run a foreground fresh-agent Ralph loop toward one immutable objective. Use only when the direct human explicitly asks for Ralph or fresh-agent iteration. Each round opens a new child with no parent conversation or prior child session; the shared workspace is long-term memory, and only a bounded structured report crosses rounds. The call returns when a worker reports completion or a concrete blocker, or at the round limit. Ordinary long-running same-session work belongs to goal tools. */ + ralph: { + /** The immutable completion objective for every fresh Ralph round. */ + objective: string; + /** Optional positive safe-integer round cap, bounded by the deployment ceiling. */ + maxRounds?: number; + } & Record; + /** Read a UTF-8 text file and return line-numbered content. */ + read: { + /** Path to read, resolved by the filesystem backend. */ + file_path: string; + /** 1-based first line to return. Defaults to 1. */ + offset?: number; + /** Maximum number of lines to return. Defaults to 2000. */ + limit?: number; + } & Record; + /** Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input. */ + read_image: { + /** Path to the image file, resolved by the filesystem backend. */ + file_path: string; + } & Record; + /** Send a message to a background subagent by its subagent id, continuing the same conversation. It becomes the subagent's next turn: if it is still working, the message waits until its current turn finishes, so it cannot redirect work already underway. This call returns no answer from the subagent — only confirmation that the message was delivered — so use it to give it more work. A failure means the message was NOT delivered. */ + send_message: { + /** The subagent id returned when the background subagent was started. */ + subagent_id: string; + /** The message to deliver to the subagent. */ + message: string; + } & Record; + /** Load the full instructions for an available skill. Call this with the exact skill name from the session skill catalog before acting on a task that names or clearly matches that skill. */ + skill: { + /** The exact skill name from the available skills list. */ + name: string; + } & Record; + /** Delegate a self-contained task to a subagent (a separate agent that works in its own context) and return its final result. Use this to offload focused, independent work — research, a scoped implementation, an analysis — so it does not consume this conversation's context. The subagent runs to completion and you receive only its final answer, not its intermediate steps. Give it a complete, standalone prompt: it does not see this conversation. Set `run_in_background: true` to start a background subagent that keeps its conversation: you receive only its subagent id, never its result, and it works on its own. Use this for work whose result you do not need returned by this call; `send_message` sends it more work. */ + subagent: { + /** A short (3-5 word) description of the delegated task, for display. */ + description: string; + /** The complete, self-contained task for the subagent. It does not share this conversation's context, so include everything it needs. */ + prompt: string; + /** Run as a background subagent that keeps its conversation and return only its subagent id. This call never returns its result; send it more work with send_message. */ + run_in_background?: boolean; + } & Record; + /** Delegate a task to a subagent that inherits this conversation: a child agent seeded with all completed turns so far (it does not see the current in-flight turn), returning only its final result. Use this when the subtask builds on this conversation's context — a follow-up analysis, a review, a continuation — without consuming this conversation's context for the work itself. You receive only its final answer, not its intermediate steps. Set `run_in_background: true` to start a background subagent that keeps its conversation: you receive only its subagent id, never its result, and it works on its own. Use this for work whose result you do not need returned by this call; `send_message` sends it more work. */ + subagent_fork: { + /** A short (3-5 word) description of the delegated task, for display. */ + description: string; + /** The task for the subagent. It already sees this conversation's completed turns, so build on them freely and state only what is new. */ + prompt: string; + /** Run as a background subagent that keeps its conversation and return only its subagent id. This call never returns its result; send it more work with send_message. */ + run_in_background?: boolean; + } & Record; + /** Request cancellation of a running background task by task id. Returns immediately; the task settles as killed once its work actually stops. */ + task_kill: { + /** Task id returned by the tool that started the background work. */ + task_id: string; + /** Optional short reason, recorded in the log and forwarded to the task. */ + reason?: string; + } & Record; + /** List your background tasks (running and finished) with their ids, kinds, and statuses. */ + task_list: Record; + /** Read a background task. Stream tasks return only output since the previous read; final-output tasks return their result after settlement. Every response ends with `[status: ...]`. Reads are non-blocking unless `wait: true`, which waits up to the configured cap. */ + task_output: { + /** Task id returned by the tool that started the background work. */ + task_id: string; + /** Block until the task reaches a terminal status or the timeout expires. A timed-out wait returns [status: running] and leaves the task alive. */ + wait?: boolean; + /** Max wait in milliseconds (only meaningful with wait: true). Defaults to the configured wait timeout; capped by the configured maximum. */ + timeout_ms?: number; + } & Record; + /** Record and update a structured task list for the current work. Send the ENTIRE list every call — it REPLACES the previous list (there are no partial updates, no per-item edits). Use it to plan multi-step work and show progress: add one todo per concrete step before you start. Mark every todo being actively worked on `in_progress` — several at once when work genuinely runs in parallel (e.g. concurrent subagents or background commands), one for sequential work; while work remains, at least one task should be `in_progress`. Mark a todo `completed` the moment it is done (do not batch completions), and allow no `in_progress` item only once all work is complete. Skip the list for trivial single-step tasks. Statuses: `pending` (not started), `in_progress` (being worked on now), `completed` (finished). */ + todo_write: { + /** The COMPLETE task list, replacing any previous list. */ + todos: ({ + /** What the task is — a short imperative line. */ + content: string; + /** pending (not started) | in_progress (now) | completed (done). */ + status: "pending" | "in_progress" | "completed"; + })[]; + } & Record; + /** Update the exact current goal revision. edit, pause, and resume require a direct top-level human request. During an automatic continuation of the current goal, complete and blocked are also allowed. blocked is rejected before the configured minimum round count; the model remains responsible for judging that the same condition persisted across those rounds and must explain it in blocked_reason. */ + update_goal: { + /** Exact id returned by get_goal. */ + goal_id: string; + /** Exact positive revision returned by get_goal. */ + revision: number; + /** edit | pause | resume | complete | blocked */ + action: "edit" | "pause" | "resume" | "complete" | "blocked"; + /** Replacement objective; valid only with action edit. */ + objective?: string; + /** Replacement cap; valid only with action edit. */ + max_goal_rounds?: number; + /** Concrete blocking condition; required only with action blocked. */ + blocked_reason?: string; + } & Record; + /** Run a JavaScript workflow script that orchestrates subagents at scale. Use this for work that fans out across many independent pieces — an audit over many files, a migration, multi-angle research, adversarial verification of findings — where you write the orchestration as a script instead of delegating turn by turn. The workflow's identity rides the `meta` parameter as JSON: required `name` (short kebab-case) and `description` strings, optional `whenToUse` string and `phases` array (`{title, detail?, provider?, model?}`). The `script` parameter is the plain JavaScript body ONLY (NOT TypeScript, and NO `export const meta` statement — meta is a parameter, not code), running with top-level await; end with `return ` — the value must be JSON-serializable and is this tool's result. Script-body hooks: - `agent(prompt, opts?): Promise` — run one subagent to completion. Without `opts.schema` it resolves to the child's final text; with `opts.schema` (an object-rooted JSON Schema using ONLY type/properties/required/additionalProperties/items/enum/const/oneOf — no pattern/format/numeric bounds) it resolves to the validated object. Resolves `null` when the child fails (filter with `.filter(Boolean)`). Other opts: `label` (display), `phase` (progress group), and independent `provider`/`model` LLM target overrides (either may be provided alone). Anything else (`effort`/`isolation`/`agentType`) is rejected loudly. - `pipeline(items, ...stages): Promise` — run each item through the stages independently with NO barrier between stages (prefer this for multi-stage work). Each stage receives `(prev, item, index)`. An ordinary stage throw drops that ITEM to `null` and skips its remaining stages. - `parallel(thunks): Promise` — run zero-argument functions concurrently and await ALL of them (a barrier; use only when a stage genuinely needs every prior result together). A throwing thunk resolves to `null`. - `phase(title)` — start a progress phase; `log(message)` — narrate progress; `args` — the tool call's `args` input, verbatim. Misused hooks (bad arguments, unknown options, unsupported schemas, tripped caps) throw errors that ALWAYS kill the script — they never dissolve into a per-item `null`. Constraints: concurrency and total-agent caps apply; no filesystem, network, timers, or Node.js APIs are provided — the agents do the work, the script only coordinates them. The run executes in the foreground: this call returns when the whole script finishes. */ + workflow: { + /** The plain-JS workflow script body (top-level await allowed; NO `export const meta` statement; end with `return `). */ + script: string; + /** The workflow identity block (plain JSON — never code). */ + meta: { + /** Short kebab-case workflow name. */ + name: string; + /** One-line description of what the workflow does. */ + description: string; + /** Optional guidance on when this workflow applies. */ + whenToUse?: string; + /** Optional phase declarations matched by phase() calls. */ + phases?: ({ + /** The phase title phase() calls match by exact string. */ + title: string; + /** Optional one-line description of the phase. */ + detail?: string; + /** Optional provider override this phase is expected to use. */ + provider?: string; + /** Optional model override this phase is expected to use. */ + model?: string; + } & Record)[]; + } & Record; + /** Optional JSON input exposed to the script as the `args` global (wrap a bare list as a field, e.g. {"files": [...]}). */ + args?: Record; + } & Record; + /** Create or fully replace a UTF-8 text file. */ + write: { + /** Path to write, resolved by the filesystem backend. */ + file_path: string; + /** Full UTF-8 text content to write. */ + content: string; + /** The wider sandbox mode this file operation needs. Only valid as a one-shot retry of an operation the sandbox just denied; requires justification and user approval. */ + sandbox_permissions?: "workspace-write" | "danger-full-access"; + /** Required with sandbox_permissions: one sentence for the user explaining why this exact file operation needs the wider access. */ + justification?: string; + } & Record; +} + +interface ToolOutputMap { + bash: { + kind: "background"; + taskId: string; + } | { + kind: "foreground"; + exitCode: number | null; + signal: string | null; + timedOut: boolean; + aborted: boolean; + timeoutMs: number; + stdout: { + text: string; + truncated: boolean; + spillPath?: string; + }; + stderr: { + text: string; + truncated: boolean; + spillPath?: string; + }; + sandbox?: { + mode: string; + denied: boolean; + enforcement?: string; + runnerFailed?: boolean; + }; + }; + create_goal: { + goal: null; + } | { + goal: { + id: string; + revision: number; + objective: string; + phase: "active" | "paused" | "blocked" | "complete"; + roundsStarted: number; + maxGoalRounds: number; + blockedReason?: { + code: string; + message: string; + }; + }; + activation: "armed" | "disarmed"; + }; + edit: { + path: string; + before: string; + after: string; + }; + get_goal: { + goal: null; + } | { + goal: { + id: string; + revision: number; + objective: string; + phase: "active" | "paused" | "blocked" | "complete"; + roundsStarted: number; + maxGoalRounds: number; + blockedReason?: { + code: string; + message: string; + }; + }; + activation: "armed" | "disarmed"; + }; + interrupt_agent: { + accepted: boolean; + }; + list_agents: ({ + kind: "child"; + id: string; + label: string; + status: "running" | "idle" | "complete"; + parent?: string; + depth?: number; + } | { + kind: "diagnostic"; + id: string; + reason: "corrupt" | "unsupported" | "unavailable"; + parent?: string; + depth?: number; + })[]; + ralph: { + runId: string; + agentsStarted: number; + result: JsonValue; + }; + read: { + path: string; + offset: number; + lines: { + number: number; + text: string; + }[]; + totalLines: number; + }; + read_image: { + path: string; + image: { + attachmentId: string; + mediaType: "image/png" | "image/jpeg" | "image/webp" | "image/gif"; + bytes: number; + width: number; + height: number; + name?: string; + }; + }; + send_message: { + messageId: string; + }; + skill: { + name: string; + provider: string; + resourceBase?: { + kind: "directory"; + path: string; + } | { + kind: "url"; + url: string; + } | { + kind: "opaque"; + description: string; + }; + content: string; + }; + subagent: { + kind: "background"; + taskId: string; + } | { + kind: "continuable"; + subagentId: string; + } | { + kind: "foreground"; + runId: string; + output: JsonValue[]; + }; + subagent_fork: { + kind: "background"; + taskId: string; + } | { + kind: "continuable"; + subagentId: string; + } | { + kind: "foreground"; + runId: string; + output: JsonValue[]; + }; + task_kill: { + outcome: "cancellation-requested" | "already-finished"; + task: { + id: string; + kind: string; + label: string; + status: "running" | "stopping" | "completed" | "killed" | "failed"; + detail?: string; + startedAt: number; + finishedAt?: number; + }; + }; + task_list: ({ + id: string; + kind: string; + label: string; + status: "running" | "stopping" | "completed" | "killed" | "failed"; + detail?: string; + startedAt: number; + finishedAt?: number; + })[]; + task_output: { + text: string; + task: { + id: string; + kind: string; + label: string; + status: "running" | "stopping" | "completed" | "killed" | "failed"; + detail?: string; + startedAt: number; + finishedAt?: number; + }; + }; + todo_write: { + todos: ({ + content: string; + status: "pending" | "in_progress" | "completed"; + })[]; + counts: { + pending: number; + inProgress: number; + completed: number; + }; + }; + update_goal: { + goal: null; + } | { + goal: { + id: string; + revision: number; + objective: string; + phase: "active" | "paused" | "blocked" | "complete"; + roundsStarted: number; + maxGoalRounds: number; + blockedReason?: { + code: string; + message: string; + }; + }; + activation: "armed" | "disarmed"; + }; + workflow: { + runId: string; + agentsStarted: number; + result: JsonValue; + }; + write: { + path: string; + operation: "create" | "update"; + before: string | null; + after: string; + }; +} + +type ToolName = keyof ToolOutputMap + +declare class ToolCallError extends Error { + readonly name: "ToolCallError"; + readonly toolName: ToolName; +} + +declare const tools: { + [K in ToolName]: (args: ToolArgsMap[K]) => Promise; +} +``` diff --git a/examples/acp-agent/tests/snapshots/inline-image-prompt/input.json b/examples/acp-agent/tests/snapshots/inline-image-prompt/input.json new file mode 100644 index 0000000000..5f6e2cb13e --- /dev/null +++ b/examples/acp-agent/tests/snapshots/inline-image-prompt/input.json @@ -0,0 +1,28 @@ +{ + "steps": [ + { + "op": "initialize" + }, + { + "op": "newSession" + }, + { + "op": "promptContent", + "content": [ + { + "type": "text", + "text": "Inspect this image, then reply with exactly " + }, + { + "type": "image", + "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", + "mimeType": "image/png" + }, + { + "type": "text", + "text": "the single word DONE." + } + ] + } + ] +} diff --git a/examples/acp-agent/tests/snapshots/inline-image-prompt/session.jsonl b/examples/acp-agent/tests/snapshots/inline-image-prompt/session.jsonl new file mode 100644 index 0000000000..89cffe656d --- /dev/null +++ b/examples/acp-agent/tests/snapshots/inline-image-prompt/session.jsonl @@ -0,0 +1,17 @@ +{"type":"session","version":0,"id":"44444444-4444-4444-8444-444444444444","createdAt":1783952000000,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"agent/inbox/spliced","seq":0,"time":1783952000001,"data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Inspect this image, then reply with exactly "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":"the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"0c0c0c0c-0000-4000-8000-000000000001"}]}} +{"type":"turn/start","seq":1,"time":1783952000002,"data":{"turn":1}} +{"type":"agent/inbox/spliced","seq":2,"time":1783952000002,"data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} +{"type":"step/start","seq":3,"time":1783952000003,"data":{"turn":1,"step":1}} +{"type":"user/message","seq":4,"time":1783952000003,"data":{"content":[{"type":"text","text":"Inspect this image, then reply with exactly "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":"the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"0c0c0c0c-0000-4000-8000-000000000001"},"surfaceOp":"append"} +{"type":"user/message","seq":5,"time":1783952000004,"data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"0c0c0c0c-0000-4000-8000-000000000002"},"surfaceOp":"append"} +{"type":"session/title","seq":6,"time":1783952000004,"data":{"title":"Inspect this image, then reply","messageSeqs":[4],"source":{"kind":"fallback"}}} +{"type":"request/header","seq":7,"time":1783952000005,"data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/context","seq":8,"time":1783952000005,"data":{"provider":"deepseek-official","model":"deepseek-v4-flash"}} +{"type":"assistant/chunk","seq":9,"time":1783952000006,"data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} +{"type":"assistant/chunk","seq":10,"time":1783952000007,"data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}}} +{"type":"assistant/chunk","seq":11,"time":1783952000008,"data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}} +{"type":"assistant/chunk","seq":12,"time":1783952000009,"data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}} +{"type":"assistant/message","seq":13,"time":1783952000009,"data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"0c0c0c0c-0000-4000-8000-000000000003"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[9,10,11,12],"surfaceOp":"append"} +{"type":"step/end","seq":14,"time":1783952000010,"data":{"turn":1,"step":1}} +{"type":"turn/end","seq":15,"time":1783952000010,"data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/examples/acp-agent/tests/snapshots/inline-image-prompt/stdout.expected.jsonl b/examples/acp-agent/tests/snapshots/inline-image-prompt/stdout.expected.jsonl new file mode 100644 index 0000000000..4f0fb2e442 --- /dev/null +++ b/examples/acp-agent/tests/snapshots/inline-image-prompt/stdout.expected.jsonl @@ -0,0 +1,4 @@ +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} +{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"DONE"}}}} +{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/examples/acp-agent/tests/snapshots/read-image/stdout.expected.jsonl b/examples/acp-agent/tests/snapshots/read-image/stdout.expected.jsonl index 82ae8907ca..4f0fb2e442 100644 --- a/examples/acp-agent/tests/snapshots/read-image/stdout.expected.jsonl +++ b/examples/acp-agent/tests/snapshots/read-image/stdout.expected.jsonl @@ -1,4 +1,4 @@ -{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"promptCapabilities":{"image":false,"audio":false,"embeddedContext":false}},"authMethods":[]}} +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false}},"authMethods":[]}} {"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}} {"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"DONE"}}}} {"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}} diff --git a/packages/acp/acp/README.i18n.yaml b/packages/acp/acp/README.i18n.yaml index 1b303a23a8..1a39a39562 100644 --- a/packages/acp/acp/README.i18n.yaml +++ b/packages/acp/acp/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/acp/acp/README.md -README.md: 9cc4a5e271c7200f6ad8799a4b8fa9e64b2ca893 -README.zh.md: eafae5602bdeb408ef548a9e706e059bd99bde17 +README.md: 40d4b2df18f8102a352d8a8eb438e88da7fe720c +README.zh.md: 57b7e5a3f987861cfe0c5453f5d5a26d565f77ed diff --git a/packages/acp/acp/README.md b/packages/acp/acp/README.md index 9cc4a5e271..40d4b2df18 100644 --- a/packages/acp/acp/README.md +++ b/packages/acp/acp/README.md @@ -2,7 +2,7 @@ English | [中文](README.zh.md) -Automation-only [Agent Client Protocol](https://agentclientprotocol.com) server over JSON-RPC stdio. Programmatic clients create fresh harness agents, send text prompts, collect committed assistant text, resolve one-shot permission requests by policy, and cancel work. The primary in-repository client is [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md). +Automation-only [Agent Client Protocol](https://agentclientprotocol.com) server over JSON-RPC stdio. Programmatic clients create fresh harness agents, send text/image prompts, collect committed assistant text/images, resolve one-shot permission requests by policy, and cancel work. The primary in-repository client is [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md). This package is a transport adapter, not a UI integration or a capability seam. It does not expose editor navigation, transcript replay, commands, modes, configuration pickers, elicitation, reasoning, plans, titles, or tool presentation. Interactive rendering and human questions belong to the Web host and client modules. @@ -21,21 +21,21 @@ Both fields are optional so another agent/request listener may supply the target | Method | Behavior | |---|---| -| `initialize` | Negotiates the supported version and advertises baseline-only prompts (no image, audio, or embedded-context capability). No session, editor, terminal, filesystem, or MCP capability is advertised. | +| `initialize` | Negotiates the supported version. Image prompts are advertised only when a durable attachment store is mounted and the configured exact provider/model resolves with explicit image input; audio and embedded context stay false. No session, editor, terminal, filesystem, or MCP capability is advertised. | | `authenticate` | No-op because the server advertises no authentication methods. | | `session/new` | Creates a fresh agent with an absolute primary `cwd`; empty `additionalDirectories` and `mcpServers` are accepted, non-empty values reject. | -| `session/prompt` | Concatenates text blocks, renders baseline resource links as bracketed textual references, rejects empty or beyond-baseline input, permits one in-flight request per session, and waits for the whole agent to become idle. Normal quiescence reports `end_turn`; explicit ACP cancellation, disposal, or a prompt whose admission was discarded (a turnless slot) reports `cancelled`. | -| `session/cancel` | Cancels only the addressed agent and settles its pending prompt as `cancelled`; unknown ids are no-ops. | -| `session/update` | Emits one `agent_message_chunk` per non-empty text block in a committed `assistant/message`. Raw deltas and non-message events are omitted. | +| `session/prompt` | Preserves ordered text and supported inline image blocks, renders resource links as bracketed textual references, and rejects audio, embedded resources, malformed/empty input, or an image when capability was not advertised. It validates the whole image batch and rechecks the session's latest exact route before any save, commits every image before the user event, permits one in-flight request per session, and waits for admission, whole-agent idle, and ordered output delivery. Normal quiescence reports `end_turn`; explicit ACP cancellation, disposal, or a prompt whose admission was discarded (a turnless slot) reports `cancelled`. | +| `session/cancel` | Cancels only the addressed agent and marks any already-started admission so the pending prompt waits for it to quiesce, publishes no late user message, and settles as `cancelled`; unknown ids are no-ops. | +| `session/update` | Emits one `agent_message_chunk` per non-empty text or image block in a committed `assistant/message`, preserving order. Images are re-read and integrity-verified before inline base64 delivery. Raw deltas and non-message events are omitted. | | `session/request_permission` | Offers one-shot allow/reject choices for bridge-owned approval requests carrying a tool call id. Clients may answer automatically. | One connection may own several sessions. The bridge keys records by branded session id and checks exact agent identity before routing events or permission requests. Each session has an independent prompt slot, workspace, cancellation path, and disposer. -Committed-message output intentionally trades token-by-token latency for a clean automation result. Uncommitted provider chunks and retry attempts cannot leak partial text; reasoning and tool activity remain in the session log for observability through other interfaces. +Committed-message output intentionally trades token-by-token latency for a clean automation result. Uncommitted provider chunks and retry attempts cannot leak partial text or images; reasoning and tool activity remain in the session log for observability through other interfaces. Per-session delivery is serialized because attachment reads are asynchronous, and a missing or corrupt committed image fails the prompt response instead of emitting a placeholder. ## Lifecycle -Client disconnect and Cordis disposal share one memoized teardown. The bridge first rejects new sessions and prompts, settles pending prompts, then drains continuable descendants only below this connection's exact owned Agents before disposing those handles in parallel and awaiting every result before reporting any failure. Other frontends sharing the Context retain their continuable forests and admission. An ACP-only plugin reload therefore leaves no orphan agent. +Client disconnect and Cordis disposal share one memoized teardown. The bridge first rejects new sessions and prompts, cancels and quiesces prompt admission, agent activity, and ordered output delivery, then drains continuable descendants only below this connection's exact owned Agents before disposing those handles in parallel and awaiting every result before reporting any failure. Other frontends sharing the Context retain their continuable forests and admission. An ACP-only plugin reload therefore leaves no orphan agent. ACP requires each prompt response to carry a `stopReason`, but the bridge does not claim a prompt-specific turn outcome. Committed assistant messages stream across the owned activity, and steering or injected work may contribute before idle. Token-limit turn endings therefore do not become prompt-level ACP stop reasons (they settle as `end_turn`); a model error on the correlated turn rejects the prompt immediately. @@ -45,15 +45,15 @@ ACP requires each prompt response to carry a `stopReason`, but the bridge does n ## Model Experience -### Prompt text +### Prompt text and images #### What the model sees -`session/prompt` text blocks are concatenated verbatim into one user message; a baseline resource link appears in that message as a bracketed `[resource_link name=… uri=…]` reference the model may open with its own tools. Protocol metadata, client capabilities, permission choices, and session ids never enter the model request. +`session/prompt` preserves text/image order in one user message; adjacent text is concatenated, and a resource link appears as a bracketed `[resource_link name=… uri=…]` reference the model may open with its own tools. Inline image base64 is discarded after batch admission, so the durable message contains only verified attachment references. Protocol metadata, client capabilities, permission choices, and session ids never enter the model request. #### Token effect -Prompt tokens are data-dependent and remain in that session's history until compaction. Concurrent ACP sessions retain independent contexts. +Prompt tokens and image charges are data-dependent and remain in that session's history until compaction. Concurrent ACP sessions retain independent contexts. #### KV Cache effect @@ -76,6 +76,6 @@ Append-only through the owning tool result. ## Known Limitations and Deferred Work - **Fresh sessions only** — load, list, resume, delete, and fork are unsupported. -- **Baseline prompts and one workspace only** — images, audio, embedded resources, non-empty additional directories, and MCP servers reject; resource links flatten to textual references rather than fetched content. +- **Raster images and one workspace only** — image prompts require a durable store plus an exact route that declares image input; only PNG, JPEG, WebP, and GIF are accepted. Audio, embedded resources, non-empty additional directories, and MCP servers reject; resource links flatten to textual references rather than fetched content. - **Committed answers only** — live progress, reasoning, tool activity, plans, titles, and usage stay off the wire. - **Connection-owned lifetime** — one connection releases all of its sessions; per-session close is not implemented. diff --git a/packages/acp/acp/README.zh.md b/packages/acp/acp/README.zh.md index eafae5602b..57b7e5a3f9 100644 --- a/packages/acp/acp/README.zh.md +++ b/packages/acp/acp/README.zh.md @@ -2,7 +2,7 @@ [English](README.md) | 中文 -通过 JSON-RPC stdio 提供的仅面向自动化的 [ACP(Agent Client Protocol)](https://agentclientprotocol.com) 服务器。程序化客户端可以创建新 harness agent(智能体)、发送文本提示词、收集已提交的 assistant 文本、按策略响应一次性权限请求并取消工作。仓库中的主要客户端是 [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)。 +通过 JSON-RPC stdio 提供的仅面向自动化的 [ACP(Agent Client Protocol)](https://agentclientprotocol.com) 服务器。程序化客户端可以创建新 harness agent(智能体)、发送文本/图片提示词、收集已提交的 assistant 文本/图片、按策略响应一次性权限请求并取消工作。仓库中的主要客户端是 [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)。 此包是传输适配器,而非 UI 集成或能力 seam。它不公开编辑器导航、transcript(文本记录)回放、命令、模式、配置选择器、信息征集、推理(reasoning)、计划、标题或工具展示。交互式渲染与向用户提问属于 Web 宿主和客户端模块。 @@ -21,21 +21,21 @@ | 方法 | 行为 | |---|---| -| `initialize` | 协商受支持的版本,并仅公布基线提示词(无图像、音频或嵌入上下文能力)。不公布会话、编辑器、终端、文件系统或 MCP 能力。 | +| `initialize` | 协商受支持的版本。只有挂载持久附件存储,且配置的确切提供方/模型解析后明确支持图片输入时,才公布图片提示词能力;音频与嵌入上下文保持 false。不公布会话、编辑器、终端、文件系统或 MCP 能力。 | | `authenticate` | 空操作,因为服务器不公布身份验证方法。 | | `session/new` | 以绝对路径作为主 `cwd` 创建新 agent;接受空的 `additionalDirectories` 和 `mcpServers`,拒绝非空值。 | -| `session/prompt` | 拼接文本块,将基线资源链接渲染为带方括号的文本引用,拒绝空输入或超出基线的输入,每个会话只允许一个正在处理的请求,并等待整个 agent 进入空闲状态。正常完全停稳时报告 `end_turn`;显式 ACP 取消、资源释放,或准入被丢弃的提示词(无轮次槽位)时报告 `cancelled`。 | -| `session/cancel` | 仅取消指定的 agent,并将其待处理提示词结算为 `cancelled`;未知 id 为空操作。 | -| `session/update` | 为每个非空文本块发出一个 `agent_message_chunk`;这些文本块来自已提交的 `assistant/message`。省略原始增量和非消息事件。 | +| `session/prompt` | 保留文本与受支持内联图片块的顺序,将资源链接渲染为带方括号的文本引用,并拒绝音频、嵌入资源、格式错误/空输入,或在未公布能力时提交图片。它会先校验完整图片批次并重新检查会话的最新确切路由,再保存任一成员;在用户事件前提交全部图片;每个会话只允许一个正在处理的请求,并等待准入、整个 agent 空闲和有序输出交付全部停稳。正常完全停稳时报告 `end_turn`;显式 ACP 取消、资源释放,或准入被丢弃的提示词(无轮次槽位)时报告 `cancelled`。 | +| `session/cancel` | 仅取消指定的 agent,并标记已经启动的准入工作,使待处理提示词等待其停稳、不发布迟到的用户消息,随后以 `cancelled` 结算;未知 id 为空操作。 | +| `session/update` | 为已提交 `assistant/message` 中的每个非空文本或图片块发出一个 `agent_message_chunk`,并保留顺序。图片在以内联 base64 交付前会重新读取并校验完整性。省略原始增量和非消息事件。 | | `session/request_permission` | 为携带工具调用 id、由桥接层拥有的批准请求提供一次性允许/拒绝选项。客户端可以自动回答。 | 一个连接可以拥有多个会话。桥接层以带品牌的会话 id 作为记录键,并在路由事件或权限请求前检查 agent 是否为同一对象。每个会话都有独立的提示词槽位、工作区、取消路径和资源释放器。 -已提交消息输出有意牺牲逐 token 输出的低延迟,以换取干净的自动化结果。未提交的提供方分片和重试尝试无法泄漏部分文本;推理与工具活动仍保留在会话日志中,以便其他界面观测。 +已提交消息输出有意牺牲逐 token 输出的低延迟,以换取干净的自动化结果。未提交的提供方分片和重试尝试无法泄漏部分文本或图片;推理与工具活动仍保留在会话日志中,以便其他界面观测。由于附件读取是异步的,每个会话会串行交付内容;已提交图片缺失或损坏时,提示词响应会失败,而不会发出占位符。 ## 生命周期 -客户端断开与 Cordis 释放共用同一个记忆化清理流程。桥接层先拒绝新会话和提示词,结算待处理提示词,然后只 drain 此连接确切拥有的 Agent 之下的可继续后代,再并行释放这些 handle,并等待全部结果结算后才报告失败。其他共享该上下文的前端会保留其可继续森林和准入。因此,仅 ACP 的插件重载不会遗留 agent。 +客户端断开与 Cordis 释放共用同一个记忆化清理流程。桥接层先拒绝新会话和提示词,取消并等待提示词准入、agent 活动和有序输出交付全部停稳,然后只 drain 此连接确切拥有的 Agent 之下的可继续后代,再并行释放这些 handle,并等待全部结果结算后才报告失败。其他共享该上下文的前端会保留其可继续森林和准入。因此,仅 ACP 的插件重载不会遗留 agent。 ACP 要求每个提示词响应都携带 `stopReason`,但桥接层不声称它表示提示词专属的轮次结果。已提交的 assistant 消息会在整个自有活动期间流式输出,agent 进入空闲状态前发生的 steering(中途引导)或注入工作也可能参与其中。因此,因 token 上限而结束的轮次不会成为提示词级 ACP 停止原因(它们以 `end_turn` 结算);关联轮次上的模型错误会立即拒绝该提示词。 @@ -45,15 +45,15 @@ ACP 要求每个提示词响应都携带 `stopReason`,但桥接层不声称它 ## 模型体验 -### 提示词文本 +### 提示词文本与图片 #### 模型看到的内容 -`session/prompt` 文本块会原样拼接为一条用户消息;基线资源链接会在该消息中表示为带方括号的 `[resource_link name=… uri=…]` 引用,模型可以使用自身工具打开它。协议元数据、客户端能力、权限选择和会话 id 绝不进入模型请求。 +`session/prompt` 会在一条用户消息中保留文本/图片顺序;相邻文本会拼接,资源链接则表示为带方括号的 `[resource_link name=… uri=…]` 引用,模型可以使用自身工具打开它。内联图片 base64 在批量准入后即被丢弃,因此持久消息只包含经过校验的附件引用。协议元数据、客户端能力、权限选择和会话 id 绝不进入模型请求。 #### Token 影响 -提示词 token 取决于数据,并保留在该会话的历史中直到上下文压缩(context compaction)。并发 ACP 会话保留独立上下文。 +提示词 token 与图片费用取决于数据,并保留在该会话的历史中直到上下文压缩(context compaction)。并发 ACP 会话保留独立上下文。 #### KV Cache 影响 @@ -76,6 +76,6 @@ ACP 要求每个提示词响应都携带 `stopReason`,但桥接层不声称它 ## 已知限制与暂缓事项 - **仅新会话**:不支持加载、列出、恢复、删除和 fork。 -- **仅基线提示词和一个 workspace**:图像、音频、嵌入资源、非空附加目录和 MCP 服务器都会被拒绝;资源链接只会展平为文本引用,不会获取其内容。 +- **仅光栅图片和一个 workspace**:图片提示词要求持久存储以及明确声明支持图片输入的确切路由;只接受 PNG、JPEG、WebP 和 GIF。音频、嵌入资源、非空附加目录和 MCP 服务器都会被拒绝;资源链接只会展平为文本引用,不会获取其内容。 - **仅已提交答案**:实时进度、推理、工具活动、计划、标题和用量不会通过协议传输。 - **由连接管理的生命周期**:一个连接会释放其所有会话;尚未实现单个会话关闭功能。 diff --git a/packages/acp/acp/package.json b/packages/acp/acp/package.json index ff5a52bc63..ba95a2cf44 100644 --- a/packages/acp/acp/package.json +++ b/packages/acp/acp/package.json @@ -36,13 +36,16 @@ "@deepseek-ai/schemastery": "workspace:^" }, "peerDependencies": { + "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-invariants": "workspace:^", + "@deepseek-ai/dsh-llm": "workspace:^", "@deepseek-ai/dsh-session": "workspace:^", "@deepseek-ai/dsh-user-approval": "workspace:^", "@deepseek-ai/cordis": "workspace:^" }, "devDependencies": { + "@deepseek-ai/dsh-attachment": "workspace:^", "@deepseek-ai/dsh-agent": "workspace:^", "@deepseek-ai/dsh-agent-loop": "workspace:^", "@deepseek-ai/dsh-agent-loop-testkit": "workspace:^", diff --git a/packages/acp/acp/src/codec.ts b/packages/acp/acp/src/codec.ts index 9fcdb68f7b..151756a03e 100644 --- a/packages/acp/acp/src/codec.ts +++ b/packages/acp/acp/src/codec.ts @@ -3,7 +3,7 @@ * @module @deepseek-ai/dsh-acp/codec */ -import type { ContentBlock as AcpContentBlock, StopReason } from '@agentclientprotocol/sdk' +import type { StopReason } from '@agentclientprotocol/sdk' import type { TurnEndReason } from '@deepseek-ai/dsh-session' /** @@ -32,35 +32,3 @@ export function turnEndToStopReason(reason: TurnEndReason): StopReason { return 'end_turn' } } - -/** - * Flatten an ACP prompt's baseline blocks to text. Text blocks concatenate - * verbatim; resource links become explicit textual references so a baseline - * client can point at files without the bridge silently dropping that context. - * @param prompt - supported ACP prompt blocks. - * @returns text in wire order, with resource links rendered as bracketed references. - */ -export function acpPromptToText(prompt: readonly AcpContentBlock[]): string { - return prompt.flatMap((block): string[] => { - switch (block.type) { - case 'text': - return [block.text] - case 'resource_link': - return [`\n[resource_link name=${JSON.stringify(block.name)} uri=${JSON.stringify(block.uri)}]\n`] - default: - return [] - } - }).join('') -} - -/** - * Whether a prompt carries content beyond the ACP baseline. The spec requires - * every agent to accept `text` and `resource_link`; richer inline payloads - * (image, audio, embedded resource) are optional capabilities this bridge does - * not advertise, so they are rejected rather than silently dropped. - * @param prompt - ACP prompt blocks to inspect. - * @returns `true` when any block is neither `text` nor `resource_link`. - */ -export function promptHasUnsupportedContent(prompt: readonly AcpContentBlock[]): boolean { - return prompt.some(block => block.type !== 'text' && block.type !== 'resource_link') -} diff --git a/packages/acp/acp/src/content.ts b/packages/acp/acp/src/content.ts new file mode 100644 index 0000000000..56e027a1b7 --- /dev/null +++ b/packages/acp/acp/src/content.ts @@ -0,0 +1,238 @@ +/** ACP wire-content admission and projection owned by the ACP adapter. @module */ + +import type { ContentBlock as AcpContentBlock } from '@agentclientprotocol/sdk' +import type { Context } from '@deepseek-ai/cordis' +import { AttachmentError } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentRef, ImageMediaType, SaveImageAttachment } from '@deepseek-ai/dsh-attachment' +import type { Agent } from '@deepseek-ai/dsh-agent' +import type { ContentBlock } from '@deepseek-ai/dsh-llm' + +/** Raster formats shared by ACP image blocks and the core attachment vocabulary. */ +const IMAGE_MEDIA_TYPES: readonly ImageMediaType[] = [ + 'image/png', + 'image/jpeg', + 'image/webp', + 'image/gif', +] + +/** Canonical RFC 4648 base64, excluding whitespace and URL-safe aliases. */ +const CANONICAL_BASE64 = /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/ + +/** Content-admission failure category used by the protocol handler. */ +export type AcpContentFailureKind = 'invalid' | 'internal' + +/** Error with a stable ACP request-failure category and no raw binary payload. */ +export class AcpContentError extends Error { + /** Whether the bridge should report invalid params or an internal failure. */ + readonly kind: AcpContentFailureKind + + /** + * @param message - safe protocol-facing detail without inline binary data. + * @param kind - request-failure category. + * @param options - optional causal chain for diagnostics. + */ + constructor(message: string, kind: AcpContentFailureKind, options?: ErrorOptions) { + super(message, options) + this.name = 'AcpContentError' + this.kind = kind + } +} + +/** Narrow a wire MIME string to the durable raster vocabulary. */ +function imageMediaType(value: string): ImageMediaType | undefined { + return IMAGE_MEDIA_TYPES.includes(value as ImageMediaType) ? value as ImageMediaType : undefined +} + +/** Strictly decode one ACP inline image without accepting base64 aliases. */ +function decodeImage(block: Extract): SaveImageAttachment { + const mediaType = imageMediaType(block.mimeType) + if (mediaType === undefined) { + throw new AcpContentError('image mimeType must be image/png, image/jpeg, image/webp, or image/gif', 'invalid') + } + if (!CANONICAL_BASE64.test(block.data)) { + throw new AcpContentError('image data must be canonical base64', 'invalid') + } + const data = Buffer.from(block.data, 'base64') + if (data.toString('base64') !== block.data) { + throw new AcpContentError('image data must be canonical base64', 'invalid') + } + return { data, mediaType } +} + +/** Resolve the exact current route and require explicit image input support. */ +async function assertImageRoute(ctx: Context, agent: Agent, signal: AbortSignal): Promise { + const routed = agent.session.requestHeader()?.config + const provider = routed?.provider ?? agent.options.provider + const model = routed?.model ?? agent.options.model + const llm = ctx.get('llm') + if (provider === undefined || model === undefined || llm === undefined) { + throw new AcpContentError('the current model route could not be resolved for image input', 'invalid') + } + let info: Awaited> + try { + info = await llm.resolveModelInfo(provider, model, signal) + } catch (error: unknown) { + throw new AcpContentError('the current model route could not be verified for image input', 'invalid', { cause: error }) + } + if (info.inputModalities === undefined || !info.inputModalities.includes('image')) { + throw new AcpContentError(`model "${model}" does not declare image input`, 'invalid') + } +} + +/** + * Determine whether initialization may truthfully advertise inline image prompts. + * Unknown service, route, capability, or deployment media support is negative. + * @param ctx - bridge context carrying optional attachment and model services. + * @param provider - configured provider route used for newly created sessions. + * @param model - configured exact model id used for newly created sessions. + * @returns whether this bridge can admit images at initialization time. + */ +export async function supportsAcpImagePrompts( + ctx: Context, + provider: string | undefined, + model: string | undefined, +): Promise { + const attachments = ctx.get('attachments') + const llm = ctx.get('llm') + if (attachments === undefined || llm === undefined || provider === undefined || model === undefined) return false + if (!attachments.imageLimits.mediaTypes.some(mediaType => IMAGE_MEDIA_TYPES.includes(mediaType))) return false + try { + const info = await llm.resolveModelInfo(provider, model) + return info.inputModalities?.includes('image') === true + } catch { + return false + } +} + +/** Render one baseline resource link into the core's current text vocabulary. */ +function resourceLinkText(block: Extract): string { + return `\n[resource_link name=${JSON.stringify(block.name)} uri=${JSON.stringify(block.uri)}]\n` +} + +/** + * Admit one ACP prompt into ordered durable core content. + * Every wire block and image is validated before the ordered image batch starts + * writing; cancellation after a successful content-addressed write may leave an + * unreachable object but never queues a late user message. + * @param ctx - bridge context carrying attachment and model services. + * @param agent - destination agent whose latest exact route controls admission. + * @param prompt - untrusted ACP prompt blocks in wire order. + * @param imageEnabled - capability result advertised during initialization. + * @param signal - admission cancellation signal. + * @returns core content with durable image references in wire order. + */ +export async function admitAcpPrompt( + ctx: Context, + agent: Agent, + prompt: readonly AcpContentBlock[], + imageEnabled: boolean, + signal: AbortSignal, +): Promise { + const images: SaveImageAttachment[] = [] + for (const block of prompt) { + switch (block.type) { + case 'text': + case 'resource_link': + break + case 'image': + if (!imageEnabled) throw new AcpContentError('inline image prompts were not advertised by this connection', 'invalid') + images.push(decodeImage(block)) + break + case 'audio': + throw new AcpContentError('audio prompt content is not supported', 'invalid') + case 'resource': + throw new AcpContentError('embedded resource prompt content is not supported', 'invalid') + /* v8 ignore next 2 -- ACP ContentBlock is a closed generated union. */ + default: + throw new AcpContentError('unsupported ACP prompt content', 'invalid') + } + } + + let refs: readonly ImageAttachmentRef[] = [] + if (images.length > 0) { + const attachments = ctx.get('attachments') + if (attachments === undefined) throw new AcpContentError('no attachment store is mounted', 'invalid') + await assertImageRoute(ctx, agent, signal) + signal.throwIfAborted() + try { + refs = await attachments.saveImages(images) + } catch (error: unknown) { + if (error instanceof AttachmentError && error.code !== 'ATTACHMENT_WRITE_FAILED') { + throw new AcpContentError(error.message, 'invalid', { cause: error }) + } + throw new AcpContentError('unable to persist the prompt image batch', 'internal', { cause: error }) + } + signal.throwIfAborted() + } + + const content: ContentBlock[] = [] + let pendingText = '' + let imageIndex = 0 + const flushText = (): void => { + if (pendingText.length === 0) return + content.push({ type: 'text', text: pendingText }) + pendingText = '' + } + for (const block of prompt) { + switch (block.type) { + case 'text': + pendingText += block.text + break + case 'resource_link': + pendingText += resourceLinkText(block) + break + case 'image': { + flushText() + const ref = refs[imageIndex++] as ImageAttachmentRef + content.push({ type: 'image', attachment: ref }) + break + } + /* v8 ignore start -- the validation pass above rejects both tags before reconstruction. */ + case 'audio': + case 'resource': + break + /* v8 ignore stop */ + /* v8 ignore next 2 -- validated by the first closed-union switch. */ + default: + break + } + } + flushText() + if (!content.some(block => block.type === 'image' || (block.type === 'text' && block.text.trim().length > 0))) { + throw new AcpContentError('empty prompt', 'invalid') + } + return content +} + +/** + * Translate one committed assistant block to ACP wire content. + * Images are re-read and integrity-verified before inline base64 delivery; + * unsupported core output blocks stay off the automation wire. + * @param ctx - bridge context carrying the authoritative attachment store. + * @param block - committed core assistant block. + * @returns ACP text/image content, or undefined for non-output blocks. + */ +export async function assistantBlockToAcp( + ctx: Context, + block: ContentBlock, +): Promise { + if (block.type === 'text') { + return block.text.length === 0 ? undefined : { type: 'text', text: block.text } + } + if (block.type !== 'image') return undefined + const attachments = ctx.get('attachments') + if (attachments === undefined) { + throw new AcpContentError('cannot deliver assistant image: no attachment store is mounted', 'internal') + } + let stored: Awaited> + try { + stored = await attachments.readImage(block.attachment) + } catch (error: unknown) { + throw new AcpContentError('cannot deliver assistant image: the attachment is unavailable or corrupt', 'internal', { cause: error }) + } + return { + type: 'image', + data: Buffer.from(stored.data).toString('base64'), + mimeType: stored.ref.mediaType, + } +} diff --git a/packages/acp/acp/src/index.ts b/packages/acp/acp/src/index.ts index d595c69e69..eeef146165 100644 --- a/packages/acp/acp/src/index.ts +++ b/packages/acp/acp/src/index.ts @@ -2,9 +2,9 @@ * Automation-only Agent Client Protocol server over JSON-RPC stdio. * * The bridge exposes fresh harness sessions to trusted programmatic clients. It - * carries prompt text, committed assistant text, cancellation, and one-shot - * permission decisions; presentation and human-interaction features stay with - * the harness's UI modules. + * carries prompt text/images, committed assistant text/images, cancellation, + * and one-shot permission decisions; presentation and human-interaction + * features stay with the harness's UI modules. * * @module @deepseek-ai/dsh-acp */ @@ -37,7 +37,8 @@ import type { Agent } from '@deepseek-ai/dsh-agent' import { SessionId, type SessionEvent, type TurnEndReason } from '@deepseek-ai/dsh-session' // Side-effect type import: declaration-merges the approval waterfall answered below. import type {} from '@deepseek-ai/dsh-user-approval' -import { acpPromptToText, promptHasUnsupportedContent, turnEndToStopReason } from './codec.ts' +import { AcpContentError, admitAcpPrompt, assistantBlockToAcp, supportsAcpImagePrompts } from './content.ts' +import { turnEndToStopReason } from './codec.ts' export const name = 'acp' /** The bridge creates and owns agents; every other concern is carried by the agent composition. */ @@ -86,14 +87,27 @@ interface SessionRecord { agent: Agent /** Exact owned-agent disposer; resolves after registry, loop, and session teardown. */ dispose: () => Promise - /** In-flight prompt and its captured turn number for exact settlement. */ + /** Ordered assistant-output delivery; every task contains its own failure. */ + outputTail: Promise + /** In-flight admission/turn/output lifecycle for exact settlement. */ inflight: { resolve: (reason: StopReason) => void reject: (error: Error) => void - messageId: string + /** Set only after rich-content admission succeeds and the message is built. */ + messageId: string | undefined turn: number | undefined /** The correlated turn's ending, set at turn/end and settled at whole-agent idle. */ endReason: TurnEndReason | undefined + /** Admission quiescence gate, including any attachment write already in progress. */ + admissionDone: Promise + finishAdmission: () => void + admissionController: AbortController + cancelRequested: boolean + settlementStarted: boolean + /** Conversion failure for committed output owned by this prompt's turn. */ + outputError: Error | undefined + /** Failure before a correlated turn exists. */ + agentError: Error | undefined } | undefined } @@ -110,6 +124,7 @@ export function apply(ctx: Context, config: AcpConfig): void { const sessions = new Map() let closed = false let conn: AgentSideConnection + let imagePromptEnabled = false /** Return the bridge-owned record for an agent, rejecting same-id impostors. */ const ownedRecord = (agent: Agent): SessionRecord | undefined => { @@ -127,19 +142,15 @@ export function apply(ctx: Context, config: AcpConfig): void { return record } - /** Send a protocol update without letting a disconnected client fail an agent turn. */ - const notify = (notification: SessionNotification): void => { - /* v8 ignore next 3 -- only a transport write failure reaches this guard. */ - void conn.sessionUpdate(notification).catch((error: unknown) => { + /** Send one ordered protocol update while containing transport-only failure. */ + const notify = async (notification: SessionNotification): Promise => { + try { + await conn.sessionUpdate(notification) + /* v8 ignore start -- the ACP SDK contains notification-handler failures; only a transport write failure reaches this guard. */ + } catch (error: unknown) { logger.warn(`acp: session/update failed: ${String(error)}`) - }) - } - - const settlePrompt = (record: SessionRecord, reason: StopReason): void => { - const inflight = record.inflight - if (inflight === undefined) return - record.inflight = undefined - inflight.resolve(reason) + } + /* v8 ignore stop */ } const rejectFromError = ( @@ -149,48 +160,89 @@ export function apply(ctx: Context, config: AcpConfig): void { inflight.reject(internalError(`turn failed: ${reason.error.message}`)) } - // Emit only committed assistant text. Raw chunks, reasoning, tools, plans, - // titles, and retry markers are presentation or trace data and stay off the - // automation wire. + /** + * Settle one exact prompt only after admission, agent activity, and ordered + * assistant delivery have all reached quiescence. + */ + const settleAfterQuiescence = ( + record: SessionRecord, + inflight: NonNullable, + ): void => { + if (inflight.settlementStarted) return + inflight.settlementStarted = true + void (async () => { + await inflight.admissionDone + await record.agent.whenIdle() + // session/event enqueues synchronously before the agent becomes idle; + // reading the live tail here includes every committed output task. + await record.outputTail + /* v8 ignore next -- this prompt owns the slot until this exact settlement clears it. */ + if (record.inflight !== inflight) return + record.inflight = undefined + if (inflight.cancelRequested) { + inflight.resolve('cancelled') + return + } + if (inflight.outputError !== undefined) { + inflight.reject(internalError(`assistant output delivery failed: ${inflight.outputError.message}`)) + return + } + if (inflight.agentError !== undefined) { + inflight.reject(internalError(`turn failed: ${inflight.agentError.message}`)) + return + } + const end = inflight.endReason + if (end === undefined) { + inflight.resolve('cancelled') + } else if (end.kind === 'error') { + rejectFromError(inflight, end) + } else { + // Token-limit and other non-terminal endings are not prompt-level stop + // reasons; ordinary quiescence reports end_turn. + inflight.resolve(end.kind === 'max-tokens' ? 'end_turn' : turnEndToStopReason(end)) + } + })() + /* v8 ignore start -- admissionDone only resolves, whenIdle is a quiescence gate, and outputTail contains its own failures. */ + .catch((error: unknown) => { + if (record.inflight !== inflight) return + record.inflight = undefined + inflight.reject(internalError(`prompt settlement failed: ${errorChain(error)}`)) + }) + /* v8 ignore stop */ + } + + // Emit only committed assistant text/images. Raw chunks, reasoning, tools, + // plans, titles, and retry markers are presentation or trace data and stay + // off the automation wire. One per-session chain preserves block/message + // order across asynchronous attachment reads. ctx.on('session/event', (session, event: SessionEvent) => { const record = sessions.get(session.header.id) if (record === undefined || record.agent.session !== session) return try { if (event.type === 'assistant/message') { - for (const block of event.data.message.content) { - if (block.type === 'text' && block.text.length > 0) { - notify({ + const inflight = record.inflight?.turn === event.data.turn ? record.inflight : undefined + const previous = record.outputTail + const delivery = previous.then(async () => { + for (const block of event.data.message.content) { + const content = await assistantBlockToAcp(ctx, block) + if (content === undefined) continue + await notify({ sessionId: record.agent.session.id, - update: { - sessionUpdate: 'agent_message_chunk', - content: { type: 'text', text: block.text }, - }, - }) - } else if (block.type === 'image') { - notify({ - sessionId: record.agent.session.id, - update: { - sessionUpdate: 'agent_message_chunk', - content: { - type: 'text', - text: `[image attachment ${block.attachment.attachmentId}]`, - }, - }, + update: { sessionUpdate: 'agent_message_chunk', content }, }) } - } + }) + record.outputTail = delivery.catch((error: unknown) => { + // assistantBlockToAcp owns conversion failures and always throws Error. + const failure = error as Error + if (inflight !== undefined) inflight.outputError ??= failure + logger.warn(`acp: assistant output conversion failed: ${errorChain(error)}`) + }) } } finally { const inflight = record.inflight if (inflight !== undefined && event.type === 'turn/end' && inflight.turn === event.data.turn) { - if (event.data.reason.kind === 'error') { - // Model failures surface immediately as prompt errors; ordinary - // endings wait for whole-agent idle below. - record.inflight = undefined - rejectFromError(inflight, event.data.reason) - } else { - inflight.endReason = event.data.reason - } + inflight.endReason = event.data.reason } } }) @@ -205,8 +257,8 @@ export function apply(ctx: Context, config: AcpConfig): void { const record = ownedRecord(agent) const inflight = record?.inflight if (record === undefined || inflight === undefined || inflight.turn === turn) return - record.inflight = undefined - inflight.reject(internalError(`turn failed: ${errorChain(error)}`)) + inflight.agentError = new Error(errorChain(error)) + settleAfterQuiescence(record, inflight) }) // Permission requests are a machine policy channel for ACP clients such as @@ -231,17 +283,18 @@ export function apply(ctx: Context, config: AcpConfig): void { const makeAgent = (connection: AgentSideConnection): AcpAgent => { conn = connection return { - initialize(_params: InitializeRequest): Promise { + async initialize(_params: InitializeRequest): Promise { // Single-version agent: the spec's "same version if supported, else // the latest supported" both resolve to this server's one version. - return Promise.resolve({ + imagePromptEnabled = await supportsAcpImagePrompts(ctx, config.provider, config.model) + return { protocolVersion: PROTOCOL_VERSION, agentInfo: { name: 'deepseek-harness-acp', version: '0.0.1' }, agentCapabilities: { - promptCapabilities: { image: false, audio: false, embeddedContext: false }, + promptCapabilities: { image: imagePromptEnabled, audio: false, embeddedContext: false }, }, authMethods: [], - }) + } }, authenticate(_params: AuthenticateRequest): Promise { @@ -269,6 +322,7 @@ export function apply(ctx: Context, config: AcpConfig): void { sessions.set(sessionId, { agent: handle.agent, dispose: () => handle.dispose(), + outputTail: Promise.resolve(), inflight: undefined, }) return { sessionId } @@ -280,66 +334,91 @@ export function apply(ctx: Context, config: AcpConfig): void { if (record.inflight !== undefined) { throw invalidParams('a prompt is already in flight for this session') } - if (promptHasUnsupportedContent(params.prompt)) { - throw invalidParams('only text and resource_link prompt content is supported') + const completion = Promise.withResolvers() + const admission = Promise.withResolvers() + const admissionController = new AbortController() + const inflight: NonNullable = { + resolve: completion.resolve, + reject: completion.reject, + messageId: undefined, + turn: undefined, + endReason: undefined, + admissionDone: admission.promise, + finishAdmission: admission.resolve, + admissionController, + cancelRequested: false, + settlementStarted: false, + outputError: undefined, + agentError: undefined, } - const text = acpPromptToText(params.prompt) - if (text.trim().length === 0) throw invalidParams('empty prompt') + // Reserve the one-prompt slot before the first asynchronous route or + // attachment operation so concurrent prompts and cancellation observe + // admission as genuinely in flight. + record.inflight = inflight - // Not driving a retired agent is this bridge's contract: an - // agent-loop-only reload disposes the loop's agents while the bridge - // record survives, so validate the record against the live registry - // before sending — a disposed machine would accept the item silently. - if (ctx.agents.get(record.agent.id) !== record.agent) { - throw internalError('prompt was not queued: the agent was disposed outside the bridge') + let admissionFailed = false + let admissionFailure: unknown + try { + // Do not persist rich content for a retired destination. Re-check + // after admission too because an agent-loop reload may race storage. + if (ctx.agents.get(record.agent.id) !== record.agent) { + throw internalError('prompt was not queued: the agent was disposed outside the bridge') + } + const content = await admitAcpPrompt( + ctx, + record.agent, + params.prompt, + imagePromptEnabled, + admissionController.signal, + ) + // No await may separate this final abort check from followup: a + // cancellation that wins admission must never enqueue a late turn. + admissionController.signal.throwIfAborted() + if (ctx.agents.get(record.agent.id) !== record.agent) { + throw internalError('prompt was not queued: the agent was disposed outside the bridge') + } + const message = createUserMessage({ content, source: { kind: 'user' } }) + inflight.messageId = message.id + record.agent.followup(message) + } catch (error: unknown) { + admissionFailed = true + admissionFailure = error + } finally { + inflight.finishAdmission() } - const message = createUserMessage({ content: [{ type: 'text', text }], source: { kind: 'user' } }) - const stopReason = await new Promise((resolve, reject) => { - // Arm the slot before followup() so a listener-driven synchronous - // turn cannot slip past correlation; a synchronous followup() - // failure (invalid input) must free the slot again or the session - // would reject every later prompt as already in flight. - const inflight: NonNullable = { - resolve, reject, messageId: message.id, turn: undefined, endReason: undefined, + + if (inflight.cancelRequested) { + settleAfterQuiescence(record, inflight) + return { stopReason: await completion.promise } + } + if (admissionFailed) { + record.inflight = undefined + if (admissionFailure instanceof AcpContentError) { + throw admissionFailure.kind === 'invalid' + ? invalidParams(admissionFailure.message) + : internalError(admissionFailure.message) } - record.inflight = inflight - try { - record.agent.followup(message) - // The machine's send() contains listener failures and accepts - // any typed input; this guards a future synchronous throw so the - // slot cannot wedge. - /* v8 ignore start -- future-proofing guard, see above */ - } catch (error: unknown) { - record.inflight = undefined - const detail = error instanceof Error ? error.message : String(error) - throw internalError(`prompt was not queued: ${detail}`) - } - /* v8 ignore stop */ - // Settlement waits for whole-agent idle: a correlated turn/end arms - // `endReason`, while a turnless slot (admission discarded the - // prompt) stays cancelled. Other producers may run further turns - // before quiescence; the prompt settles only when the agent stops. - void record.agent.whenIdle().then(() => { - if (record.inflight !== inflight) return - record.inflight = undefined - const end = inflight.endReason - if (end === undefined) { - inflight.resolve('cancelled') - } else { - // Token-limit and other non-terminal endings are not prompt-level - // stop reasons (see README); only normal quiescence reports end_turn. - inflight.resolve(end.kind === 'max-tokens' ? 'end_turn' : turnEndToStopReason(end)) - } - }) - }) + if (admissionFailure instanceof RequestError) throw admissionFailure + // The admission codec and same-process agent seam throw Error values. + const detail = (admissionFailure as Error).message + throw internalError(`prompt was not queued: ${detail}`) + } + + settleAfterQuiescence(record, inflight) + const stopReason = await completion.promise return { stopReason } }, cancel(params: CancelNotification): Promise { const record = sessions.get(SessionId(params.sessionId)) if (record === undefined) return Promise.resolve() + const inflight = record.inflight + if (inflight !== undefined) { + inflight.cancelRequested = true + inflight.admissionController.abort(new Error('ACP prompt cancelled')) + settleAfterQuiescence(record, inflight) + } record.agent.cancel({ kind: 'user' }) - settlePrompt(record, 'cancelled') return Promise.resolve() }, } @@ -362,10 +441,24 @@ export function apply(ctx: Context, config: AcpConfig): void { // on persistence or scoped cleanup, and the top-level agents must not keep // running model and tool calls for its whole duration. for (const record of records) { + const inflight = record.inflight + if (inflight !== undefined) { + inflight.cancelRequested = true + inflight.admissionController.abort(new Error('ACP bridge disposed')) + settleAfterQuiescence(record, inflight) + } record.agent.cancel({ kind: 'user' }) - settlePrompt(record, 'cancelled') } quiescing = (async () => { + // Preserve the same prompt boundary during connection teardown: a rich + // admission already writing must stop before its slot settles, and every + // committed output conversion must drain while attachment services remain + // available. session/event enqueues output synchronously before idle. + await Promise.all(records.map(async (record) => { + await record.inflight?.admissionDone + await record.agent.whenIdle() + await record.outputTail + })) // Continuable subagents outlive the turn that started them, and their // Activations own descendant teardown. Drain only these sessions' forests // child-first BEFORE disposing the top-level agents, so no descendant is diff --git a/packages/acp/acp/tests/bridge.spec.ts b/packages/acp/acp/tests/bridge.spec.ts index 619a628ea1..2823f717db 100644 --- a/packages/acp/acp/tests/bridge.spec.ts +++ b/packages/acp/acp/tests/bridge.spec.ts @@ -1,5 +1,6 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' +import { AttachmentError } from '@deepseek-ai/dsh-attachment' import { SessionId } from '@deepseek-ai/dsh-session' import { makeBridgeHarness, textResponse, type BridgeHarness } from './harness.ts' @@ -28,6 +29,17 @@ describe('automation-only ACP bridge', () => { }) }) + it('advertises image prompts only with an exact capable route and attachment store', async () => { + harness = await makeBridgeHarness({ imageCapable: true }) + const capable = await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + expect(capable.agentCapabilities?.promptCapabilities?.image).toBe(true) + await harness.dispose() + + harness = await makeBridgeHarness({ imageCapable: true, attachments: false }) + const noStore = await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + expect(noStore.agentCapabilities?.promptCapabilities?.image).toBe(false) + }) + it('negotiates an unsupported version and accepts the required no-op authentication call', async () => { harness = await makeBridgeHarness() const response = await harness.client.initialize({ protocolVersion: 0, clientCapabilities: {} }) @@ -77,6 +89,73 @@ describe('automation-only ACP bridge', () => { expect(harness.adapter.requests[0]?.messages.at(-1)?.content).toEqual([{ type: 'text', text: 'first second' }]) }) + it('admits mixed text/image prompts in wire order and logs references only', async () => { + harness = await makeBridgeHarness({ imageCapable: true, script: [textResponse('done')] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const resolve = vi.spyOn(harness.ctx.llm, 'resolveModelInfo') + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + + await harness.client.prompt({ + sessionId, + prompt: [ + { type: 'text', text: 'before' }, + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + { type: 'text', text: 'between' }, + { type: 'image', data: 'Ag==', mimeType: 'image/jpeg' }, + { type: 'text', text: 'after' }, + ], + }) + + expect(resolve).toHaveBeenCalledWith('mock', 'mock', expect.any(AbortSignal)) + expect(harness.attachments?.saved.map(input => [...input.data])).toEqual([[1], [2]]) + const requestContent = harness.adapter.requests[0]?.messages.at(-1)?.content + expect(requestContent?.map(block => block.type)).toEqual(['text', 'image', 'text', 'image', 'text']) + expect(requestContent?.[0]).toEqual({ type: 'text', text: 'before' }) + expect(requestContent?.[2]).toEqual({ type: 'text', text: 'between' }) + expect(requestContent?.[4]).toEqual({ type: 'text', text: 'after' }) + const firstImage = requestContent?.[1] + const secondImage = requestContent?.[3] + if (firstImage?.type !== 'image' || secondImage?.type !== 'image') throw new Error('expected ordered image blocks') + expect(firstImage.attachment.mediaType).toBe('image/png') + expect(firstImage.attachment.bytes).toBe(1) + expect(secondImage.attachment.mediaType).toBe('image/jpeg') + expect(secondImage.attachment.bytes).toBe(1) + const agent = harness.ctx.agents.get(SessionId(sessionId)) + expect(JSON.stringify(agent?.session.events)).not.toContain('AQ==') + }) + + it('rejects a malformed image batch atomically and frees the prompt slot', async () => { + harness = await makeBridgeHarness({ imageCapable: true, script: [textResponse('recovered')] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + + await expect(harness.client.prompt({ + sessionId, + prompt: [ + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + { type: 'image', data: 'not base64', mimeType: 'image/png' }, + ], + })).rejects.toThrow(/canonical base64/) + expect(harness.attachments?.saved).toEqual([]) + + await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'retry' }] })) + .resolves.toEqual({ stopReason: 'end_turn' }) + }) + + it('reports durable image write failures as internal prompt failures', async () => { + harness = await makeBridgeHarness({ imageCapable: true }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + vi.spyOn(harness.attachments!, 'saveImages').mockRejectedValueOnce( + new AttachmentError('disk failed', 'ATTACHMENT_WRITE_FAILED'), + ) + + await expect(harness.client.prompt({ + sessionId, + prompt: [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }], + })).rejects.toThrow(/unable to persist the prompt image batch/) + }) + it('renders the deployment persona for an ACP-created agent', async () => { harness = await makeBridgeHarness({ persona: 'Automation persona for {{model}} in {{cwd}}.', script: [textResponse('ok')] }) await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) @@ -107,7 +186,7 @@ describe('automation-only ACP bridge', () => { })).resolves.toHaveProperty('sessionId') }) - it('rejects empty and beyond-baseline prompts before a turn starts', async () => { + it('rejects empty and unadvertised image prompts before a turn starts', async () => { harness = await makeBridgeHarness() await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) @@ -117,7 +196,7 @@ describe('automation-only ACP bridge', () => { await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'image', data: '', mimeType: 'image/png' }], - })).rejects.toThrow(/only text and resource_link/) + })).rejects.toThrow(/inline image prompts were not advertised/) expect(harness.ctx.agents.get(SessionId(sessionId))?.session.events.some(event => event.type === 'turn/start')).toBe(false) }) diff --git a/packages/acp/acp/tests/codec.spec.ts b/packages/acp/acp/tests/codec.spec.ts index 335ead9798..2a48336af0 100644 --- a/packages/acp/acp/tests/codec.spec.ts +++ b/packages/acp/acp/tests/codec.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' import type { TurnEndReason } from '@deepseek-ai/dsh-session' -import { acpPromptToText, turnEndToStopReason } from '../src/codec.ts' +import { turnEndToStopReason } from '../src/codec.ts' describe('ACP codec', () => { it.each([ @@ -13,12 +13,4 @@ describe('ACP codec', () => { ] satisfies Array<[TurnEndReason, string]>)('maps %o to %s', (reason, expected) => { expect(turnEndToStopReason(reason)).toBe(expected) }) - - it('drops unsupported blocks from baseline text conversion', () => { - expect(acpPromptToText([{ - type: 'image', - data: '', - mimeType: 'image/png', - }])).toBe('') - }) }) diff --git a/packages/acp/acp/tests/content.spec.ts b/packages/acp/acp/tests/content.spec.ts new file mode 100644 index 0000000000..476708d687 --- /dev/null +++ b/packages/acp/acp/tests/content.spec.ts @@ -0,0 +1,232 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import type { Context } from '@deepseek-ai/cordis' +import { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentRef, SaveImageAttachment } from '@deepseek-ai/dsh-attachment' +import type { Agent } from '@deepseek-ai/dsh-agent' +import { + AcpContentError, + admitAcpPrompt, + assistantBlockToAcp, + supportsAcpImagePrompts, +} from '../src/content.ts' + +const REF: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${'1'.repeat(64)}`), + mediaType: 'image/png', + bytes: 1, + width: 1, + height: 1, +} + +interface AdmissionFixture { + ctx: Context + agent: Agent + saveImages: ReturnType Promise>> + resolveModelInfo: ReturnType +} + +function admissionFixture(options: { + attachments?: boolean + llm?: boolean + provider?: string | undefined + model?: string | undefined + header?: { provider?: string; model?: string } +} = {}): AdmissionFixture { + const saveImages = vi.fn(async (inputs: readonly SaveImageAttachment[]) => inputs.map((input, index) => ({ + ...REF, + attachmentId: AttachmentId(`sha256:${String(index + 1).padStart(64, '0')}`), + mediaType: input.mediaType, + bytes: input.data.byteLength, + }))) + const resolveModelInfo = vi.fn(async (provider: string, model: string) => ({ + provider, + id: model, + name: model, + inputModalities: ['text', 'image'] as const, + })) + const attachments = options.attachments === false ? undefined : { saveImages } + const llm = options.llm === false ? undefined : { resolveModelInfo } + const ctx = { + get(name: string) { + if (name === 'attachments') return attachments + if (name === 'llm') return llm + return undefined + }, + } as unknown as Context + const provider = 'provider' in options ? options.provider : 'mock' + const model = 'model' in options ? options.model : 'vision' + const agent = { + options: { provider, model }, + session: { requestHeader: () => options.header === undefined ? undefined : { config: options.header } }, + } as unknown as Agent + return { ctx, agent, saveImages, resolveModelInfo } +} + +describe('ACP rich content codec', () => { + afterEach(() => { + vi.restoreAllMocks() + }) + + it('advertises image input only when every deployment prerequisite is explicit', async () => { + const absent = (attachments: unknown, llm: unknown): Context => ({ + get: (name: string) => name === 'attachments' ? attachments : name === 'llm' ? llm : undefined, + }) as unknown as Context + const store = { imageLimits: { mediaTypes: ['image/png'] } } + const noMediaStore = { imageLimits: { mediaTypes: [] } } + const imageLlm = { resolveModelInfo: vi.fn().mockResolvedValue({ inputModalities: ['text', 'image'] }) } + const textLlm = { resolveModelInfo: vi.fn().mockResolvedValue({ inputModalities: ['text'] }) } + const unknownLlm = { resolveModelInfo: vi.fn().mockResolvedValue({}) } + const brokenLlm = { resolveModelInfo: vi.fn().mockRejectedValue(new Error('catalog down')) } + + await expect(supportsAcpImagePrompts(absent(undefined, imageLlm), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, undefined), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, imageLlm), undefined, 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, imageLlm), 'p', undefined)).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(noMediaStore, imageLlm), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, brokenLlm), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, unknownLlm), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, textLlm), 'p', 'm')).resolves.toBe(false) + await expect(supportsAcpImagePrompts(absent(store, imageLlm), 'p', 'm')).resolves.toBe(true) + }) + + it('validates every rich wire block before any image write', async () => { + const fixture = admissionFixture() + const signal = new AbortController().signal + + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'image', data: 'AQ==', mimeType: 'image/tiff' }, + ] as never, true, signal)).rejects.toThrow(/mimeType/) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'image', data: 'not base64', mimeType: 'image/png' }, + ], true, signal)).rejects.toThrow(/canonical base64/) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'image', data: 'AB==', mimeType: 'image/png' }, + ], true, signal)).rejects.toThrow(/canonical base64/) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'audio', data: 'AQ==', mimeType: 'audio/wav' }, + ], true, signal)).rejects.toThrow(/audio prompt/) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'resource', resource: { uri: 'file:///tmp/a', text: 'a' } }, + ], true, signal)).rejects.toThrow(/embedded resource/) + expect(fixture.saveImages).not.toHaveBeenCalled() + }) + + it('requires the advertised capability, store, and exact image-capable route', async () => { + const prompt = [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }] as const + const capable = admissionFixture() + await expect(admitAcpPrompt(capable.ctx, capable.agent, prompt, false, new AbortController().signal)) + .rejects.toThrow(/not advertised/) + + const noStore = admissionFixture({ attachments: false }) + await expect(admitAcpPrompt(noStore.ctx, noStore.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/no attachment store/) + + const noProvider = admissionFixture({ provider: undefined }) + await expect(admitAcpPrompt(noProvider.ctx, noProvider.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/route could not be resolved/) + const noModel = admissionFixture({ model: undefined }) + await expect(admitAcpPrompt(noModel.ctx, noModel.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/route could not be resolved/) + const noLlm = admissionFixture({ llm: false }) + await expect(admitAcpPrompt(noLlm.ctx, noLlm.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/route could not be resolved/) + + const broken = admissionFixture() + broken.resolveModelInfo.mockRejectedValueOnce(new Error('catalog down')) + await expect(admitAcpPrompt(broken.ctx, broken.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/route could not be verified/) + const unknown = admissionFixture() + unknown.resolveModelInfo.mockResolvedValueOnce({ provider: 'mock', id: 'vision', name: 'vision' }) + await expect(admitAcpPrompt(unknown.ctx, unknown.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/does not declare image input/) + const textOnly = admissionFixture() + textOnly.resolveModelInfo.mockResolvedValueOnce({ + provider: 'mock', id: 'vision', name: 'vision', inputModalities: ['text'], + }) + await expect(admitAcpPrompt(textOnly.ctx, textOnly.agent, prompt, true, new AbortController().signal)) + .rejects.toThrow(/does not declare image input/) + + const routed = admissionFixture({ provider: 'fallback', model: 'fallback', header: { provider: 'live', model: 'vision-2' } }) + await expect(admitAcpPrompt(routed.ctx, routed.agent, prompt, true, new AbortController().signal)).resolves.toHaveLength(1) + expect(routed.resolveModelInfo).toHaveBeenCalledWith('live', 'vision-2', expect.any(AbortSignal)) + }) + + it('classifies image-policy failures separately from durable write failures', async () => { + const fixture = admissionFixture() + const prompt = [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }] as const + fixture.saveImages.mockRejectedValueOnce(new AttachmentError('too many', 'TOO_MANY_IMAGES')) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, prompt, true, new AbortController().signal)) + .rejects.toMatchObject({ kind: 'invalid', message: 'too many' }) + fixture.saveImages.mockRejectedValueOnce(new AttachmentError('disk failed', 'ATTACHMENT_WRITE_FAILED')) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, prompt, true, new AbortController().signal)) + .rejects.toMatchObject({ kind: 'internal', message: 'unable to persist the prompt image batch' }) + fixture.saveImages.mockRejectedValueOnce(new Error('unknown store failure')) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, prompt, true, new AbortController().signal)) + .rejects.toBeInstanceOf(AcpContentError) + }) + + it('honors cancellation on both sides of the durable image write', async () => { + const prompt = [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }] as const + const before = admissionFixture() + const beforeController = new AbortController() + beforeController.abort(new Error('cancel before write')) + await expect(admitAcpPrompt(before.ctx, before.agent, prompt, true, beforeController.signal)) + .rejects.toThrow('cancel before write') + expect(before.saveImages).not.toHaveBeenCalled() + + const after = admissionFixture() + const afterController = new AbortController() + after.saveImages.mockImplementationOnce(async () => { + afterController.abort(new Error('cancel after write')) + return [REF] + }) + await expect(admitAcpPrompt(after.ctx, after.agent, prompt, true, afterController.signal)) + .rejects.toThrow('cancel after write') + expect(after.saveImages).toHaveBeenCalledOnce() + }) + + it('reconstructs image-only and baseline prompts without empty text blocks', async () => { + const fixture = admissionFixture() + const imageOnly = await admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + ], true, new AbortController().signal) + expect(imageOnly).toHaveLength(1) + expect(imageOnly[0]?.type).toBe('image') + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'text', text: 'before' }, + { type: 'resource_link', name: 'Guide', uri: 'https://example.test/guide' }, + { type: 'text', text: 'after' }, + ], true, new AbortController().signal)).resolves.toEqual([{ + type: 'text', + text: 'before\n[resource_link name="Guide" uri="https://example.test/guide"]\nafter', + }]) + await expect(admitAcpPrompt(fixture.ctx, fixture.agent, [ + { type: 'text', text: ' \n ' }, + ], true, new AbortController().signal)).rejects.toThrow(/empty prompt/) + }) + + it('projects only non-empty text and verified durable images to ACP', async () => { + const fixture = admissionFixture() + await expect(assistantBlockToAcp(fixture.ctx, { type: 'text', text: '' })).resolves.toBeUndefined() + await expect(assistantBlockToAcp(fixture.ctx, { type: 'text', text: 'hello' })).resolves.toEqual({ + type: 'text', text: 'hello', + }) + await expect(assistantBlockToAcp(fixture.ctx, { type: 'reasoning', text: 'private' })).resolves.toBeUndefined() + + const noStore = admissionFixture({ attachments: false }) + await expect(assistantBlockToAcp(noStore.ctx, { type: 'image', attachment: REF })) + .rejects.toThrow(/no attachment store/) + const readImage = vi.fn().mockRejectedValue(new AttachmentError('gone', 'ATTACHMENT_NOT_FOUND')) + const missingCtx = { get: (name: string) => name === 'attachments' ? { readImage } : undefined } as unknown as Context + await expect(assistantBlockToAcp(missingCtx, { type: 'image', attachment: REF })) + .rejects.toThrow(/unavailable or corrupt/) + const storedCtx = { + get: (name: string) => name === 'attachments' + ? { readImage: vi.fn().mockResolvedValue({ ref: REF, data: Uint8Array.of(1) }) } + : undefined, + } as unknown as Context + await expect(assistantBlockToAcp(storedCtx, { type: 'image', attachment: REF })).resolves.toEqual({ + type: 'image', data: 'AQ==', mimeType: 'image/png', + }) + }) +}) diff --git a/packages/acp/acp/tests/dispose.spec.ts b/packages/acp/acp/tests/dispose.spec.ts index 48b5e76095..4aa32f078c 100644 --- a/packages/acp/acp/tests/dispose.spec.ts +++ b/packages/acp/acp/tests/dispose.spec.ts @@ -1,6 +1,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' import type { Agent } from '@deepseek-ai/dsh-agent' +import type { StreamChunk } from '@deepseek-ai/dsh-llm' import { SessionId } from '@deepseek-ai/dsh-session' import { makeBridgeHarness, type BridgeHarness } from './harness.ts' @@ -26,6 +27,37 @@ describe('ACP connection ownership', () => { expect(harness.ctx.agents.get(SessionId(sessionId))).toBeUndefined() }) + it('disposal drains asynchronous assistant image delivery before releasing sessions', async () => { + const script: StreamChunk[][] = [] + harness = await makeBridgeHarness({ script }) + const ref = await harness.attachments!.saveImage({ data: Uint8Array.of(4), mediaType: 'image/png' }) + script.push([ + { type: 'block-start', index: 0, blockType: 'image' }, + { type: 'block-end', index: 0, block: { type: 'image', attachment: ref } }, + { type: 'finish', reason: { kind: 'stop' } }, + ]) + const readStarted = Promise.withResolvers() + const releaseRead = Promise.withResolvers() + harness.attachments!.beforeRead = () => { + readStarted.resolve(undefined) + return releaseRead.promise + } + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const prompt = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'show it' }] }) + await readStarted.promise + + let disposed = false + const disposal = harness.acpFiber.dispose().finally(() => { disposed = true }) + await Promise.resolve() + expect(disposed).toBe(false) + + releaseRead.resolve(undefined) + await disposal + await expect(prompt).resolves.toEqual({ stopReason: 'cancelled' }) + expect(harness.ctx.agents.get(SessionId(sessionId))).toBeUndefined() + }) + it('drains continuable subagents before disposing its own sessions', async () => { harness = await makeBridgeHarness() const order: string[] = [] diff --git a/packages/acp/acp/tests/edges.spec.ts b/packages/acp/acp/tests/edges.spec.ts index cdb5764b53..84bbff3b3d 100644 --- a/packages/acp/acp/tests/edges.spec.ts +++ b/packages/acp/acp/tests/edges.spec.ts @@ -54,6 +54,51 @@ describe('ACP automation output boundary', () => { expect(harness.updates).toHaveLength(0) }) + it('delivers output from a bridge-owned session driven by another in-process producer', async () => { + harness = await makeBridgeHarness({ script: [textResponse('external')] }) + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(SessionId(sessionId))! + + agent.followup(createUserMessage({ content: [{ type: 'text', text: 'go' }], source: { kind: 'plugin', plugin: 'test' } })) + await agent.whenIdle() + + expect(harness.updates).toEqual([{ + sessionUpdate: 'agent_message_chunk', + content: { type: 'text', text: 'external' }, + }]) + }) + + it('contains output conversion failure outside an ACP prompt', async () => { + harness = await makeBridgeHarness({ script: [[ + { type: 'block-start', index: 0, blockType: 'image' }, + { + type: 'block-end', + index: 0, + block: { + type: 'image', + attachment: { + attachmentId: `sha256:${'a'.repeat(64)}` as never, + mediaType: 'image/png', + bytes: 1, + width: 1, + height: 1, + }, + }, + }, + { type: 'finish', reason: { kind: 'stop' } }, + ]] }) + const warn = vi.spyOn(harness.ctx.logger, 'warn') + await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) + const { sessionId } = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) + const agent = harness.ctx.agents.get(SessionId(sessionId))! + + agent.followup(createUserMessage({ content: [{ type: 'text', text: 'go' }], source: { kind: 'plugin', plugin: 'test' } })) + await agent.whenIdle() + await vi.waitFor(() => { expect(warn).toHaveBeenCalledWith(expect.stringContaining('output conversion failed')) }) + expect(harness.updates).toEqual([]) + }) + // `session/update` is a JSON-RPC notification, so a client-side handler // failure never reaches the bridge; this pins that the prompt still settles // normally with such a client. The bridge's own write-failure guard is diff --git a/packages/acp/acp/tests/harness.ts b/packages/acp/acp/tests/harness.ts index c5b03c39a2..ae66f9f841 100644 --- a/packages/acp/acp/tests/harness.ts +++ b/packages/acp/acp/tests/harness.ts @@ -1,6 +1,7 @@ /** In-memory ACP transport fixture over the real agent factory and loop. */ import { Context } from '@deepseek-ai/cordis' +import { createHash } from 'node:crypto' import { ClientSideConnection, ndJsonStream, @@ -11,7 +12,9 @@ import { type SessionNotification, type Stream, } from '@agentclientprotocol/sdk' -import { type GenerateOptions, LlmAdapter, type StreamChunk } from '@deepseek-ai/dsh-llm' +import AttachmentStore, { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment' +import type { ImageAttachmentLimits, ImageAttachmentRef, SaveImageAttachment, StoredImageAttachment } from '@deepseek-ai/dsh-attachment' +import { type GenerateOptions, LlmAdapter, type LlmResolvedModelInfo, type StreamChunk } from '@deepseek-ai/dsh-llm' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit' import * as AcpPlugin from '../src/index.ts' @@ -21,7 +24,10 @@ import type { AcpConfig } from '../src/index.ts' class MockAdapter extends LlmAdapter { readonly requests: GenerateOptions[] = [] - constructor(private readonly script: (StreamChunk[] | 'hang')[]) { + constructor( + private readonly script: (StreamChunk[] | 'hang')[], + private readonly imageCapable: boolean, + ) { super() } @@ -31,7 +37,21 @@ class MockAdapter extends LlmAdapter { } override listModels(provider: string) { - return Promise.resolve(provider === 'mock' ? [{ provider: 'mock', id: 'mock', name: 'Mock' }] : []) + return Promise.resolve(provider === 'mock' ? [{ + provider: 'mock', + id: 'mock', + name: 'Mock', + inputModalities: this.imageCapable ? ['text', 'image'] as const : ['text'] as const, + }] : []) + } + + override resolveModel(provider: string, model: string): Promise { + return Promise.resolve({ + provider, + id: model, + name: model, + inputModalities: this.imageCapable ? ['text', 'image'] : ['text'], + }) } async * stream(options: GenerateOptions): AsyncIterable { @@ -57,6 +77,49 @@ class MockAdapter extends LlmAdapter { } } +const IMAGE_LIMITS: ImageAttachmentLimits = { + maxImageBytes: 1024, + maxImagesPerMessage: 4, + maxMessageImageBytes: 2048, + maxImagePixels: 1024, + mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], +} + +/** In-memory durable store for ACP wire-order and lifecycle tests. */ +class MemoryAttachmentStore extends AttachmentStore { + readonly imageLimits = IMAGE_LIMITS + readonly saved: SaveImageAttachment[] = [] + readonly objects = new Map() + beforeValidate: (() => Promise) | undefined + beforeRead: (() => Promise) | undefined + + async validateImage(input: SaveImageAttachment): Promise { + await this.beforeValidate?.() + if (input.data.byteLength === 0) throw new AttachmentError('Image is empty.', 'INVALID_IMAGE') + } + + saveImage(input: SaveImageAttachment): Promise { + this.saved.push(input) + const digest = createHash('sha256').update(input.data).digest('hex') + const ref: ImageAttachmentRef = { + attachmentId: AttachmentId(`sha256:${digest}`), + mediaType: input.mediaType, + bytes: input.data.byteLength, + width: 1, + height: 1, + } + this.objects.set(ref.attachmentId, { ref, data: Uint8Array.from(input.data) }) + return Promise.resolve(ref) + } + + async readImage(ref: ImageAttachmentRef): Promise { + await this.beforeRead?.() + const stored = this.objects.get(ref.attachmentId) + if (stored === undefined) throw new AttachmentError('Attachment object is missing.', 'ATTACHMENT_NOT_FOUND') + return { ref: stored.ref, data: Uint8Array.from(stored.data) } + } +} + /** Scripted text response ending in a clean stop. */ export function textResponse(text: string): StreamChunk[] { return [ @@ -93,6 +156,7 @@ export interface BridgeHarness { ctx: Context client: ClientSideConnection adapter: MockAdapter + attachments: MemoryAttachmentStore | undefined updates: CapturedUpdate[] sessionUpdates: { sessionId: string; update: CapturedUpdate }[] permissionRequests: RequestPermissionRequest[] @@ -113,10 +177,13 @@ export async function makeBridgeHarness(options: { script?: (StreamChunk[] | 'hang')[] config?: AcpConfigOverrides persona?: string + imageCapable?: boolean + attachments?: boolean } = {}): Promise { - const adapter = new MockAdapter(options.script ?? []) + const adapter = new MockAdapter(options.script ?? [], options.imageCapable === true) const ctx = new Context() await mountAgentLoopTestDependencies(ctx, { systemPrompt: { persona: options.persona ?? '' } }) + if (options.attachments !== false) await ctx.plugin(MemoryAttachmentStore) const loopFiber = await ctx.plugin(AgentLoop, { agents: [] }) ctx.llm.registerAdapter(['mock'], adapter) @@ -135,6 +202,7 @@ export async function makeBridgeHarness(options: { const harness: BridgeHarness = { ctx, adapter, + attachments: ctx.get('attachments') as MemoryAttachmentStore | undefined, updates, sessionUpdates, permissionRequests, diff --git a/packages/acp/acp/tests/turns.spec.ts b/packages/acp/acp/tests/turns.spec.ts index 848d112215..e11023ce46 100644 --- a/packages/acp/acp/tests/turns.spec.ts +++ b/packages/acp/acp/tests/turns.spec.ts @@ -1,4 +1,4 @@ -import { createUserMessage } from '@deepseek-ai/dsh-llm' +import { createUserMessage, type StreamChunk } from '@deepseek-ai/dsh-llm' import { afterEach, describe, expect, it, vi } from 'vitest' import { PROTOCOL_VERSION } from '@agentclientprotocol/sdk' import { SessionId } from '@deepseek-ai/dsh-session' @@ -41,35 +41,100 @@ describe('ACP prompt lifecycle', () => { await vi.waitFor(() => { expect(messageText(harness!)).toBe('cut off') }) }) - it('renders an assistant image as an explicit attachment placeholder', async () => { - const attachmentId = `sha256:${'a'.repeat(64)}` as never - harness = await makeBridgeHarness({ - script: [[ - { type: 'block-start', index: 0, blockType: 'image' }, - { - type: 'block-end', - index: 0, - block: { - type: 'image', - attachment: { - attachmentId, - mediaType: 'image/png', - bytes: 1, - width: 1, - height: 1, - }, - }, + it('delivers a committed assistant image as verified ACP base64', async () => { + const script: StreamChunk[][] = [] + harness = await makeBridgeHarness({ script }) + const ref = await harness.attachments!.saveImage({ data: Uint8Array.of(1), mediaType: 'image/png' }) + script.push([ + { type: 'block-start', index: 0, blockType: 'image' }, + { + type: 'block-end', + index: 0, + block: { + type: 'image', + attachment: ref, }, - { type: 'finish', reason: { kind: 'stop' } }, - ]], - }) + }, + { type: 'finish', reason: { kind: 'stop' } }, + ]) const sessionId = await newSession(harness) await harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'show it' }] }) - await vi.waitFor(() => { - expect(messageText(harness!)).toBe(`[image attachment ${String(attachmentId)}]`) + expect(harness.updates).toContainEqual({ + sessionUpdate: 'agent_message_chunk', + content: { type: 'image', data: 'AQ==', mimeType: 'image/png' }, }) }) + it('preserves committed text/image/text order on the ACP wire', async () => { + const script: StreamChunk[][] = [] + harness = await makeBridgeHarness({ script }) + const ref = await harness.attachments!.saveImage({ data: Uint8Array.of(2), mediaType: 'image/jpeg' }) + script.push([ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'block-end', index: 0, block: { type: 'text', text: 'before' } }, + { type: 'block-start', index: 1, blockType: 'image' }, + { type: 'block-end', index: 1, block: { type: 'image', attachment: ref } }, + { type: 'block-start', index: 2, blockType: 'text' }, + { type: 'block-end', index: 2, block: { type: 'text', text: 'after' } }, + { type: 'finish', reason: { kind: 'stop' } }, + ]) + const sessionId = await newSession(harness) + + await harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'show it' }] }) + + expect(harness.updates).toEqual([ + { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'before' } }, + { sessionUpdate: 'agent_message_chunk', content: { type: 'image', data: 'Ag==', mimeType: 'image/jpeg' } }, + { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'after' } }, + ]) + }) + + it('does not settle a prompt before ordered output delivery drains', async () => { + const script: StreamChunk[][] = [] + harness = await makeBridgeHarness({ script }) + const ref = await harness.attachments!.saveImage({ data: Uint8Array.of(3), mediaType: 'image/png' }) + script.push([ + { type: 'block-start', index: 0, blockType: 'image' }, + { type: 'block-end', index: 0, block: { type: 'image', attachment: ref } }, + { type: 'finish', reason: { kind: 'stop' } }, + ]) + const readStarted = Promise.withResolvers() + const delivery = Promise.withResolvers() + harness.attachments!.beforeRead = () => { + readStarted.resolve(undefined) + return delivery.promise + } + const sessionId = await newSession(harness) + let settled = false + + const prompt = harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] }) + .finally(() => { settled = true }) + await readStarted.promise + expect(settled).toBe(false) + delivery.resolve(undefined) + await expect(prompt).resolves.toEqual({ stopReason: 'end_turn' }) + }) + + it('fails prompt delivery when a committed image attachment is missing', async () => { + const missing = { + attachmentId: `sha256:${'a'.repeat(64)}` as never, + mediaType: 'image/png' as const, + bytes: 1, + width: 1, + height: 1, + } + harness = await makeBridgeHarness({ script: [[ + { type: 'block-start', index: 0, blockType: 'image' }, + { type: 'block-end', index: 0, block: { type: 'image', attachment: missing } }, + { type: 'finish', reason: { kind: 'stop' } }, + ]] }) + const sessionId = await newSession(harness) + + await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'show it' }] })) + .rejects.toThrow(/assistant output delivery failed/) + expect(harness.updates).toEqual([]) + }) + it('rejects a failed turn and never publishes its partial chunks', async () => { harness = await makeBridgeHarness({ script: [errorResponse('provider boom')] }) const sessionId = await newSession(harness) @@ -194,6 +259,84 @@ describe('ACP prompt lifecycle', () => { await expect(first).resolves.toEqual({ stopReason: 'cancelled' }) }) + it('reserves the prompt slot during image admission and cancels without a late followup', async () => { + harness = await makeBridgeHarness({ imageCapable: true, script: [] }) + const validationStarted = Promise.withResolvers() + const releaseValidation = Promise.withResolvers() + harness.attachments!.beforeValidate = () => { + validationStarted.resolve(undefined) + return releaseValidation.promise + } + const sessionId = await newSession(harness) + let settled = false + const first = harness.client.prompt({ + sessionId, + prompt: [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }], + }).finally(() => { settled = true }) + await validationStarted.promise + + await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'second' }] })) + .rejects.toThrow(/already in flight/) + await harness.client.cancel({ sessionId }) + expect(settled).toBe(false) + releaseValidation.resolve(undefined) + + await expect(first).resolves.toEqual({ stopReason: 'cancelled' }) + expect(harness.adapter.requests).toEqual([]) + const events = harness.ctx.agents.get(SessionId(sessionId))?.session.events ?? [] + expect(events.some(event => event.type === 'user/message' || event.type === 'turn/start')).toBe(false) + }) + + it('does not queue admitted content into an agent retired during storage', async () => { + harness = await makeBridgeHarness({ imageCapable: true, script: [] }) + const validationStarted = Promise.withResolvers() + const releaseValidation = Promise.withResolvers() + harness.attachments!.beforeValidate = () => { + validationStarted.resolve(undefined) + return releaseValidation.promise + } + const sessionId = await newSession(harness) + const prompt = harness.client.prompt({ + sessionId, + prompt: [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }], + }) + await validationStarted.promise + + await harness.loopFiber.dispose() + releaseValidation.resolve(undefined) + + await expect(prompt).rejects.toThrow(/disposed outside the bridge/) + expect(harness.attachments!.saved).toHaveLength(1) + expect(harness.adapter.requests).toEqual([]) + }) + + it('honors cancellation in the admission-to-followup handoff gap', async () => { + harness = await makeBridgeHarness({ imageCapable: true, script: [] }) + const sessionId = await newSession(harness) + const saveImages = harness.attachments!.saveImages.bind(harness.attachments!) + vi.spyOn(harness.attachments!, 'saveImages').mockImplementationOnce(async (inputs) => { + const refs = await saveImages(inputs) + queueMicrotask(() => { void harness!.client.cancel({ sessionId }) }) + return refs + }) + + await expect(harness.client.prompt({ + sessionId, + prompt: [{ type: 'image', data: 'AQ==', mimeType: 'image/png' }], + })).resolves.toEqual({ stopReason: 'cancelled' }) + expect(harness.adapter.requests).toEqual([]) + }) + + it('wraps an unexpected same-process followup failure and frees the prompt slot', async () => { + harness = await makeBridgeHarness({ script: [] }) + const sessionId = await newSession(harness) + const agent = harness.ctx.agents.get(SessionId(sessionId))! + vi.spyOn(agent, 'followup').mockImplementationOnce(() => { throw new Error('synthetic followup failure') }) + + await expect(harness.client.prompt({ sessionId, prompt: [{ type: 'text', text: 'go' }] })) + .rejects.toThrow(/prompt was not queued: synthetic followup failure/) + }) + it('cancels a running turn and records the aborted outcome', async () => { harness = await makeBridgeHarness({ script: ['hang'] }) const sessionId = await newSession(harness) diff --git a/packages/support/acp-snapshot/README.i18n.yaml b/packages/support/acp-snapshot/README.i18n.yaml index ca9e9e67f0..3afae2d055 100644 --- a/packages/support/acp-snapshot/README.i18n.yaml +++ b/packages/support/acp-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md -README.md: 06f1cb67cfcd954254db480ea696d10d81b37438 -README.zh.md: c4a5643f8c5d7e5b62a160cd32e5969187d34046 +README.md: 31f4ec0caeb995a10202d4a452ee7e433749762f +README.zh.md: f97bac11de690fe980595c77375aead47b3b0214 diff --git a/packages/support/acp-snapshot/README.md b/packages/support/acp-snapshot/README.md index 06f1cb67cf..31f4ec0cae 100644 --- a/packages/support/acp-snapshot/README.md +++ b/packages/support/acp-snapshot/README.md @@ -61,7 +61,7 @@ Every scenario compares `stdout.expected.jsonl` with cwd-rooted separators canon The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md). -Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the JSON-RPC and Web snapshot recorders. Input scripts cover initialization, fresh-session creation, text prompting, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. +Constraints: `suite.ts` and `harness.ts` import vitest (the harness polls its durable-boundary waits through `vi.waitFor`), so the package entry is importable only inside a vitest run (the launcher and normalizers have no such dependency but ship from the same entry). The launcher and suite factory are ACP-specific by design — the launcher speaks the SDK's `ClientSideConnection` — while the normalizers are transport-neutral session-log/text helpers also consumed by the JSON-RPC and Web snapshot recorders. Input scripts cover initialization, fresh-session creation, shorthand text prompts, exact structured ACP prompt blocks, cancellation, expected RPC failures, and durable turn-boundary waits. Permission round-trips are a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) mapped to the agent-issued `optionId`; an absent or exhausted queue answers `cancelled`, and an unoffered kind rejects the run. ## Model Experience diff --git a/packages/support/acp-snapshot/README.zh.md b/packages/support/acp-snapshot/README.zh.md index c4a5643f8c..f97bac11de 100644 --- a/packages/support/acp-snapshot/README.zh.md +++ b/packages/support/acp-snapshot/README.zh.md @@ -61,7 +61,7 @@ defineAcpSnapshotSuite({ 示例还发布 `cordis.snapshot.yml` 回放 overlay,位于 `cordis.yml` 旁边(bin 在 `DSH_SNAPSHOT=replay` 下交换它们,见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md));回放 fixture 由 [`dsh-llm-replay`](../llm-replay/README.md) 提供,该包通过对子级设置的 `DSH_SNAPSHOT_*` env var 指向它。`pnpm run test:snapshot:record` 调用实时 LLM,并重写已记录场景的模型 fixture;`pnpm run test:snapshot:refresh` 保持无密钥,运行回放 overlay,并从已提交模型脚本重写 stdout、可比较会话日志预期输出,以及各 pin 自有的提示词与工具 schema sidecar。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md) 中。 -约束:`suite.ts` 与 `harness.ts` 导入 vitest(harness 通过 `vi.waitFor` 轮询其持久边界等待),因此包入口只能在 vitest 运行中导入(启动器和规范化器没有此依赖,但从同一入口发布)。启动器和套件工厂按设计专用于 ACP,启动器使用 SDK 的 `ClientSideConnection`;规范化器是与传输无关的会话日志/文本辅助工具,还由 JSON-RPC 和 Web 快照录制器消费。输入脚本覆盖初始化、新建会话、文本提示、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择(`allow_once`、`reject_once` 等)的 FIFO 队列,映射到 agent 发出的 `optionId`;缺少或耗尽的队列回答 `cancelled`,未提供类别会拒绝运行。 +约束:`suite.ts` 与 `harness.ts` 导入 vitest(harness 通过 `vi.waitFor` 轮询其持久边界等待),因此包入口只能在 vitest 运行中导入(启动器和规范化器没有此依赖,但从同一入口发布)。启动器和套件工厂按设计专用于 ACP,启动器使用 SDK 的 `ClientSideConnection`;规范化器是与传输无关的会话日志/文本辅助工具,还由 JSON-RPC 和 Web 快照录制器消费。输入脚本覆盖初始化、新建会话、文本提示简写、精确结构化 ACP 提示词块、取消、预期 RPC 失败和持久轮次边界等待。权限往返是选项类别选择(`allow_once`、`reject_once` 等)的 FIFO 队列,映射到 agent 发出的 `optionId`;缺少或耗尽的队列回答 `cancelled`,未提供类别会拒绝运行。 ## 模型体验 diff --git a/packages/support/acp-snapshot/src/harness.ts b/packages/support/acp-snapshot/src/harness.ts index d37bf289b5..21800862fc 100644 --- a/packages/support/acp-snapshot/src/harness.ts +++ b/packages/support/acp-snapshot/src/harness.ts @@ -25,6 +25,7 @@ import { vi } from 'vitest' import { ClientSideConnection, PROTOCOL_VERSION, + type ContentBlock as AcpContentBlock, type RequestPermissionRequest, type RequestPermissionResponse, type SessionNotification, @@ -69,6 +70,7 @@ export type InputStep = | { op: 'newSession' } | { op: 'newSessionExpectError'; additionalDirectories?: string[] } | { op: 'prompt'; text: string } + | { op: 'promptContent'; content: AcpContentBlock[] } | { op: 'promptAndWaitForAgentMessage'; text: string; waitForText: string } | { op: 'promptExpectError'; text: string } | { @@ -422,6 +424,12 @@ async function runStep( await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] }) return } + case 'promptContent': { + const sessionId = getSessionId() + if (sessionId === undefined) throw new Error('snapshot-harness: promptContent before newSession') + await client.prompt({ sessionId, prompt: step.content }) + return + } case 'promptAndWaitForAgentMessage': { const sessionId = getSessionId() if (sessionId === undefined) throw new Error('snapshot-harness: promptAndWaitForAgentMessage before newSession') diff --git a/packages/support/acp-snapshot/tests/harness.spec.ts b/packages/support/acp-snapshot/tests/harness.spec.ts index 68dc58c770..5bd97b101d 100644 --- a/packages/support/acp-snapshot/tests/harness.spec.ts +++ b/packages/support/acp-snapshot/tests/harness.spec.ts @@ -407,6 +407,24 @@ describe('runScenario', () => { expect((JSON.parse(sessionLine) as { cwd?: string }).cwd).toBe(result.cwd) }) + it('drives a structured prompt-content step without flattening its wire blocks', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({}) + const result = await runScenario( + { + steps: [...boot, { + op: 'promptContent', + content: [ + { type: 'text', text: 'before' }, + { type: 'image', data: 'AQ==', mimeType: 'image/png' }, + { type: 'text', text: 'after' }, + ], + }], + }, + { agent: AGENT, mode: 'replay', fixtureFile }, + ) + expect(result.rawStdout).toContain('"stopReason":"end_turn"') + }) + it('forwards override/child fixture paths into the child env and captures stderr', { timeout: 20_000 }, async () => { const { dir, fixtureFile } = await scenario({ echoEnv: true, stderrNote: 'fake bin booted' }) const childFiles = [join(dir, 'session.1.jsonl'), join(dir, 'session.2.jsonl')] @@ -1097,6 +1115,7 @@ describe('runScenario', () => { it.each([ [{ op: 'prompt', text: 'x' }, /prompt before newSession/], + [{ op: 'promptContent', content: [{ type: 'text', text: 'x' }] }, /promptContent before newSession/], [{ op: 'promptAndWaitForAgentMessage', text: 'x', waitForText: 'later' }, /promptAndWaitForAgentMessage before newSession/], [{ op: 'promptExpectError', text: 'x' }, /promptExpectError before newSession/], [{ op: 'promptAndCancel', text: 'x' }, /promptAndCancel before newSession/], diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 8bfc8937e9..a2d0633514 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -778,6 +778,9 @@ importers: '@deepseek-ai/dsh-agent-loop-testkit': specifier: workspace:^ version: link:../../support/agent-loop-testkit + '@deepseek-ai/dsh-attachment': + specifier: workspace:^ + version: link:../../attachment/attachment '@deepseek-ai/dsh-invariants': specifier: workspace:^ version: link:../../support/invariants