Merge pull request #2252 from deepseek-harness/worktree/fix-1463-rich-content-bridge
fix: bridge durable image content across MCP and ACP
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-07-mcp-client-plugin.md
|
||||
2026-07-07-mcp-client-plugin.md: 84fbda6f7832d82854144d8e3687bfe23fec6fa4
|
||||
2026-07-07-mcp-client-plugin.zh.md: 3270803f7cb2cea08e170fdeb84c253c97e16a8e
|
||||
2026-07-07-mcp-client-plugin.md: f9d997fd06dbf14f86ec344a2d5f16c7412119d0
|
||||
2026-07-07-mcp-client-plugin.zh.md: a4b89a948c14d564d2fc37bee6f91fb923e9ed49
|
||||
|
||||
@@ -141,11 +141,11 @@ Tools are never silently skipped; which tools are available never depends on plu
|
||||
A unified `execute` handler for all tools from one MCP server:
|
||||
|
||||
1. Resolve `rawName` (the executor closes over it) and call `client.callTool({ name: rawName, arguments }, { signal: exec.signal })` with the configured timeout — the public name is never sent to the server.
|
||||
2. Map the result:
|
||||
- Multiple `text` content blocks → join with `'\n'` into a single `TextBlock` (required: `flattenText` uses `join('')` without separator, so multiple blocks would lose inter-block boundaries).
|
||||
- `image` content blocks → discard with a `ctx.logger.warn` (the harness has no image content block type; [drop-image Agent Note](../simplification/2026-07-04-drop-image-content-block.md)).
|
||||
- `isError: true` → map to the harness `isError` result path (`{ content: [...], isError: true }`).
|
||||
3. Cancellation: `exec.signal` (from the agent loop's cancel) is passed through to the MCP SDK's `callTool`, which sends `$/cancelRequest` to the server.
|
||||
2. Preserve canonical success as `{ content: JsonValue[], structuredContent? }`; complete MCP JSON blocks remain the programmatic/Code Mode value. `isError: true` throws before any image persistence so the registry owns the failure path.
|
||||
3. Prepare a separate ordered Native projection. Text runs join with `'\n'`; resource links preserve name and URI as text; audio, embedded resources, malformed blocks, and unknown types become explicit diagnostics. If any image exists, the bridge strictly decodes the complete batch, resolves the calling agent's latest exact route, requires an attachment store plus explicit model image input, and delegates all-member validation and ordered persistence to `AttachmentStore.saveImages()`. Any decode, capability, or storage refusal renders every image as diagnostic text and returns no partial references.
|
||||
4. Keep `output.render` synchronous and pure. The executor stages its richer projection in a generation-local `WeakMap` keyed by the exact execution; `finalizeContent` installs it only when the registry's post-execute result still has the original canonical value and fallback content. A policy block, value replacement, or content replacement remains authoritative, and a re-sync cannot let an older generation consume new execution state.
|
||||
5. Code Mode receives the untouched canonical value. Its generic dispatch bridge defers a successful final content sequence containing an image through the outer `run_code` result, so MCP requires no private parent-token special case.
|
||||
6. Cancellation: `exec.signal` (from the agent loop's cancel) is passed through to the MCP SDK's `callTool`, exact-model lookup, and the pre-storage gate.
|
||||
|
||||
### Subprocess environment (stdio transport)
|
||||
|
||||
@@ -189,13 +189,25 @@ Rejected. The remote name is untrusted, non-unique across deployments, and chang
|
||||
|
||||
Rejected. `flattenText()` in the DeepSeek serializer uses `join('')` (no separator) when flattening `ContentBlock[]` to wire format. Multiple text blocks would silently lose inter-block boundaries — a correctness bug. All existing tools return a single TextBlock; the MCP bridge follows suit.
|
||||
|
||||
### Replace the canonical MCP result with core `ContentBlock[]`
|
||||
|
||||
Rejected. Programmatic callers need protocol-complete MCP blocks and `structuredContent`, while Native consumers need durable core images rather than base64. One canonical protocol value plus a separate projection preserves both contracts.
|
||||
|
||||
### Add a generic RichContent service or perform I/O in `output.render`
|
||||
|
||||
Rejected. Core already owns the role-neutral content vocabulary, and a second service would duplicate its logging and ordering contracts. `output.render` is pure, synchronous, and replayable, so attachment I/O belongs in async execution with an exact finalization handoff.
|
||||
|
||||
### Let each image-returning tool special-case Code Mode parents
|
||||
|
||||
Rejected. That couples leaf tools to composite-tool internals and misses future rich tools. The generic Code Mode bridge observes the final post-policy content and forwards image-bearing results uniformly.
|
||||
|
||||
## Testing
|
||||
|
||||
Coverage is named per tier; each behavior lives at the cheapest tier that can express it.
|
||||
|
||||
- **Unit** (`tests/mcp-client.spec.ts`, `tests/apply.spec.ts`, mocked MCP SDK): the `publicToolName` algorithm (clean, normalize, truncate-and-hash, determinism, distinct-identity separation), raw-vs-public wire discipline, cross-server and native-tool coexistence, duplicate-`serverName` load failure and reservation release, invalid-tool-list rejection, generation swap/rollback, failed-re-sync retention, result mapping, cancellation, config schema validation. 100% per-file coverage gates the package.
|
||||
- **E2E** (`tests/mcp-client.e2e.ts`, keyless): the real MCP protocol against the in-repo fixture server, `@modelcontextprotocol/server-everything`, and `@modelcontextprotocol/server-filesystem` over stdio, and against an in-process `StreamableHTTPServerTransport` server over Streamable HTTP — discovery under the namespace, dotted-name normalization end to end, execution round-trips, duplicate-`serverName` rejection, disposal.
|
||||
- **Snapshot**: deliberately none. MCP tools introduce no new presentation shape — they register as raw `ToolDefinition`s and UI consumers use the generic-card fallback already pinned by their presentation suites. Adding an MCP server to a runnable snapshot composition would mutate its pinned system-prompt fixture and make every replay depend on spawning an external MCP server process for no new behavior. If a later change gives MCP tools their own render intent, that change names its snapshot coverage then.
|
||||
- **Unit** (`tests/mcp-client.spec.ts`, `tests/apply.spec.ts`, mocked MCP SDK): the `publicToolName` algorithm (clean, normalize, truncate-and-hash, determinism, distinct-identity separation), raw-vs-public wire discipline, cross-server and native-tool coexistence, duplicate-`serverName` load failure and reservation release, invalid-tool-list rejection, generation swap/rollback, failed-re-sync retention, lossless canonical results, mixed rich ordering, atomic malformed batches, exact capability/store refusal, explicit non-image diagnostics, post-execute policy precedence, cancellation, and config schema validation. 100% per-file coverage gates the package.
|
||||
- **E2E** (`tests/mcp-client.e2e.ts`, keyless): the real MCP protocol against the in-repo fixture server, `@modelcontextprotocol/server-everything`, and `@modelcontextprotocol/server-filesystem` over stdio, and against an in-process `StreamableHTTPServerTransport` server over Streamable HTTP — discovery under the namespace, dotted-name normalization end to end, execution round-trips, durable image save/read with base64 retained only in the canonical value, explicit refusal without an image route, duplicate-`serverName` rejection, and disposal.
|
||||
- **Snapshot**: the assembled ACP example owns the transport-visible inline-image transcript and the Code Mode image-forwarding transcript; package E2E owns the real MCP wire because the runnable snapshot must stay keyless and deterministic rather than spawning third-party server packages. MCP tool cards still use the generic-card fallback and require no package-specific UI snapshot.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -206,3 +218,4 @@ Coverage is named per tier; each behavior lives at the cheapest tier that can ex
|
||||
- **Tool schema quality**: MCP servers may expose poorly-described tools (vague descriptions, incomplete JSON schemas). The harness passes them through as-is — garbage-in-garbage-out; that is the server author's responsibility, not the bridge's.
|
||||
- **Stdio process management**: a misbehaving MCP server that ignores signals could wedge dispose. The Cordis fiber disposal has bounded quiescence; a stuck transport eventually times out at the framework level.
|
||||
- Crash recovery is automatic within the [reconnect budget](2026-08-06-mcp-client-auto-reconnect.md); manual reload remains the path after exhaustion or with `reconnect.enabled: false`.
|
||||
- Image payloads can enter model context only through the shared durable attachment store and an exact positive route capability. Audio and embedded-resource payloads remain execution-local with explicit diagnostics.
|
||||
|
||||
@@ -141,11 +141,11 @@ MCP 仅保证工具名在[单个服务器内](https://modelcontextprotocol.io/sp
|
||||
为来自同一个 MCP 服务器的所有工具提供统一的 `execute` 处理器:
|
||||
|
||||
1. 解析 `rawName`(执行器闭包持有它),以配置的超时时间调用 `client.callTool({ name: rawName, arguments }, { signal: exec.signal })`——公开名称永远不发送给服务器。
|
||||
2. 映射结果:
|
||||
- 多个 `text` 内容块 → 以 `'\n'` 连接为单个 `TextBlock`(之所以必须这样做,是因为 `flattenText` 使用无分隔符的 `join('')`,多个内容块会丢失块间边界)。
|
||||
- `image` 内容块 → 丢弃并 `ctx.logger.warn`(harness 没有图片内容块类型;[删除图片 Agent Note](../simplification/2026-07-04-drop-image-content-block.md))。
|
||||
- `isError: true` → 映射到 harness 的 `isError` 结果路径(`{ content: [...], isError: true }`)。
|
||||
3. 取消:`exec.signal`(来自 agent loop(智能体循环)的取消)透传给 MCP SDK 的 `callTool`,后者向服务器发送 `$/cancelRequest`。
|
||||
2. 把规范成功值保留为 `{ content: JsonValue[], structuredContent? }`;完整 MCP JSON 块仍是程序化调用/Code Mode 值。`isError: true` 会在持久化任何图片前抛出,使失败路径归注册表所有。
|
||||
3. 另行准备有序 Native 投影。连续文本块以 `'\n'` 连接;资源链接以文本保留名称和 URI;音频、嵌入资源、格式错误的块和未知类型成为明确诊断。只要存在图片,桥接层就严格解码完整批次,解析调用 agent 的最新确切路由,要求附件存储以及模型明确支持图片输入,再把全成员校验和有序持久化委托给 `AttachmentStore.saveImages()`。任何解码、能力或存储拒绝都会把全部图片渲染为诊断文本,且不返回部分引用。
|
||||
4. 保持 `output.render` 同步且纯净。执行器把更丰富的投影暂存在按同步世代创建、以确切执行为键的 `WeakMap` 中;只有注册表的 post-execute 结果仍保留原规范值和兜底内容时,`finalizeContent` 才安装该投影。策略阻止、值替换或内容替换仍具有权威性,重新同步也无法让旧世代消费新执行状态。
|
||||
5. Code Mode 接收未改动的规范值。其通用分发桥接层会把包含图片的成功最终内容序列经外层 `run_code` 结果延后,因此 MCP 无需私有父 token 特例。
|
||||
6. 取消:`exec.signal`(来自 agent loop 的取消)透传给 MCP SDK 的 `callTool`、确切模型查询和存储前门禁。
|
||||
|
||||
### 子进程环境(stdio 传输)
|
||||
|
||||
@@ -189,13 +189,25 @@ v1 否决。它能防止跨服务器冲突,但无法将 MCP 注册与原生 ha
|
||||
|
||||
否决。DeepSeek 序列化器中的 `flattenText()` 在将 `ContentBlock[]` 扁平化为协议格式(wire format)时使用 `join('')`(无分隔符)。多个 text 块会静默丢失块间边界——这是正确性缺陷。所有现有工具返回单个 TextBlock;MCP 桥接遵循同一做法。
|
||||
|
||||
### 用核心 `ContentBlock[]` 替换规范 MCP 结果
|
||||
|
||||
不予采用。程序化调用方需要协议完整的 MCP 块和 `structuredContent`,Native 消费方则需要持久核心图片而不是 base64。一份规范协议值加一份独立投影可以同时保留两项契约。
|
||||
|
||||
### 添加通用 RichContent 服务,或在 `output.render` 中执行 I/O
|
||||
|
||||
不予采用。核心已经拥有角色无关的内容词汇,第二套服务会重复其日志与顺序契约。`output.render` 必须纯净、同步且可回放,因此附件 I/O 属于异步执行,再经确切的最终化交接安装结果。
|
||||
|
||||
### 让每个返回图片的工具分别特殊处理 Code Mode 父调用
|
||||
|
||||
不予采用。这会把叶子工具与组合工具内部机制耦合,并漏掉未来丰富工具。通用 Code Mode 桥接层观察最终 post-policy 内容,统一转发含图片结果。
|
||||
|
||||
## 测试
|
||||
|
||||
覆盖范围按层级列出;每项行为都放在能够表达它的最低成本层级。
|
||||
|
||||
- **单元测试**(`tests/mcp-client.spec.ts`、`tests/apply.spec.ts`,mock MCP SDK):`publicToolName` 算法(干净名称、规范化、截断加 hash、确定性、不同标识的分离)、raw 与 public 的协议纪律、跨服务器与原生工具共存、重复 `serverName` 加载失败与预留释放、无效工具列表拒绝、注册代切换/回滚、重新同步失败时保留上一代注册、结果映射、取消、配置 schema 校验。100% 逐文件覆盖率门禁约束该包。
|
||||
- **E2E**(`tests/mcp-client.e2e.ts`,无需密钥):使用真实 MCP 协议对接仓库内的 fixture(测试前置数据)服务器、`@modelcontextprotocol/server-everything` 和 `@modelcontextprotocol/server-filesystem`(stdio 传输),以及进程内 `StreamableHTTPServerTransport` 服务器(Streamable HTTP 传输)——命名空间下的发现、带点号名称的端到端规范化、执行往返、重复 `serverName` 拒绝、dispose。
|
||||
- **快照**:刻意不做。MCP 工具不引入新的展示形态——它们以原始 `ToolDefinition` 注册,UI 消费方使用各自展示测试套件已固定的通用卡片兜底。将 MCP 服务器添加到某个可运行的快照组合会改变其已固定的系统提示词 fixture,且使每次回放依赖于 spawn 外部 MCP 服务器进程,而新增行为为零。如果后续变更为 MCP 工具引入专属渲染意图,该变更届时自行声明快照覆盖。
|
||||
- **单元测试**(`tests/mcp-client.spec.ts`、`tests/apply.spec.ts`,mock MCP SDK):`publicToolName` 算法(干净名称、规范化、截断加 hash、确定性、不同标识的分离)、raw 与 public 的协议纪律、跨服务器与原生工具共存、重复 `serverName` 加载失败与预留释放、无效工具列表拒绝、注册代切换/回滚、重新同步失败时保留上一代注册、无损规范结果、丰富内容混合顺序、格式错误批次原子性、确切能力/存储拒绝、明确的非图片诊断、post-execute 策略优先级、取消,以及配置 schema 校验。100% 逐文件覆盖率门禁约束该包。
|
||||
- **E2E**(`tests/mcp-client.e2e.ts`,无需密钥):使用真实 MCP 协议对接仓库内的 fixture(测试前置数据)服务器、`@modelcontextprotocol/server-everything` 和 `@modelcontextprotocol/server-filesystem`(stdio 传输),以及进程内 `StreamableHTTPServerTransport` 服务器(Streamable HTTP 传输)——命名空间下的发现、带点号名称的端到端规范化、执行往返、持久图片保存/读取且 base64 只保留在规范值中、缺少图片路由时明确拒绝、重复 `serverName` 拒绝,以及 dispose。
|
||||
- **快照**:组装后的 ACP 示例负责传输可见的内联图片 transcript 与 Code Mode 图片转发 transcript;包 E2E 负责真实 MCP 协议,因为可运行快照必须保持无密钥且确定,而不是 spawn 第三方服务器包。MCP 工具卡片仍使用通用卡片兜底,无需包专属 UI 快照。
|
||||
|
||||
## 后果
|
||||
|
||||
@@ -206,3 +218,4 @@ v1 否决。它能防止跨服务器冲突,但无法将 MCP 注册与原生 ha
|
||||
- **工具 schema 质量**:MCP 服务器可能暴露描述不佳的工具(模糊的描述、不完整的 JSON Schema)。harness 原样透传——垃圾进垃圾出;这是服务器作者的责任,不是桥接的。
|
||||
- **Stdio 进程管理**:行为异常的 MCP 服务器如果忽略信号,可能卡住 dispose。Cordis fiber 的 dispose 具有有界的完全停稳过程;卡住的传输层最终会在框架层面超时。
|
||||
- 崩溃恢复在[重连预算](2026-08-06-mcp-client-auto-reconnect.md)内自动进行;耗尽后或配置 `reconnect.enabled: false` 时回退为手动重新加载。
|
||||
- 图片载荷只有通过共享持久附件存储和确切正向路由能力,才能进入模型上下文。音频与嵌入资源载荷仍只存在于执行局部,并附带明确诊断。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md
|
||||
2026-07-20-code-mode-typed-tool-returns.md: 747b33384238957190932e07a5f41d01cf1b65b6
|
||||
2026-07-20-code-mode-typed-tool-returns.zh.md: 9ad41285ca76427c39a63ae54f80fe34e62f2520
|
||||
2026-07-20-code-mode-typed-tool-returns.md: a8a251f5f0d39f4deedc42e08eb45c2b5fa11807
|
||||
2026-07-20-code-mode-typed-tool-returns.zh.md: 3d7ae4f98c569d3908f14fc918aebe7190e7ce61
|
||||
|
||||
@@ -14,7 +14,7 @@ The [canonical tool-output contract](../architecture/2026-07-20-canonical-tool-o
|
||||
|
||||
## Decision
|
||||
|
||||
Code Mode is a typed projection of the visible tool registry. Each successful binding resolves to the final canonical `JsonValue` after post-execute policy, while a failed binding rejects with a real `ToolCallError`. Intermediate values remain inside the run and cross the worker boundary whole. Only the outer `run_code` logs, completion value, or failure diagnostic enter the configurable output ledger and the model-facing spill pipeline.
|
||||
Code Mode is a typed projection of the visible tool registry. Each successful binding resolves to the final canonical `JsonValue` after post-execute policy, while a failed binding rejects with a real `ToolCallError`. Intermediate values remain inside the run and cross the worker boundary whole. The outer `run_code` logs, completion value, or failure diagnostic enter the configurable output ledger and model-facing spill pipeline; a successfully settled sub-call whose final Native content contains an image additionally defers that complete ordered content through the parent result as logged, source-attributed context.
|
||||
|
||||
This note owns the return and failure contract layered on the original [Code Mode foundation](2026-06-15-code-mode.md). The unified schema vocabulary is owned by the [JSON-value schema DSL note](../architecture/2026-07-20-unified-json-value-schema-dsl.md), and Native rendering and policy projection remain owned by the canonical-output note.
|
||||
|
||||
@@ -49,7 +49,7 @@ declare const tools: {
|
||||
|
||||
### Binding values and failures
|
||||
|
||||
Before dispatch the bridge snapshots binding arguments as lossless JSON and snapshots the detached value again for an independent durable summary event. Host-side detachment, immutable execution, and output-schema projection all use iterative traversals rather than nested structured clone or recursive freezing. `undefined`, non-finite numbers, `-0`, sparse arrays, cycles, functions, and exotic objects reject that call before the tool runs. Successful dispatch returns `ToolExecutionResult.value`; Native `content`, metadata, and internal error information do not cross to the program.
|
||||
Before dispatch the bridge snapshots binding arguments as lossless JSON and snapshots the detached value again for an independent durable summary event. Host-side detachment, immutable execution, and output-schema projection all use iterative traversals rather than nested structured clone or recursive freezing. `undefined`, non-finite numbers, `-0`, sparse arrays, cycles, functions, and exotic objects reject that call before the tool runs. Successful dispatch returns `ToolExecutionResult.value`; Native `content`, metadata, and internal error information do not cross to the program. Image-bearing final content is not a second binding value: the bridge ferries it after the outer result so the next model request can see the durable image, while post-execute block/content replacement remains authoritative and text-only results are not duplicated.
|
||||
|
||||
Code Mode declares its rejection capability on the runtime request as `{ name: "ToolCallError", memberNameProperty: "toolName" }`. The runtime Service Definition treats those names as data: the worker materializes and injects the actual constructor used for `tools` binding failures, so `error instanceof ToolCallError` works without making a generic runtime know about tools. The worker constructs failures and defines their public fields through module-captured error and property-definition intrinsics plus null-prototype descriptors, so model mutations cannot replace the promised rejection with a worker failure. The error has the standard `Error` message plus the exact `toolName`; it deliberately omits `ToolFailure.info`, error codes, and Native content. This is an exception contract for control flow, not a failure union for programmatic classification.
|
||||
|
||||
@@ -73,13 +73,13 @@ Temporary Cordis Plugins follow the same rule: `cordis_mount` returns `{ id, plu
|
||||
|
||||
### Persistence, metadata, and spill
|
||||
|
||||
Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/code-dispatch` but does not persist canonical values. `tool/result` continues to persist only rendered content, error, and optional metadata. `SESSION_FORMAT_VERSION` remains unchanged (pre-release shape churn does not bump it) and replay cannot recreate intermediate canonical program values.
|
||||
Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/code-dispatch` but does not persist canonical values. `tool/result` continues to persist only rendered content, error, and optional metadata. A successful final content sequence containing an image is also wrapped in a source-attributed user message and deferred through the outer result; the normal session event makes that model-visible input reconstructable. `SESSION_FORMAT_VERSION` remains unchanged (pre-release shape churn does not bump it) and replay cannot recreate intermediate canonical program values.
|
||||
|
||||
The opaque `exec.parent` token marks nested calls. Presentation metadata and generic or tool-owned spill projections skip those calls because they have no direct result card and their canonical values never enter context. The outer `run_code` call alone produces one card and may spill its final post-policy presentation; `run_code` intentionally declares neither a result presenter nor presentation metadata, so UI adapters complete the card through their generic raw-content fallback using durable `tool/result.content`.
|
||||
|
||||
## Testing
|
||||
|
||||
Compile-time and snapshot tests pin exact `ToolArgsMap`, `ToolOutputMap`, `ToolName`, schema-to-TypeScript coverage, and exotic names. Registry and real-worker tests cover scalar, array, object, and null values; raw string rendering; absent `undefined`; consumer-declared real rejection classes, including `ToolCallError`; invalid arguments and completions, including intrinsic-looking forged prototypes; model-mutated JSON-boundary globals, prototype methods, constructor slots, and inherited descriptor fields; typed binding failures after those mutations; large uncapped intermediate bindings; nested spill suppression; exact and over-limit 64 MiB accounting; combined logs/value/diagnostic accounting; giant thrown stacks; bounded failure spill; hostile forged traffic; and built-package execution.
|
||||
Compile-time and snapshot tests pin exact `ToolArgsMap`, `ToolOutputMap`, `ToolName`, schema-to-TypeScript coverage, exotic names, and assembled Code Mode image forwarding. Registry and real-worker tests cover scalar, array, object, and null values; raw string rendering; absent `undefined`; consumer-declared real rejection classes, including `ToolCallError`; invalid arguments and completions, including intrinsic-looking forged prototypes; model-mutated JSON-boundary globals, prototype methods, constructor slots, and inherited descriptor fields; typed binding failures after those mutations; large uncapped intermediate bindings; nested spill suppression; generic image-bearing context deferral plus post-execute replacement/block precedence; exact and over-limit 64 MiB accounting; combined logs/value/diagnostic accounting; giant thrown stacks; bounded failure spill; hostile forged traffic; and built-package execution.
|
||||
|
||||
Keyless real-worker integration tests pin the two handle workflows that prose results could not safely support. A background bash call returns its job id, the outer run settles, and a later run polls that id to completion; separate cases prove pre-abort creates no task, post-publication call abort preserves the task, foreground execution stays signal-coupled, and `job_kill` owns cancellation. A Cordis program reads an active or pending mount's id and `waitingFor` fields directly, unmounts by that id, and confirms removal without parsing rendered text.
|
||||
|
||||
@@ -93,6 +93,10 @@ Keyless real-worker integration tests pin the two handle workflows that prose re
|
||||
|
||||
**Silently inspect or truncate an oversized completion.** Rejected because changing a JSON value into a string is lossy and type-incorrect. The explicit `output-limit` failure lets the model choose a smaller result, while the retained logs and diagnostic can still use normal outer spill.
|
||||
|
||||
**Require each rich leaf tool to inspect `exec.parent` and defer itself.** Rejected because it couples leaf tools to Code Mode internals, duplicates policy handling, and misses future rich tools. The dispatch bridge owns generic forwarding from the already settled final result.
|
||||
|
||||
**Expose Native rich content as part of every binding's canonical value.** Rejected because a canonical value is lossless JSON and tool-specific; attachment blocks are a model projection with durable lifecycle semantics. Keeping the value and projection separate preserves typed programs without dropping images from later model context.
|
||||
|
||||
## Consequences
|
||||
|
||||
Code programs can compose tools through stable values instead of reverse-engineering Native prose. Native and Both Mode retain their existing text and UI presentation, while Code Mode receives output-schema types and exact runtime JSON. Tool authors must treat the canonical value as their programmatic API and put display-only formatting in the renderer.
|
||||
@@ -107,6 +111,6 @@ The worker performs bounded-depth flat-wire transport and lossless validation bu
|
||||
- Intermediate values have no byte cap and can exhaust process or worker memory through retention, flat-wire copies, or structured-clone cost.
|
||||
- The 64 MiB hard cap applies only to the outer variable payloads, excluding fixed result-envelope syntax and presentation whitespace; spill cannot recover bytes rejected beyond that cap.
|
||||
- Provider or executor acquisition limits may already have discarded source data before a canonical value reaches Code Mode.
|
||||
- Unsupported MCP output schemas fall back to `JsonValue`; richer Native multimedia projection is deferred.
|
||||
- Unsupported MCP output schemas fall back to `JsonValue`; admitted MCP images use the generic deferred projection, while audio and embedded-resource payloads remain diagnostic-only.
|
||||
- There is one result card per outer `run_code`, never per nested call.
|
||||
- Code failures expose `ToolCallError` message and tool name only, without a programmatic error-code union.
|
||||
|
||||
@@ -14,7 +14,7 @@ Code Mode 过去会把每个嵌套工具的结果从 `ContentBlock[]` 重新投
|
||||
|
||||
## 决策
|
||||
|
||||
Code Mode 是可见工具注册表的类型化投影。每个成功的绑定调用都会解析为 post-execute 策略处理后的最终规范 `JsonValue`,失败的绑定调用则会以真正的 `ToolCallError` 拒绝 Promise。中间值只存在于本次运行中,并完整跨越 worker 边界。只有外层 `run_code` 的日志、完成值或失败诊断会进入可配置的输出账本以及面向模型的 spill 流水线。
|
||||
Code Mode 是可见工具注册表的类型化投影。每个成功的绑定调用都会解析为 post-execute 策略处理后的最终规范 `JsonValue`,失败的绑定调用则会以真正的 `ToolCallError` 拒绝 Promise。中间值只存在于本次运行中,并完整跨越 worker 边界。外层 `run_code` 的日志、完成值或失败诊断会进入可配置的输出账本以及面向模型的输出落盘流水线;如果成功结算的子调用最终 Native 内容包含图片,其完整有序内容还会经父结果延后为写入日志且带来源归属的上下文。
|
||||
|
||||
本文档定义叠加在原始 [Code Mode 基础](2026-06-15-code-mode.md)之上的返回值与失败约定。统一 schema 词汇由 [JSON 值 schema DSL Agent Note](../architecture/2026-07-20-unified-json-value-schema-dsl.md)负责定义;Native 渲染与策略投影仍由规范输出 Agent Note 负责定义。
|
||||
|
||||
@@ -49,7 +49,7 @@ declare const tools: {
|
||||
|
||||
### 绑定值与失败
|
||||
|
||||
分发前,桥接层会把绑定参数快照为无损 JSON,再对分离后的值生成一次快照,供独立的持久摘要事件使用。宿主侧的值分离、执行数据的不可变处理与输出 schema 投影均采用迭代遍历,而不使用嵌套结构化克隆或递归冻结。`undefined`、非有限数、`-0`、稀疏数组、循环引用、函数和非普通对象都会使该调用在工具运行前被拒绝。成功分发会返回 `ToolExecutionResult.value`;Native `content`、元数据和内部错误信息不会传入程序。
|
||||
分发前,桥接层会把绑定参数快照为无损 JSON,再对分离后的值生成一次快照,供独立的持久摘要事件使用。宿主侧的值分离、执行数据的不可变处理与输出 schema 投影均采用迭代遍历,而不使用嵌套结构化克隆或递归冻结。`undefined`、非有限数、`-0`、稀疏数组、循环引用、函数和非普通对象都会使该调用在工具运行前被拒绝。成功分发会返回 `ToolExecutionResult.value`;Native `content`、元数据和内部错误信息不会传入程序。含图片的最终内容不是第二份绑定值:桥接层会在外层结果之后转运它,使下一次模型请求可以看到持久图片;post-execute 阻止/内容替换仍具有权威性,纯文本结果不会重复。
|
||||
|
||||
Code Mode 通过运行时请求中的 `{ name: "ToolCallError", memberNameProperty: "toolName" }` 声明其以异常拒绝 Promise 的能力。运行时 Service Definition 只把这些名称视为数据:worker 会动态生成并注入真正用于 `tools` 绑定失败的构造函数,因此无需让通用运行时了解工具,`error instanceof ToolCallError` 也能成立。worker 使用模块初始化时捕获的 Error 构造函数与属性定义内建方法,配合原型为 null 的属性描述符,构造失败对象并定义其公开字段,因此模型代码的修改不会把约定承诺的 reject 变成 worker 失败。该错误包含标准的 `Error` 消息和确切的 `toolName`,并有意省略 `ToolFailure.info`、错误代码与 Native 内容。这是一项用于控制流的异常约定,而不是供程序分类的失败联合。
|
||||
|
||||
@@ -73,13 +73,13 @@ Code Mode 通过运行时请求中的 `{ name: "ToolCallError", memberNameProper
|
||||
|
||||
### 持久化、元数据与 spill
|
||||
|
||||
嵌套分发在 `tool/code-dispatch` 上记录子调用完整渲染后的 `content`/`isError`,但不会持久化规范值。`tool/result` 继续只持久化渲染后的内容、错误和可选元数据。`SESSION_FORMAT_VERSION` 保持不变(预发布阶段的形状变动不递增版本号),回放也无法重建程序的规范中间值。
|
||||
嵌套分发在 `tool/code-dispatch` 上记录子调用完整渲染后的 `content`/`isError`,但不会持久化规范值。`tool/result` 继续只持久化渲染后的内容、错误和可选元数据。包含图片的成功最终内容序列还会包装成带来源归属的用户消息,并经外层结果延后;普通会话事件使该模型可见输入可以重建。`SESSION_FORMAT_VERSION` 保持不变(预发布阶段的形状变动不递增版本号),回放也无法重建程序的规范中间值。
|
||||
|
||||
不透明的 `exec.parent` token 用于标识嵌套调用。由于这些调用没有直接对应的结果卡片,而且其规范值永远不会进入上下文,展示元数据以及通用或工具自有的 spill 投影都会跳过它们。只有外层 `run_code` 调用会生成一张卡片,并且可能对 post-policy 处理后的最终展示执行 spill;`run_code` 有意既不声明结果展示器,也不声明展示元数据,因此 UI 适配器会通过通用的原始内容回退机制,使用持久化的 `tool/result.content` 补全该卡片。
|
||||
|
||||
## 测试
|
||||
|
||||
编译期测试与快照测试锁定了精确的 `ToolArgsMap`、`ToolOutputMap`、`ToolName`、schema 到 TypeScript 的覆盖范围以及特殊名称。注册表与真实 worker 测试覆盖标量、数组、对象和 null 值;字符串原文渲染;缺席的 `undefined`;消费方声明、实际用于拒绝 Promise 的异常类,包括 `ToolCallError`;无效参数与完成值,包括伪装为内建原型的伪造原型;模型代码修改过的 JSON 边界全局对象、原型方法、构造函数槽位,以及继承而来的属性描述符字段;上述修改后的类型化绑定失败;不设上限的大型中间绑定值;嵌套 spill 抑制;64 MiB 上限内外的精确计量;日志、值与诊断的组合计量;抛出的超大堆栈;有界失败的 spill;不可信对端伪造的流量;以及构建后包的执行。
|
||||
编译期测试与快照测试锁定了精确的 `ToolArgsMap`、`ToolOutputMap`、`ToolName`、schema 到 TypeScript 的覆盖范围、特殊名称,以及组装后的 Code Mode 图片转发。注册表与真实 worker 测试覆盖标量、数组、对象和 null 值;字符串原文渲染;缺席的 `undefined`;消费方声明、实际用于拒绝 Promise 的异常类,包括 `ToolCallError`;无效参数与完成值,包括伪装为内建原型的伪造原型;模型代码修改过的 JSON 边界全局对象、原型方法、构造函数槽位,以及继承而来的属性描述符字段;上述修改后的类型化绑定失败;不设上限的大型中间绑定值;嵌套输出落盘抑制;通用含图片上下文延后以及 post-execute 替换/阻止优先级;64 MiB 上限内外的精确计量;日志、值与诊断的组合计量;抛出的超大堆栈;有界失败的输出落盘;不可信对端伪造的流量;以及构建后包的执行。
|
||||
|
||||
无密钥的真实 worker 集成测试锁定了自然语言结果无法安全支持的两种句柄工作流。后台 bash 调用返回 job id,外层运行结束,之后的运行再根据该 id 轮询直至任务完成;其他用例分别证明,预先中止不会创建任务、发布后的调用取消会保留任务、前台执行仍与信号耦合,并且由 `job_kill` 负责取消。Cordis 程序会直接读取 active 或 pending 挂载的 id 和 `waitingFor` 字段,按该 id 卸载,并在不解析渲染文本的情况下确认挂载已移除。
|
||||
|
||||
@@ -93,6 +93,10 @@ Code Mode 通过运行时请求中的 `{ name: "ToolCallError", memberNameProper
|
||||
|
||||
**静默检查格式化或截断过大的完成值:**不予采纳。把 JSON 值改成字符串既有损又违反类型。显式的 `output-limit` 失败让模型可以选择返回更小的结果,而保留的日志和诊断仍可使用普通的外层 spill 机制。
|
||||
|
||||
**要求每个丰富叶子工具检查 `exec.parent` 并自行延后。** 不予采用,因为这会把叶子工具与 Code Mode 内部机制耦合、重复策略处理,并遗漏未来丰富工具。分发桥接层负责从已经结算的最终结果通用转发。
|
||||
|
||||
**把 Native 丰富内容暴露为每个绑定规范值的一部分。** 不予采用,因为规范值是无损 JSON 且由工具定义;附件块是具有持久生命周期语义的模型投影。保持值与投影分离,既能保留类型化程序,也不会从后续模型上下文中丢弃图片。
|
||||
|
||||
## 后果
|
||||
|
||||
Code Mode 程序可以通过稳定值组合工具,无需逆向解析 Native 自然语言。Native 和 Both Mode 保留现有文本与 UI 展示,Code Mode 则获得输出 schema 类型和精确的运行时 JSON。工具作者必须把规范值视为程序化 API,并将仅用于展示的格式化放入渲染器。
|
||||
@@ -107,6 +111,6 @@ worker 会以嵌套深度有界的扁平协议格式传输数据并执行无损
|
||||
- 中间值没有字节上限,可能因值的保留、扁平协议格式副本或结构化克隆开销而耗尽进程或 worker 内存。
|
||||
- 64 MiB 硬上限只适用于外层可变负载,不计固定的结果封装语法与展示空白;spill 无法恢复超出该上限后被拒绝的字节。
|
||||
- 提供方或执行器的采集上限可能在规范值到达 Code Mode 前就已丢弃部分源数据。
|
||||
- 不支持的 MCP 输出 schema 会回退为 `JsonValue`;更丰富的 Native 多媒体投影留待后续实现。
|
||||
- 不支持的 MCP 输出 schema 会回退为 `JsonValue`;已准入的 MCP 图片使用通用延后投影,而音频和嵌入资源载荷仍只提供诊断。
|
||||
- 每个外层 `run_code` 只有一张结果卡片,嵌套调用不会各自生成卡片。
|
||||
- Code Mode 失败只暴露 `ToolCallError` 的消息与工具名,不提供程序可用的错误代码联合。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: fcbe301f705f8e01095eac9a02329904626af843
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 338f27c6db934092b62ed6107f8706fe451b6a4b
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 5162e0d6f63715ffd83cff1a622e07c110a7938e
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: d3a5cf18a7d308753d5e46741cea2b2993536f70
|
||||
|
||||
@@ -16,7 +16,7 @@ Peer products converge on an attachment rail above the editor, but their storage
|
||||
|
||||
## Decision
|
||||
|
||||
Pasted or dropped raster images are the Web composer's first consumer of a durable attachment capability. Unsent files remain temporary client-owned draft state. The host validates and durably commits every accepted user image before appending its message event. A provider adapter that produces structured image output must durably commit the output before appending its assistant block. Canonical user and assistant content contains only role-neutral `ImageBlock` references.
|
||||
Pasted or dropped raster images are the Web composer's first consumer of a durable attachment capability. Unsent files remain temporary client-owned draft state. Every rich-content intake adapter decodes its wire blocks, proves route capability, and delegates the complete image batch to the attachment service before appending its message event. A provider adapter that produces structured image output must durably commit the output before appending its assistant block. Canonical user and assistant content contains only role-neutral `ImageBlock` references.
|
||||
|
||||
Version one supports PNG, JPEG, WebP, and GIF paste and drag-and-drop, image-only or mixed prompts, historical user and assistant image rendering, and original-image preview on a single click (display and interaction specifics superseded in part by the [attachment-display alignment note](2026-08-11-web-attachment-display-alignment.md)). File picking, generic files, PDF, audio, video, image copying, and a custom context menu remain separate follow-ups.
|
||||
|
||||
@@ -114,7 +114,7 @@ type PromptInputPart =
|
||||
}
|
||||
```
|
||||
|
||||
Base64 crosses JSON-RPC once and is discarded after persistence. The host validates canonical base64, image count, aggregate bytes, individual bytes, the declared MIME against a fully decoded raster, intrinsic dimensions, and decoded-pixel count. It awaits the seam's storage-free `validateImage` for every batch member before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Storage commits then run in submission order to bound full-raster decoder memory. If a later storage I/O operation fails, the host appends no user event, but an earlier immutable content-addressed object may remain unreferenced; version one leaves cleanup to future reference-aware garbage collection instead of adding destructive rollback to the deduplicated store. Only after every image succeeds does it call the agent with normalized text and durable image blocks in the submitted order. A failure exposes no attachment path or raw bytes.
|
||||
Base64 crosses a wire boundary once and is discarded after persistence. Each front door validates canonical base64 and declared MIME shape, then calls `AttachmentStore.saveImages()` with the whole decoded batch. The service owns image count, aggregate bytes, individual bytes, fully decoded raster/MIME agreement, intrinsic dimensions, and decoded-pixel count; it validates every batch member before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Storage commits then run in submission order to bound full-raster decoder memory. If a later storage I/O operation fails, the caller appends no model-visible event and receives no partial references, but an earlier immutable content-addressed object may remain unreferenced; version one leaves cleanup to future reference-aware garbage collection instead of adding destructive rollback to the deduplicated store. Only after every image succeeds does the front door call the agent with normalized text and durable image blocks in wire order. A failure exposes no attachment path or raw bytes.
|
||||
|
||||
`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and rejects invalidated late loads before allocating an object URL so an unmounted session or disposed service cannot repopulate the cache.
|
||||
|
||||
@@ -128,7 +128,7 @@ The Pi-AI adapter is the first visual-input route: it resolves `ctx.attachments`
|
||||
|
||||
Core supports structured assistant image blocks, but no current production provider route is certified for image output. Any future output-capable adapter must retrieve provider bytes under bounded size and time policy, validate them through the same attachment service, persist them, and only then publish the atomic `ImageBlock`. A URL in assistant Markdown remains text and is never downloaded automatically.
|
||||
|
||||
Provider-neutral token estimation does not guess visual pricing from image dimensions; provider-reported usage remains authoritative. ACP renders an explicit image marker until that protocol API gains native image support rather than silently omitting the block.
|
||||
Provider-neutral token estimation does not guess visual pricing from image dimensions; provider-reported usage remains authoritative. ACP advertises image prompts only when its configured exact route and attachment deployment can accept them, persists inline input before publishing the user event, and re-reads committed assistant image references for native ACP image updates. MCP keeps canonical raw blocks for programmatic callers while projecting admitted images to durable core blocks; Code Mode carries any settled image-bearing sub-result through the outer result as logged source-attributed context.
|
||||
|
||||
Compaction replays the selected conversation prefix, including image references, into the configured summarization route. A visual-capable route resolves those references through its adapter; a text-only route fails explicitly instead of silently dropping the visual context. The synthesized checkpoint remains text-only, and `compaction-basic` rejects image summary output with `UNSUPPORTED_CONTENT`.
|
||||
|
||||
@@ -148,22 +148,24 @@ Malformed base64, unsupported or mismatched media, truncated image payloads, exc
|
||||
|
||||
| Surface | Responsibility |
|
||||
| --- | --- |
|
||||
| `packages/attachment/attachment` | Opaque attachment identifier, image reference, limits, failures, and `ctx.attachments` service. |
|
||||
| `packages/attachment/attachment` | Opaque attachment identifier, image reference, limits, failures, and single/batch admission through `ctx.attachments`. |
|
||||
| `packages/attachment/attachment-local` | Private content-addressed storage, complete raster decoding, integrity verification, and configuration. |
|
||||
| `packages/llm/llm` | Role-neutral `ImageBlock` and input-modality metadata. |
|
||||
| `packages/llm/llm-pi-ai` | Resolve durable supported image input into native provider content. |
|
||||
| `packages/llm/llm-deepseek` | Reject image content explicitly. |
|
||||
| `packages/compaction/compaction-basic` | Preserve images in summary input and reject non-text checkpoint output explicitly. |
|
||||
| `packages/host/apiproxy` and `packages/bundle/base` | Narrow upload wire, persist-before-event ordering, session-authorized reads, limits and model preflight, plus default profile composition. |
|
||||
| `packages/host/apiproxy` and `packages/bundle/base` | Narrow upload wire, shared batch admission, limits and routed-model preflight, persist-before-event ordering, session-authorized reads, and default profile composition. |
|
||||
| `packages/client/connection` and `packages/client/runtime` | Bounded request buffering, wire types, fixture images, prompt uploads, attachment reads, and durable-reference folding. |
|
||||
| `packages/client/ui-conversation` | Per-session draft images, attachment rail, user and assistant image controls, and original preview. |
|
||||
| `packages/acp/acp` | Explicit fallback rendering for image blocks. |
|
||||
| `packages/acp/acp` | Conditional native image capability, atomic inline-image admission, and verified assistant-image delivery. |
|
||||
| `packages/mcp/mcp-client` | Lossless canonical MCP results plus capability-gated durable image projection and explicit diagnostics for unsupported rich blocks. |
|
||||
| `packages/core/tools` | Generic Code Mode forwarding of settled image-bearing sub-results after the outer result. |
|
||||
|
||||
The attachment packages form the interface/implementation side of one capability seam. Composer behavior stays in the conversation object layer, provider conversion stays in adapters, and no change is required in `agent-loop`.
|
||||
|
||||
### Implementation
|
||||
|
||||
The implemented slice includes the attachment seam, role-neutral image block, Pi-AI input conversion, DeepSeek rejection, durable host ordering, Web upload/read protocol, current image-limit enforcement, bounded Web request bodies, in-memory draft images, paste/drop rail, user and assistant history rendering, single-click preview, compaction handling, and keyless assembled Web coverage.
|
||||
The implemented slice includes the attachment seam and shared batch admission, role-neutral image block, Pi-AI input conversion, DeepSeek rejection, durable Web/ACP/MCP ordering, Web upload/read protocol, conditional ACP image wire support, lossless MCP canonical results with durable image projection, generic Code Mode rich-result forwarding, current image-limit enforcement, bounded Web request bodies, in-memory draft images, paste/drop rail, user and assistant history rendering, single-click preview, compaction handling, and keyless assembled Web and ACP coverage.
|
||||
|
||||
No compatibility shim is required for the pre-release prompt wire; all call sites and fixtures change with the introducing slice.
|
||||
|
||||
@@ -193,12 +195,25 @@ Composer presentation can use a generic attachment rail, but provider semantics
|
||||
|
||||
UI state can be stale and does not protect direct SDK, ACP, replay, or uncatalogued model paths. Silent filtering changes user intent. Provider enforcement remains mandatory, while UI checks are optional earlier feedback.
|
||||
|
||||
### Add a generic RichContent service above the core content vocabulary
|
||||
|
||||
Rejected because the core already has the role-neutral `ContentBlock` vocabulary and attachment references. A second generic service would duplicate ordering, capability, logging, and lifetime semantics while still requiring each wire adapter to parse its own protocol. Narrow image adapters around the existing core preserve ownership and leave audio/resources to earn their own lifecycle contracts.
|
||||
|
||||
### Normalize MCP results into core content as the canonical tool value
|
||||
|
||||
Rejected because Code Mode and programmatic callers need the complete MCP JSON blocks and optional `structuredContent`; replacing that value with a Native projection would make the bridge lossy. MCP retains the protocol value and prepares a separate model projection, with final post-execute policy remaining authoritative.
|
||||
|
||||
### Perform attachment reads and writes inside synchronous output renderers
|
||||
|
||||
Rejected because tool renderers are pure, synchronous, and replayable. MCP prepares image projection during async execution and installs it only at the registry's finalization boundary; ACP performs async admission and output conversion in its transport lifecycle. Code Mode forwarding observes the already settled final content instead of giving individual image tools private parent-token behavior.
|
||||
|
||||
## Testing
|
||||
|
||||
- Storage tests cover content-addressed deduplication, private permissions, admission failures, corruption/missing-object failures, and reading history after deployment limits are lowered.
|
||||
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, image-admission/model-selection races (queued and steering placements), pending publication, idle release without publication, text-only queue edits, and selection against current derived history after compaction.
|
||||
- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, draft/session-scope/application object-URL cleanup, and a deferred historical read that completes after disposal; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail.
|
||||
- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, recursively nested tool-result images, preserved summary input, and explicit image-output rejection.
|
||||
- Attachment, MCP, ACP, and Code Mode tests cover all-member validation before writes, mixed text/image ordering, no inline base64 in durable events, exact route-capability gates, explicit unsupported-content diagnostics, post-execute replacement/block precedence, cancellation during admission, verified assistant-image delivery, and generic nested-image forwarding. A keyless assembled ACP snapshot sends a real inline PNG and pins only its durable reference in the session log.
|
||||
- A credentialed real-API test sends a PNG through the Anthropic `claude-opus-4-8` route and requires the model to identify its QR code.
|
||||
- The current production adapter set has no certified image-output route; output-provider certification remains outside version one.
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ Status: implemented
|
||||
|
||||
## 决策
|
||||
|
||||
粘贴或拖放的光栅图片是 Web 输入区对持久附件能力的首个应用场景。未发送文件仍是由客户端持有的临时草稿状态。宿主在追加相应消息事件前,校验并持久提交每张已接受的用户图片。生成结构化图片输出的提供方适配器在追加相应助手块前,也必须持久提交输出。规范用户内容与助手内容只包含角色无关的 `ImageBlock` 引用。
|
||||
粘贴或拖放的光栅图片是 Web 输入区对持久附件能力的首个应用场景。未发送文件仍是由客户端持有的临时草稿状态。每个丰富内容接入适配器都会解码自身协议块、证明路由能力,并在追加消息事件前把完整图片批次委托给附件服务。生成结构化图片输出的提供方适配器在追加相应助手块前,也必须持久提交输出。规范用户内容与助手内容只包含角色无关的 `ImageBlock` 引用。
|
||||
|
||||
第一版支持粘贴和拖放 PNG、JPEG、WebP 与 GIF,支持仅图片或混合提示词,支持渲染历史用户图片与助手图片,并支持单击预览原图(展示与交互细节部分由[附件展示对齐 Note](2026-08-11-web-attachment-display-alignment.md)取代)。文件选择、通用文件、PDF、音频、视频、图片复制和自定义上下文菜单仍分别作为后续工作。
|
||||
|
||||
@@ -114,7 +114,7 @@ type PromptInputPart =
|
||||
}
|
||||
```
|
||||
|
||||
Base64 只跨越一次 JSON-RPC,并在持久化后丢弃。宿主会校验规范 base64、图片数量、总字节数、单张图片字节数、声明的 MIME 与完整解码后的光栅图片是否一致、固有尺寸和解码像素数。它会在保存任何成员之前,等待seam 上不触碰存储的 `validateImage` 完成对每个批次成员的校验,因此一张畸形图片不会把批次中的有效成员留成无引用对象。随后按提交顺序执行存储提交,以限制完整光栅解码器的内存占用。如果后续存储 I/O 操作失败,宿主不会追加用户事件,但先前的不可变内容寻址对象可能保持无引用状态;第一版将清理留给未来按引用感知的垃圾回收,而不向去重存储添加破坏性回滚。只有每张图片都成功后,宿主才会用规范化文本和按提交顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。
|
||||
Base64 只跨越一次协议边界,并在持久化后丢弃。每个入口都会校验规范 base64 与声明的 MIME 形状,再用完整解码批次调用 `AttachmentStore.saveImages()`。服务负责图片数量、总字节数、单张图片字节数、声明 MIME 与完整解码后的光栅图片是否一致、固有尺寸和解码像素数;它会在保存任何成员之前校验每个批次成员,因此一张畸形图片不会把批次中的有效成员留成无引用对象。随后按提交顺序执行存储提交,以限制完整光栅解码器的内存占用。如果后续存储 I/O 操作失败,调用方不会追加模型可见事件,也不会收到部分引用,但先前的不可变内容寻址对象可能保持无引用状态;第一版将清理留给未来按引用感知的垃圾回收,而不向去重存储添加破坏性回滚。只有每张图片都成功后,入口才会用规范化文本和按协议顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。
|
||||
|
||||
`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符,宿主才提供字节。会话处于渲染状态时,客户端会按会话和附件标识符对加载操作去重;已渲染会话释放时会撤销已解析的 URL,并在分配对象 URL 前拒绝已失效的延迟加载,以免已卸载的会话或已释放的服务重新写入缓存。
|
||||
|
||||
@@ -128,7 +128,7 @@ Pi-AI 适配器是首条视觉输入路径:它在请求时解析 `ctx.attachme
|
||||
|
||||
核心层支持结构化助手图片块,但当前没有任何生产提供方路径通过图片输出认证。未来任何支持输出的适配器都必须在有界的大小和时间策略下获取提供方字节,通过同一个附件服务校验并持久化字节,之后才能以原子方式发布 `ImageBlock`。助手 Markdown 中的 URL 仍是文本,绝不自动下载。
|
||||
|
||||
提供方无关的 token 估算不会根据图片尺寸猜测视觉定价;提供方返回的用量仍是权威值。在 ACP(Agent Client Protocol)接口原生支持图片前,ACP 会渲染明确的图片标记,而不是静默省略该块。
|
||||
提供方无关的 token 估算不会根据图片尺寸猜测视觉定价;提供方返回的用量仍是权威值。只有配置的确切路由与附件部署可以接受图片时,ACP(Agent Client Protocol)才公布图片提示词能力;它会在发布用户事件前持久化内联输入,并重新读取已提交的助手图片引用来发送原生 ACP 图片更新。MCP 为程序化调用方保留规范原始块,同时把已准入图片投影为持久核心块;Code Mode 会把任何已经结算且含图片的子结果经外层结果转运为带来源归属且写入日志的上下文。
|
||||
|
||||
压缩会把选定的会话前缀(包含图片引用)回放到已配置的摘要生成路径中。支持视觉的路径会通过适配器解析这些引用;仅文本路径会明确失败,而不是静默丢弃视觉上下文。合成的检查点仍仅包含文本,`compaction-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。
|
||||
|
||||
@@ -148,22 +148,24 @@ Pi-AI 适配器是首条视觉输入路径:它在请求时解析 `ctx.attachme
|
||||
|
||||
| 接口 | 职责 |
|
||||
| --- | --- |
|
||||
| `packages/attachment/attachment` | 不透明附件标识符、图片引用、限制、错误和 `ctx.attachments` 服务。 |
|
||||
| `packages/attachment/attachment` | 不透明附件标识符、图片引用、限制、错误,以及通过 `ctx.attachments` 提供的单张/批量准入。 |
|
||||
| `packages/attachment/attachment-local` | 私有内容寻址存储、完整光栅解码、完整性校验和配置。 |
|
||||
| `packages/llm/llm` | 角色无关的 `ImageBlock` 和输入模态元数据。 |
|
||||
| `packages/llm/llm-pi-ai` | 将持久且受支持的图片输入解析为提供方原生内容。 |
|
||||
| `packages/llm/llm-deepseek` | 明确拒绝图片内容。 |
|
||||
| `packages/compaction/compaction-basic` | 在摘要输入中保留图片,并明确拒绝非文本检查点输出。 |
|
||||
| `packages/host/apiproxy` 和 `packages/bundle/base` | 范围狭窄的上传协议、先持久化再追加事件的顺序、会话授权读取、限制和模型前置检查,以及默认 profile 组合。 |
|
||||
| `packages/host/apiproxy` 和 `packages/bundle/base` | 范围狭窄的上传协议、共享批量准入、限制和路由模型前置检查、先持久化再追加事件的顺序、会话授权读取,以及默认 profile 组合。 |
|
||||
| `packages/client/connection` 和 `packages/client/runtime` | 有界请求缓冲、协议类型、fixture(测试前置数据)图片、提示词上传、附件读取和持久引用折叠。 |
|
||||
| `packages/client/ui-conversation` | 每个会话的草稿图片、附件栏、用户与助手图片控件和原图预览。 |
|
||||
| `packages/acp/acp` | 图片块的明确兜底渲染。 |
|
||||
| `packages/acp/acp` | 条件式原生图片能力、原子内联图片准入,以及经过校验的助手图片交付。 |
|
||||
| `packages/mcp/mcp-client` | 无损规范 MCP 结果、经能力门禁的持久图片投影,以及针对不受支持丰富块的明确诊断。 |
|
||||
| `packages/core/tools` | 在外层结果之后通用转发已经结算且含图片的 Code Mode 子结果。 |
|
||||
|
||||
附件包构成一个能力 seam 的接口与实现侧。输入区行为留在会话对象层,提供方转换留在适配器中,无需修改 `agent-loop`。
|
||||
|
||||
### 实现
|
||||
|
||||
已实现的范围包括附件 seam、角色无关的图片块、Pi-AI 输入转换、DeepSeek 拒绝、宿主持久化顺序、Web 上传与读取协议、当前图片限制执行、大小受限的 Web 请求体、内存草稿图片、粘贴与拖放附件栏、用户与助手历史图片渲染、单击预览、压缩处理,以及组装后无需密钥的 Web 覆盖。
|
||||
已实现的范围包括附件服务边界与共享批量准入、角色无关的图片块、Pi-AI 输入转换、DeepSeek 拒绝、Web/ACP/MCP 的持久化顺序、Web 上传与读取协议、条件式 ACP 图片协议支持、无损 MCP 规范结果与持久图片投影、通用 Code Mode 丰富结果转发、当前图片限制执行、大小受限的 Web 请求体、内存草稿图片、粘贴与拖放附件栏、用户与助手历史图片渲染、单击预览、压缩处理,以及组装后无需密钥的 Web 与 ACP 覆盖。
|
||||
|
||||
预发布提示词协议不需要兼容包装层;引入相应切片时会同时修改所有调用点和 fixture。
|
||||
|
||||
@@ -193,12 +195,25 @@ Pi-AI 适配器是首条视觉输入路径:它在请求时解析 `ctx.attachme
|
||||
|
||||
UI 状态可能陈旧,也无法保护直接 SDK、ACP、回放或未收录模型的路径。静默过滤会改变用户意图。提供方强制检查仍是必需项,UI 检查则是可选的提前反馈。
|
||||
|
||||
### 在核心内容词汇之上添加通用 RichContent 服务
|
||||
|
||||
不予采用,因为核心已经拥有角色无关的 `ContentBlock` 词汇与附件引用。第二套通用服务会重复顺序、能力、日志和生命周期语义,同时每个协议适配器仍需解析自身协议。围绕现有核心构建范围狭窄的图片适配器,可以保持归属清晰,并让音频/资源在确有需要时建立自己的生命周期契约。
|
||||
|
||||
### 把 MCP 结果规范化为核心内容,并将其作为规范工具值
|
||||
|
||||
不予采用,因为 Code Mode 和程序化调用方需要完整 MCP JSON 块及可选 `structuredContent`;用 Native 投影替换该值会让桥接有损。MCP 保留协议值,并另行准备模型投影;最终 post-execute 策略仍具有权威性。
|
||||
|
||||
### 在同步输出渲染器中执行附件读写
|
||||
|
||||
不予采用,因为工具渲染器必须纯净、同步且可回放。MCP 在异步执行期间准备图片投影,只在注册表最终化边界安装;ACP 在自己的传输生命周期中执行异步准入和输出转换。Code Mode 转发观察已经结算的最终内容,而不是让各图片工具各自处理私有父 token 行为。
|
||||
|
||||
## 测试
|
||||
|
||||
- 存储测试覆盖内容寻址去重、私有权限、准入失败、对象损坏或缺失时的失败,以及收紧部署限制后读取历史数据。
|
||||
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体、图片准入与模型选择的竞态(排队与 steering 两种放置)、待发布状态、未发布即空闲时的门槛释放、仅文本的队列编辑,以及压缩后依据当前派生历史进行的选择。
|
||||
- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序、草稿、会话作用域和应用层级的对象 URL 清理,以及一项在释放后才完成的延迟历史读取;keyless 的组装后构建产物通道(`apps/web/tests/image-display.snapshot.ts`,`DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`)覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox,以及 composer 粘贴缩略图条。
|
||||
- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、递归嵌套在工具结果中的图片、保留摘要输入,以及明确拒绝图片输出。
|
||||
- 附件、MCP、ACP 与 Code Mode 测试覆盖写入前校验全部成员、图文混合顺序、持久事件不含内联 base64、确切路由能力门禁、明确的不支持内容诊断、post-execute 替换/阻止优先级、准入期间取消、经过校验的助手图片交付,以及通用嵌套图片转发。组装后的无密钥 ACP 快照发送真实内联 PNG,并在会话日志中只固定其持久引用。
|
||||
- 需要凭据的实际 API 测试会通过 Anthropic `claude-opus-4-8` 路径发送一张 PNG,并要求模型识别其中的二维码。
|
||||
- 当前生产适配器集合没有经过认证的图片输出路由;输出提供方认证仍不在第一版范围内。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-08-native-windows-pull-request-ci.md
|
||||
2026-08-08-native-windows-pull-request-ci.md: d7db73b2049ae2d08f8996bcfd5b54fa15901478
|
||||
2026-08-08-native-windows-pull-request-ci.zh.md: 474b7f71aca8fbb5e0082fd2e4faf8e462bf0c0f
|
||||
2026-08-08-native-windows-pull-request-ci.md: 31a1a1893b0c6248a30ac6e12b409282f608a689
|
||||
2026-08-08-native-windows-pull-request-ci.zh.md: ba2520c2514580e5af367df86dd3195485a0a34c
|
||||
|
||||
@@ -18,7 +18,7 @@ Every pull request also starts an ordinary independent `windows-native` job name
|
||||
|
||||
The native job is deliberately absent from `all-checks-passed.needs` and does not use `continue-on-error`: the aggregate neither waits for it nor changes conclusion because of it, while the job retains its own unmasked result. Workspace build, production-site, and 100%-per-file coverage failures make the native job fail. The broader static, documentation, package, and built-artifact portability inventory remains observational. Linux remains the owner of duplicate lint and snapshot enforcement, while native Windows independently enforces supported-source coverage.
|
||||
|
||||
The 16-core lane gives coverage a two-worker budget, split into one instrumented worker and one exempt-heavy worker, runs two top-level gates concurrently, and allows eight publint workers. Every Vitest project uses forked workers because Node 24's CJS lexer fatal reproduced in shared worker threads on Windows and POSIX; the two-gate schedule prevents the exempt-heavy Oxlint probe from racing the workspace build over its temporary contract files. Asynchronous fixtures whose real process, Git, SQLite, watcher, or lazy grammar startup can exceed Vitest's default polling window use explicit bounded waits without changing their asserted outcomes. The LSP sources and the ACL-sandbox sources remain in the Windows denominator: stub-based failure-path suites carry every in-process ACL-sandbox file to 100%, and only the runner entry stays excluded — it executes exclusively as a spawned child outside the instrumented run, its behavior pinned end-to-end by the runner suite. Narrow annotated V8 ignores cover only unreachable branches (peer-platform arms and lifecycle-unreachable guards), with their behavior tests retained on the owning platform.
|
||||
The 16-core lane gives coverage a two-worker budget, split into one instrumented worker and one exempt-heavy worker, runs two top-level gates concurrently, and allows eight publint workers. Every Vitest project uses forked workers because Node 24's CJS lexer fatal reproduced in shared worker threads on Windows and POSIX; the two-gate schedule prevents the exempt-heavy Oxlint probe from racing the workspace build over its temporary contract files. Both coverage gates set Vitest's default per-test and polling budgets to 30 seconds because unrelated process, Git, SQLite, watcher, grammar, and static-gate fixtures can exceed 15 seconds only under the complete lane's concurrent Windows instrumentation. The script-only translation-pairing merge suite runs in the exempt-heavy gate because it imports only `scripts/` sources and child processes; V8 instrumentation contributes no threshold coverage there but magnifies Git-process latency. Lefthook concurrency fixtures retain their outcomes with 30-second case budgets and a 10-second process-ready probe, while the installer allows five seconds for a preempted lock owner to publish its record after exclusive creation. Workspace-context composition fixtures use a test-owned signal without an unrelated one-second deadline. These lane-scoped budgets preserve asserted outcomes, while the 60-minute job deadline still bounds a stuck run. The LSP sources and the ACL-sandbox sources remain in the Windows denominator: stub-based failure-path suites carry every in-process ACL-sandbox file to 100%, and only the runner entry stays excluded — it executes exclusively as a spawned child outside the instrumented run, its behavior pinned end-to-end by the runner suite. Narrow annotated V8 ignores cover only unreachable branches (peer-platform arms and lifecycle-unreachable guards), with their behavior tests retained on the owning platform.
|
||||
|
||||
The 16-core allocation is the measured capacity point for this inventory. Relative to the previous two-core serial job, six coverage workers produced complete passes in 6 minutes 27 seconds and 7 minutes 50 seconds, but later exact-head repeats exposed unreliable fixtures and worker exits under four, three, and two concurrent instrumented workers. The selected budget therefore reduces that fan-out to one while retaining the exempt-heavy suite as a second concurrent coverage worker and preserving two-way top-level overlap. A 32-core comparison reduced aggregate gate time by only 1.47 seconds and still triggered the CJS-lexer fatal inside a fork worker, so additional cores did not provide a reliable wall-clock improvement.
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Status: implemented
|
||||
|
||||
原生作业被刻意排除在 `all-checks-passed.needs` 之外,且不使用 `continue-on-error`:聚合流程既不等待它,也不会因它改变结论;该作业则保留自身未被掩盖的结果。工作区构建、生产网站和逐文件 100% 覆盖率检查失败会使原生作业失败。更广泛的静态检查、文档、包和构建产物可移植性清单仍作为观测项报告。重复的 lint 与快照强制检查仍由 Linux 负责,原生 Windows 则独立强制执行受支持源码覆盖率。
|
||||
|
||||
16 核通道为覆盖率分配 2 个工作线程,其中 1 个用于插桩套件,1 个用于免覆盖率项较多的套件;同时运行 2 项顶层门禁,并允许 8 个 publint 工作线程。每个 Vitest 项目都使用 fork 工作线程,因为 Node 24 的 CJS lexer 致命故障可在 Windows 与 POSIX 的共享工作线程中复现;双门禁调度可避免免覆盖率项较多的 Oxlint 探测与工作区构建在临时约定文件上发生竞态。对于真实进程、Git、SQLite、watcher 或延迟语法启动可能超过 Vitest 的默认轮询窗口的异步 fixture,系统会使用显式的有界等待,而不改变其断言结果。LSP 源码与 ACL 沙箱源码仍计入 Windows 分母:基于 stub 的失败路径套件把每个进程内 ACL 沙箱文件都带到 100%,只有 runner 入口保持排除——它只作为 spawn 出的子进程在插桩运行之外执行,其行为由 runner 套件端到端钉住。窄范围且带注释的 V8 ignore 只覆盖不可达分支(另一平台专属分支、生命周期内不可达的防御守卫),其行为测试仍保留在所属平台。
|
||||
16 核通道为覆盖率分配 2 个工作线程,其中 1 个用于插桩套件,1 个用于免覆盖率项较多的套件;同时运行 2 项顶层门禁,并允许 8 个 publint 工作线程。每个 Vitest 项目都使用 fork 工作线程,因为 Node 24 的 CJS lexer 致命故障可在 Windows 与 POSIX 的共享工作线程中复现;双门禁调度可避免免覆盖率项较多的 Oxlint 探测与工作区构建在临时约定文件上发生竞态。两项覆盖率门禁都将 Vitest 默认的单测试和轮询时间预算设为 30 秒,因为在完整通道并发的 Windows 插桩下,多个互不相关的进程、Git、SQLite、watcher、语法和静态门禁 fixture 可能超过 15 秒。translation-pairing 合并套件只导入 `scripts/` 源码和子进程,因此放入免覆盖率项较多的门禁;V8 插桩不会为它贡献任何阈值覆盖率,却会放大 Git 进程延迟。Lefthook 并发 fixture 保留原有结果,采用 30 秒单用例预算与 10 秒进程就绪探测;安装器则允许被抢占的 lock 持有者在独占创建后用 5 秒发布记录。workspace-context 组合 fixture 使用测试自有、没有无关 1 秒截止时间的信号。这些只属于该通道的预算保留了原有断言结果,60 分钟的作业截止时间仍会约束卡死的运行。LSP 源码与 ACL 沙箱源码仍计入 Windows 分母:基于 stub 的失败路径套件把每个进程内 ACL 沙箱文件都带到 100%,只有 runner 入口保持排除——它只作为 spawn 出的子进程在插桩运行之外执行,其行为由 runner 套件端到端钉住。窄范围且带注释的 V8 ignore 只覆盖不可达分支(另一平台专属分支、生命周期内不可达的防御守卫),其行为测试仍保留在所属平台。
|
||||
|
||||
16 核配置是这项清单经实测选定的容量规格。与此前的双核串行作业相比,6 个覆盖率工作线程曾分别以 6 分 27 秒和 7 分 50 秒跑出完整通过结果,但后续的分支头精确复跑先后在 4 个、3 个和 2 个插桩工作线程并发时暴露出不稳定的 fixture 与工作线程退出。因此,所选预算将这一扇出降至 1,同时保留免覆盖率项较多的套件作为第二个并发覆盖率工作线程,并继续让两项顶层门禁重叠执行。32 核对比仅将聚合门禁时间缩短 1.47 秒,且仍在 fork 工作线程内触发 CJS lexer 致命故障,因此增加核心数没有带来可靠的墙钟时间改善。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-23-acp-automation-only-protocol.md
|
||||
2026-07-23-acp-automation-only-protocol.md: e7a6670a44db9d4fd2bba5f013083e35d60fc313
|
||||
2026-07-23-acp-automation-only-protocol.zh.md: cdefe80c3e868a41541c7ebea42e3811c7fb9249
|
||||
2026-07-23-acp-automation-only-protocol.md: deeba55ebb48468af80a6c74a704b18e07f33477
|
||||
2026-07-23-acp-automation-only-protocol.zh.md: ab23280ca7d2f39b33b80f1d9fefe977345b8676
|
||||
|
||||
@@ -8,15 +8,17 @@ English | [中文](2026-07-23-acp-automation-only-protocol.zh.md)
|
||||
|
||||
The ACP bridge had become a second interactive product UI. It translated durable events into editor cards, terminal metadata, diffs, plans, titles, reasoning, commands, modes, model and permission pickers, session navigation, and human elicitation. Those responsibilities duplicated the TUI and the Web client while coupling an automation transport to UI services, persistence queries, presentation policy, and editor-specific conventions.
|
||||
|
||||
ACP still has one useful role: another agent or automated controller can start a harness process, create an isolated session, send text, receive the committed answer, cancel work, and answer a permission request. The out-of-process ACP subagent backend depends on that standard protocol boundary.
|
||||
ACP still has one useful role: another agent or automated controller can start a harness process, create an isolated session, send text or a narrowly supported inline image, receive the committed text/image answer, cancel work, and answer a permission request. The out-of-process ACP subagent backend depends on that standard protocol boundary.
|
||||
|
||||
The snapshot suite complicates removal. Most ACP scenarios exercise the assembled agent backend rather than ACP presentation, so deleting the suite with the editor bridge would discard broad keyless behavioral coverage.
|
||||
|
||||
## Decision
|
||||
|
||||
`@deepseek-ai/dsh-acp` is an automation transport under [`packages/acp/acp`](../../../../packages/acp/acp/README.md), outside the `ui` package group. Its public protocol is intentionally small: version negotiation, fresh text sessions with one in-flight prompt each, committed assistant text updates, per-session cancellation, concurrent sessions, and connection-owned teardown. Prompts carry the spec-required baseline only — text plus resource links flattened to bracketed textual references; the bridge rejects additional directories, MCP servers, beyond-baseline prompt content (image, audio, embedded resources), empty prompts, unknown sessions, and overlapping prompts.
|
||||
`@deepseek-ai/dsh-acp` is an automation transport under [`packages/acp/acp`](../../../../packages/acp/acp/README.md), outside the `ui` package group. Its public protocol is intentionally small: version negotiation, fresh sessions with one in-flight prompt each, committed assistant text/image updates, per-session cancellation, concurrent sessions, and connection-owned teardown. Prompts preserve text and supported raster images in wire order, while resource links flatten to bracketed textual references; the bridge rejects additional directories, MCP servers, audio, embedded resources, malformed or empty prompts, unknown sessions, and overlapping prompts.
|
||||
|
||||
The bridge emits only committed `assistant/message` text. Reasoning, raw chunks, tool activity, todos, plans, titles, retry markers, terminal metadata, diffs, locations, and resource links remain in the durable session log or in UI-specific transports. It does not provide session load/list/delete, commands, modes, configuration selectors, model switching, plan review, or human elicitation.
|
||||
Image capability is truthful rather than structural: `initialize` advertises it only when a durable attachment store exists and the configured exact provider/model resolves with explicit image input. Each image prompt rechecks the session's latest exact route, strictly decodes every block, and delegates the complete batch to `AttachmentStore.saveImages()` before publishing the user event. Cancellation reserves and aborts the admission slot before any asynchronous work, waits for already-started writes to quiesce before the prompt settles, and never publishes a late message; before the prompt enters the Agent inbox it neither cancels nor waits for unrelated Agent work. A completed content-addressed write may remain unreachable because destructive rollback is not valid for a deduplicated store. Caller-correctable image-policy failures map to invalid parameters, while route lookup, storage corruption, and persistence failures remain internal faults.
|
||||
|
||||
The bridge emits only committed `assistant/message` text and images. A per-session promise chain preserves block and message order while assistant image references are asynchronously re-read and integrity-verified for ACP base64 delivery; a missing or corrupt object fails prompt delivery instead of becoming a placeholder. Reasoning, raw chunks, tool activity, todos, plans, titles, retry markers, terminal metadata, diffs, locations, and resource links remain in the durable session log or in UI-specific transports. It does not provide session load/list/delete, commands, modes, configuration selectors, model switching, plan review, or human elicitation.
|
||||
|
||||
One-shot `session/request_permission` remains. It is a machine policy channel for bridge-owned agents, not a human approval UI: the answerer accepts only an exact agent object in the bridge's live session map, delegates foreign or call-less requests, and maps failed RPCs to the fail-closed unavailable outcome. The client chooses allow once, reject once, or cancel, and the bridge never turns that response into a durable grant. Asking policy stays in the approval seam and its producers; [`dsh-subagent-acp`](../../../../packages/subagent/subagent-acp/README.md) uses this channel programmatically.
|
||||
|
||||
@@ -24,13 +26,13 @@ The app composition contains the agent spine, persistence, checkpoint policy, an
|
||||
|
||||
The transport programs interface-level agent, session, and approval services rather than the concrete agent loop. Tool execution stays inside the harness; ACP never delegates shell execution to an editor. stdout carries framed JSON-RPC only, so the app mounts no stdout logger and the bridge does not monkey-patch process output.
|
||||
|
||||
Disconnect and plugin disposal share one memoized quiescence boundary. Both successful and failed transport closure settle pending prompts as cancelled, dispose every bridge-owned agent, and await loop and session cleanup. A create that loses the close race disposes its unpublished handle.
|
||||
Disconnect and plugin disposal share one memoized quiescence boundary. Both successful and failed transport closure cancel prompt admission and agents, drain ordered output, settle pending prompts as cancelled, dispose every bridge-owned agent, and await loop and session cleanup. A create that loses the close race disposes its unpublished handle.
|
||||
|
||||
## Snapshot boundary
|
||||
|
||||
The ACP snapshot suite still boots the assembled ACP example and retains scenarios that pin backend behavior. Only scenarios driven through deleted UI methods leave the suite; semantic-checkpoint recovery runs through the headless `stream-json` example because ACP no longer loads sessions.
|
||||
|
||||
Protocol and lifecycle tests pin stop-reason and prompt codecs, version negotiation, fresh-session creation, text and resource-link flattening, rejection of empty or unsupported prompts, exact-agent permission ownership, multi-session isolation, prompt settlement, per-session cancellation, failed transport closure, ACP-only reload cleanup, and teardown quiescence. Built and real-stdio smokes reject stray stdout. The `session/new` branch that loses a real stdio close race remains coverage-exempt because the in-memory transport cannot reproduce that ordering; it disposes the unpublished handle, while the surrounding disposal tests pin the no-orphan invariant.
|
||||
Protocol and lifecycle tests pin stop-reason codecs, version negotiation, truthful image capability, fresh-session creation, ordered text/image admission, resource-link flattening, all-member validation before writes, absence of inline base64 in durable events, rejection of empty or unsupported prompts, exact-agent permission ownership, multi-session isolation, prompt settlement after ordered output, verified assistant-image delivery, cancellation during admission without a late followup or cancellation of unrelated Agent work, exclusion of unrelated pre-inbox failures, failed transport closure, ACP-only reload cleanup, and teardown quiescence. An assembled keyless snapshot sends a real inline PNG through the runnable ACP example and pins only its durable reference in the session log. Built and real-stdio smokes reject stray stdout. The `session/new` branch that loses a real stdio close race remains coverage-exempt because the in-memory transport cannot reproduce that ordering; it disposes the unpublished handle, while the surrounding disposal tests pin the no-orphan invariant.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -44,10 +46,16 @@ Protocol and lifecycle tests pin stop-reason and prompt codecs, version negotiat
|
||||
|
||||
**Delete the ACP snapshot suite or migrate every scenario in this change.** Rejected because most scenarios test the backend and remain valuable, while a full harness migration is an independent testing change. Only scenarios whose driver was a deleted UI method leave this suite.
|
||||
|
||||
**Advertise image support whenever the ACP SDK has an image block.** Rejected because protocol vocabulary does not prove this deployment can persist bytes or that the configured exact route accepts visual input. Unknown capability is false at initialization; prompt admission rechecks the live route.
|
||||
|
||||
**Flatten inline and assistant images to markers or persist ACP base64 in session events.** Rejected because markers silently lose model/user intent and base64 makes durable logs the binary store. ACP translates between its wire block and the existing durable `ImageBlock` reference at the transport boundary.
|
||||
|
||||
**Create a generic RichContent service for ACP, MCP, and Web.** Rejected because core `ContentBlock` plus the attachment seam already own the shared contract. Each front door keeps only protocol parsing, capability proof, and lifecycle orchestration; shared batch limits and image validation stay in `AttachmentStore.saveImages()`.
|
||||
|
||||
## Consequences
|
||||
|
||||
ACP has a narrow contract suitable for agents and automation, while TUI and Web own human interaction and presentation. The package has fewer injected services, dependencies, protocol branches, and lifecycle states, and it no longer claims compatibility as a general editor entry point.
|
||||
|
||||
Automation clients receive complete committed text rather than token deltas or structured tool UI. They inspect durable logs or another API when they need reasoning, tool traces, titles, or richer state. Fresh-session-only operation also means callers that need durable browsing or resume use a host API rather than ACP.
|
||||
Automation clients receive complete committed text/images rather than token deltas or structured tool UI. They inspect durable logs or another API when they need reasoning, tool traces, titles, or richer state. Fresh-session-only operation also means callers that need durable browsing or resume use a host API rather than ACP.
|
||||
|
||||
Backend snapshot coverage therefore remains transport-coupled to ACP even though that transport is incidental to the behavior under test.
|
||||
|
||||
@@ -8,15 +8,17 @@ Status: implemented
|
||||
|
||||
ACP(Agent Client Protocol)桥接层已经变成第二套交互式产品 UI。它将持久事件转换为编辑器卡片、终端元数据、diff、计划、标题、推理(reasoning)、命令、模式、模型和权限选择器、会话导航以及面向人类的询问。这些职责与 TUI 和 Web 客户端重复,同时将自动化传输层与 UI 服务、持久化查询、展示策略和编辑器特定约定耦合在一起。
|
||||
|
||||
ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控制器可以启动 harness 进程、创建隔离会话、发送文本、接收已提交的回答、取消工作并回答权限请求。跨进程 ACP subagent 后端依赖这个标准协议边界。
|
||||
ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控制器可以启动 harness 进程、创建隔离会话、发送文本或范围狭窄的受支持内联图片、接收已提交的文本/图片回答、取消工作并回答权限请求。跨进程 ACP subagent 后端依赖这个标准协议边界。
|
||||
|
||||
快照套件使移除工作更复杂。大多数 ACP 场景测试的是组装后的 agent 后端,而不是 ACP 展示层;如果随编辑器桥接层一起删除整个套件,就会丢失大量无密钥行为测试。
|
||||
|
||||
## 决策
|
||||
|
||||
`@deepseek-ai/dsh-acp` 是位于 [`packages/acp/acp`](../../../../packages/acp/acp/README.md) 下、独立于 `ui` 包组的自动化传输层。其公开协议特意保持精简:版本协商、全新文本会话(每个会话最多允许一个进行中的提示词)、已提交的助手文本更新、按会话取消、并发会话,以及由连接负责的资源清理。提示词只承载规范要求的基线内容——文本,加上被展平为方括号文本引用的资源链接;桥接层会拒绝附加目录、MCP 服务器、超出基线的提示词内容(图片、音频、内嵌资源)、空提示词、未知会话和重叠提示词。
|
||||
`@deepseek-ai/dsh-acp` 是位于 [`packages/acp/acp`](../../../../packages/acp/acp/README.md) 下、独立于 `ui` 包组的自动化传输层。其公开协议特意保持精简:版本协商、全新会话(每个会话最多允许一个进行中的提示词)、已提交的助手文本/图片更新、按会话取消、并发会话,以及由连接负责的资源清理。提示词按协议顺序保留文本与受支持光栅图片,资源链接则展平为方括号文本引用;桥接层会拒绝附加目录、MCP 服务器、音频、嵌入资源、格式错误或空提示词、未知会话和重叠提示词。
|
||||
|
||||
桥接层只发出已提交的 `assistant/message` 文本。推理、原始分片、工具活动、待办事项、计划、标题、重试标记、终端元数据、diff、位置和资源链接仍保留在持久会话日志或 UI 专用传输层中。它不提供会话加载、列出与删除、命令、模式、配置选择器、模型切换、plan 评审或面向人类的询问。
|
||||
图片能力必须真实,而不能只看结构:只有持久附件存储存在,且配置的确切提供方/模型解析后明确支持图片输入时,`initialize` 才会公布该能力。每个图片提示词都会重新检查会话的最新确切路由、严格解码全部块,并在发布用户事件前把完整批次委托给 `AttachmentStore.saveImages()`。取消会在任何异步工作前预留并中止准入槽位,使提示词在已经启动的写入停稳后才结算,而且绝不发布迟到消息;提示词进入 Agent inbox 前既不会取消,也不会等待无关的 Agent 工作。已经完成的内容寻址写入可能保持不可达,因为对去重存储执行破坏性回滚并不正确。可由调用方修正的图片策略失败会映射为无效参数,路由查询、存储损坏和持久化失败则仍属于内部故障。
|
||||
|
||||
桥接层只发出已提交的 `assistant/message` 文本与图片。每个会话使用一条 Promise 链,在异步重新读取并校验助手图片引用、将其转换为 ACP base64 交付时保持块与消息顺序;对象缺失或损坏会使提示词交付失败,而不是变成占位符。推理、原始分片、工具活动、待办事项、计划、标题、重试标记、终端元数据、diff、位置和资源链接仍保留在持久会话日志或 UI 专用传输层中。它不提供会话加载、列出与删除、命令、模式、配置选择器、模型切换、plan 评审或面向人类的询问。
|
||||
|
||||
保留一次性 `session/request_permission`。它是为桥接层拥有的 agent 提供的机器策略通道,而不是面向人类的审批 UI:应答者只接受桥接层当前会话映射中登记的同一 agent 对象;不属于桥接层当前 agent 的请求或未关联具体调用的请求会继续委派;RPC 失败则映射为故障时默认拒绝的 `unavailable` 结果。客户端可选择允许一次、拒绝一次或取消,桥接层绝不会将该响应转换为持久授权。询问策略仍归审批 seam 及其生产者所有;[`dsh-subagent-acp`](../../../../packages/subagent/subagent-acp/README.md) 会以程序化方式使用该通道。
|
||||
|
||||
@@ -24,13 +26,13 @@ ACP 仍有一个有用的职责:另一个 agent(智能体)或自动化控
|
||||
|
||||
传输层调用 agent、会话和审批的接口服务,而不依赖具体的 agent loop(智能体循环)。工具执行仍留在 harness 内;ACP 绝不会把 shell 执行委派给编辑器。stdout 只承载分帧 JSON-RPC,因此 app 不挂载 stdout logger,桥接层也不会 monkey-patch 进程输出。
|
||||
|
||||
断开连接与插件 dispose(资源释放)共享同一个经记忆化的完全停稳边界。传输关闭无论成功还是失败,都会将待处理提示词以已取消状态结算,dispose 每个由桥接层拥有的 agent,并等待 agent loop 和会话清理完成。创建流程如果在与关闭的竞态中落败,就会 dispose 其尚未发布的 handle。
|
||||
断开连接与插件 dispose(资源释放)共享同一个经记忆化的完全停稳边界。传输关闭无论成功还是失败,都会取消提示词准入和 agent、排空有序输出、将待处理提示词以已取消状态结算、dispose 每个由桥接层拥有的 agent,并等待循环和会话清理完成。创建流程如果在与关闭的竞态中落败,就会 dispose 其尚未发布的 handle。
|
||||
|
||||
## 快照边界
|
||||
|
||||
ACP 快照套件仍会启动组装后的 ACP 示例,并保留用于锁定后端行为的场景。从该套件移出的只有通过已删除的 UI 方法驱动的场景;由于 ACP 不再加载会话,语义检查点恢复通过 headless `stream-json` 示例执行。
|
||||
|
||||
协议与生命周期测试会锁定停止原因编解码器和提示词编解码器、版本协商、新会话创建、文本与资源链接展平、拒绝空提示词或不受支持的提示词、基于同一 agent 对象的权限归属、多会话隔离、提示词结算、按会话取消、传输关闭失败、ACP 专属重载清理,以及拆卸完全停稳。构建产物冒烟测试与真实 stdio 冒烟测试会拒绝混入 stdout 的额外输出。`session/new` 中在真实 stdio 关闭竞态中落败的分支仍豁免覆盖率要求,因为内存传输层无法复现这一顺序;该分支会 dispose 尚未发布的 handle,而周边 dispose 测试会锁定无遗留资源不变式。
|
||||
协议与生命周期测试会锁定停止原因编解码器、版本协商、真实图片能力、新会话创建、有序文本/图片准入、资源链接展平、写入前校验全部成员、持久事件中不含内联 base64、拒绝空提示词或不受支持的提示词、基于同一 agent 对象的权限归属、多会话隔离、在有序输出后结算提示词、经过校验的助手图片交付、准入期间取消且不产生迟到 followup 或取消无关 Agent 工作、排除进入 inbox 前的无关失败、传输关闭失败、ACP 专属重载清理,以及拆卸完全停稳。组装后的无密钥快照通过可运行 ACP 示例发送一张真实内联 PNG,并在会话日志中只固定其持久引用。构建产物冒烟测试与真实 stdio 冒烟测试会拒绝混入 stdout 的额外输出。`session/new` 中在真实 stdio 关闭竞态中落败的分支仍豁免覆盖率要求,因为内存传输层无法复现这一顺序;该分支会 dispose 尚未发布的 handle,而周边 dispose 测试会锁定无遗留资源不变式。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
@@ -44,10 +46,16 @@ ACP 快照套件仍会启动组装后的 ACP 示例,并保留用于锁定后
|
||||
|
||||
**删除 ACP 快照套件,或在本次变更中迁移每个场景。** 不予采用,因为大多数场景测试后端且仍有价值,而完整的 harness 迁移是一项独立的测试变更。只有通过已删除的 UI 方法驱动的场景才离开该套件。
|
||||
|
||||
**只要 ACP SDK 具有图片块就公布图片支持。** 不予采用,因为协议词汇不能证明当前部署可以持久化字节,也不能证明配置的确切路由接受视觉输入。初始化时能力未知即为 false;提示词准入会重新检查实时路由。
|
||||
|
||||
**把内联图片和助手图片展平为标记,或把 ACP base64 持久化进会话事件。** 不予采用,因为标记会静默丢失模型/用户意图,base64 则会让持久日志变成二进制存储。ACP 在传输边界把自身协议块与现有持久 `ImageBlock` 引用相互转换。
|
||||
|
||||
**为 ACP、MCP 和 Web 创建通用 RichContent 服务。** 不予采用,因为核心 `ContentBlock` 与附件 seam 已经拥有共享契约。每个入口只保留协议解析、能力证明与生命周期编排;共享批次限制和图片校验留在 `AttachmentStore.saveImages()` 中。
|
||||
|
||||
## 结果
|
||||
|
||||
ACP 具有适合 agent 与自动化的精简约定,而 TUI 和 Web 拥有面向人类的交互与展示。该包注入的服务、依赖、协议分支和生命周期状态更少,也不再将自身定位为通用编辑器入口。
|
||||
|
||||
自动化客户端收到完整的已提交文本,而不是 token 增量或结构化工具 UI。当它们需要推理、工具跟踪信息、标题或更丰富的状态时,需要查看持久日志或其他 API。只支持全新会话也意味着,需要浏览持久会话或恢复会话的调用方必须使用 host API,而不是 ACP。
|
||||
自动化客户端收到完整的已提交文本/图片,而不是 token 增量或结构化工具 UI。当它们需要推理、工具跟踪信息、标题或更丰富的状态时,需要查看持久日志或其他 API。只支持全新会话也意味着,需要浏览持久会话或恢复会话的调用方必须使用 host API,而不是 ACP。
|
||||
|
||||
因此,后端快照测试仍与 ACP 传输层耦合,尽管对于受测行为而言,该传输层只是附带因素。
|
||||
|
||||
Reference in New Issue
Block a user