refactor(code-runtime): remove subprocess backend

This commit is contained in:
Tianyi Cui
2026-07-30 14:24:47 +08:00
parent c1d08edd83
commit 4d5345794d
44 changed files with 192 additions and 1614 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-06-15-code-mode.md
2026-06-15-code-mode.md: a075ea66c150afb1b48f98c600b72eec57869962
2026-06-15-code-mode.zh.md: c269c0a8709087e7b1e27051cbda3924f8dd9109
2026-06-15-code-mode.md: b6a24ecd9700e32912b8112b59cbd8b6ab131eb5
2026-06-15-code-mode.zh.md: 4d0a4cf8fa31cf9d9954e5bd95f823dfc0668444

View File

@@ -20,7 +20,7 @@ Three decisions, each elaborated in its own section below:
1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation.
2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign.
3. **Two implementations preserve one fresh-worker contract**: `@deepseek-ai/dsh-code-runtime-worker` runs the worker in the harness process, while `@deepseek-ai/dsh-code-runtime-subprocess` passes a bundled eval runner to `ctx.subprocess` for another execution world. Both execute host-stripped TypeScript in a fresh Node worker with an empty environment, bridged bindings, configurable heap/output/time caps, and hard termination. Their trust posture is bash-equivalent by design; stronger isolation comes from the mounted execution world.
3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority.
This note owns Code Mode's presentation, composition, isolation, and settlement foundation. The later [typed tool-return Agent Note](2026-07-20-code-mode-typed-tool-returns.md) owns the generated output map, canonical binding values, `ToolCallError`, and the lossless outer-output boundary.
@@ -64,7 +64,7 @@ Each sub-dispatch appends a log-only `tool/code-dispatch-start` event at pool en
- `CodeBindingNamespace = { global: string; functions: Record<string, (args: unknown) => Promise<CodeJsonValue>>; errorClass?: { name: string; memberNameProperty: string } }` — the runtime exposes each namespace as a global object of async functions inside the program; the optional descriptor asks the runtime to inject a real program-visible rejection class without teaching the seam consumer-specific names. `CodeJsonValue` is this dependency-light seam's structural lossless-JSON type, so binding arguments and resolutions cross the implementation's serialization boundary whole.
- `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }` — program execution outcomes resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary.
- `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout, a lossy completion is not an overflow, and a substrate exit is none of them.
- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for both shipped backends; a Python backend would pair with its own SDK generator) and `isolation` (`'worker-thread'` for both shipped backends; the subprocess provider may add a container boundary). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all).
- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all).
Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator.
@@ -79,11 +79,9 @@ Requests contain every runtime input; implementations own validated timeout and
5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. `maxOutputBytes` bounds only the combined serialized outer logs, completion, or diagnostic; intermediate binding values have no byte cap. Expiry, cancellation, and completion terminate the worker, and heap exits or outer overflow are explicit failures.
6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md).
`@deepseek-ai/dsh-code-runtime-subprocess` preserves those program, binding, output, and worker-budget semantics across another process world. It passes a dependency-free eval runner directly through the subprocess provider and carries binding traffic over bounded newline-delimited UTF-8 JSON frames on raw pipes. The heap-bounded worker rejects expanded completion wires before MessagePort transfer; terminal settlement asks the launcher to reap its controller and keeps process-group escalation armed until `waitForExit()` confirms whole-tree quiescence. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns why this generic backend replaces provider-specific Code Runtime packages; `dsh-code-runtime-worker` remains the single-process and single-file-distribution path.
### Trust posture
The runtimes provide containment, not an independent security boundary: model code can reach Node APIs and has the authority of its mounted execution world. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty worker environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary mount container-class filesystem/subprocess providers for both code and bash.
The worker runtime provides containment, not a security boundary: model code can reach Node APIs and has authority comparable to the bash tool. `worker.terminate()` stops the thread but not OS processes it spawned. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary need a container-class backend for both code and bash; the runtime's isolation descriptor lets them distinguish that backend.
### What the model sees
@@ -95,7 +93,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem
## Testing
- **Runtime implementations:** Real-worker suites cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger and per-hop frame boundaries, compute and wall budgets, hostile binding traffic, empty environment, descendant lifetime cleanup, and disposal to quiescence. Built-package tests run both the direct worker entry and the subprocess composition under plain Node; the latter also has a Loader-driven `cordis.yml` test.
- **Worker runtime:** Real-worker tests cover typed binding values and failures, every lossless JSON completion root, invalid and over-limit output, exact combined ledger boundaries, compute and wall budgets, hostile binding traffic, empty environment, and disposal to quiescence. A built-package test runs the worker entry under plain Node.
- **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup.
- **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior.
- **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards.
@@ -120,7 +118,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem
## Risks
**A worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool and gating uses the same seams. The subprocess implementation can run inside a container-class execution world, but its worker descriptor does not itself claim that boundary.
**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design.
**`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end.
@@ -128,7 +126,7 @@ Deployments switching to `'code'` must update any native-only `toolOrder`. Assem
**Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`.
**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The direct worker runtime applies no per-binding byte cap; the subprocess runtime bounds each transport frame with `maxFrameBytes`, but repeated or concurrent calls can still consume process or worker memory. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is separately bounded by `maxOutputBytes`.
**Large lossless JSON values can exhaust memory.** Tool bindings snapshot lossless JSON before dispatch and return canonical JSON resolutions whole. The runtime validates both sides of the worker port and applies no per-binding byte cap; structured-clone cost and process or worker memory are the practical bounds. The combined outer-output ledger for logs, the completion value, and a failure diagnostic is the only byte-capped boundary.
**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs.

View File

@@ -20,7 +20,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
1. **Code Mode 是 `ToolRegistry`(`dsh-tools`)的一等呈现模式**,通过经校验的 `mode` 配置选择:`'native'`(默认,贡献可见能力 schema)、`'code'`(注册表仅贡献其保留的 `run_code` 传输通道加一份生成的 SDK `.d.ts` 到系统提示词中)或 `'both'`(原生 schema 加传输通道 + SDK)。注册表在源头塑造其权威贡献;协作式提示词组装的结果仍具权威性,记录在日志中的请求头精确反映该返回的呈现。
2. **代码执行是一个能力 seam**——`packages/code-runtime/` 包含接口包 `@deepseek-ai/dsh-code-runtime`,拥有 `ctx.codeRuntime`([能力 seam](../architecture/2026-06-13-capability-seams.md);消费方 = `dsh-tools`,core 消费 seam 的先例见 `agent-loop` → `dsh-llm`)。运行时对工具一无所知:它接收一段程序和命名的异步绑定,执行程序,报告 `{ value, logs, error? }`。语言和基底是后端属性,因此未来的 Python 或容器后端只是另一个实现包,而非重新设计。
3. **两种实现保持同一份全新 worker 契约**:`@deepseek-ai/dsh-code-runtime-worker` 在 harness 进程内运行 worker,`@deepseek-ai/dsh-code-runtime-subprocess` 则将一个内置的 eval runner 传给 `ctx.subprocess`,用于另一执行环境。二者都在具有空环境的全新 Node worker 内执行由宿主剥离类型的 TypeScript,并提供桥接绑定、可配置的堆/输出/时间上限和硬终止。其信任姿态在设计上等同于 bash;更强的隔离来自挂载的执行环境。
3. **交付的实现是 `@deepseek-ai/dsh-code-runtime-worker`**:每次运行 spawn 一个全新的 Node worker 线程,对模型的 TypeScript 进行 type-strip 后执行,绑定通过消息端口桥接,环境为空,堆/输出/时间上限可配置,并支持硬终止。其信任姿态在设计上等同于 bash——无需 unsafe-acknowledgement flag——因为 harness 已经交付了 `dsh-bash-local`,后者以严格*更高*的环境权限执行模型编写的任意 shell 命令。
本说明负责定义 Code Mode 的呈现、组合、隔离与结算基础。后续的[类型化工具返回值 Agent Note](2026-07-20-code-mode-typed-tool-returns.md)负责定义生成的输出映射、规范绑定值、`ToolCallError` 和无损外层输出边界。
@@ -64,7 +64,7 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
- `CodeBindingNamespace = { global: string; functions: Record<string, (args: unknown) => Promise<CodeJsonValue>>; errorClass?: { name: string; memberNameProperty: string } }`——运行时将每个命名空间作为程序内部的全局异步函数对象暴露;可选描述符要求运行时注入真正的、程序可见的 reject 类,而无需让 seam 获知消费方专用名称。`CodeJsonValue` 是这个低依赖 seam 的结构化无损 JSON 类型,因此绑定参数与解析值可以完整跨越实现的序列化边界。
- `CodeRunResult = { value?: CodeJsonValue; logs: string[]; error?: CodeRunFailure }`——程序执行失败时,执行 promise 仍会 fulfill,并通过 `error` 字段返回失败结果。只有调用方/seam 误用(例如重复的绑定命名空间)时,`run()` 才会 reject;消费方仍在自己的错误边界处理不合规后端的拒绝。
- `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit' | 'invalid-output' | 'output-limit'; message: string }`——按[防御性模式](../../../../docs/defensive-patterns.md)独立报告的正交结果;超时的 run 不是异常,abort 不是超时,有损完成值不是溢出,基底退出也与上述情况相互独立。
- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——两种已交付后端均为 `'typescript'`;Python 后端会配对自己的 SDK 生成器)和 `isolation`(两种已交付后端均为 `'worker-thread'`;子进程提供方可以增加容器边界)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。
- 两个只读的后端描述符,仅供信息参考而非门禁判定:`language`(程序必须使用的语言——交付的后端为 `'typescript'`;Python 后端会声明自己,并在呈现侧配对自己的 SDK 生成器)和 `isolation`(交付的后端为 `'worker-thread'`;未来可为 `'process'`、`'container'` 等)。`dsh-tools` 在 MVP 中要求 `language === 'typescript'`——其代码生成输出 TS——否则组装会大声失败,与 `toolOrder` 违规时的配置错误惯用法相同(如 `mode` 为非 native 但根本没有加载 `ctx.codeRuntime`)。
请求包含所有运行时输入;实现方拥有经校验的超时和上限默认值。注册表仅在组装 Code Mode 时查找可选的运行时,因此 native 模式不依赖它。缺失或语言不兼容的运行时会大声失败。替代基底或语言可以在同一 seam 背后替换实现,配对相应的 SDK 生成器。
@@ -79,11 +79,9 @@ Cloudflare 的 [Code Mode](https://blog.cloudflare.com/code-mode/) 提出了一
5. **强制独立预算。** `computeMs` 计量 worker 忙碌时间,允许慢速的 awaited 工具而不放过热循环。`maxWallMs` 约束总经过时间,包括未解析的等待。`maxOutputBytes` 只约束序列化后的外层日志、完成值或诊断的组合;中间绑定值没有字节数上限。到期、取消和完成都终止 worker,堆退出或外层溢出会作为显式失败报告。
6. **dispose 至完全停稳**:服务自身的 dispose(资源释放)终止进行中的 worker 并*等待*其退出后再 resolve,遵循[防御性模式](../../../../docs/defensive-patterns.md)。
`@deepseek-ai/dsh-code-runtime-subprocess` 在另一进程执行环境中保持相同的程序、绑定、输出与 worker 预算语义。它通过子进程提供方直接传递一个无依赖的 eval runner,并在原始管道上使用有界的、以换行符分隔的 UTF-8 JSON 帧承载绑定通信。受堆上限约束的 worker 会在通过 MessagePort 传输前拒绝展开后的完成值 wire;终态结算会请求 launcher 回收其 controller,并让进程组升级终止机制保持待命,直至 `waitForExit()` 确认整棵进程树完全停稳。[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)说明为何这个通用后端会取代提供方专用的 Code Runtime 包;`dsh-code-runtime-worker` 仍用于单进程和单文件发行版。
### 信任姿态
这些运行时提供的是隔离,而非独立安全边界:模型代码可以访问 Node API,并拥有挂载的执行环境所授予的权限。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空 worker 环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署应当为代码和 bash 都挂载容器级文件系统/子进程提供方。
worker 运行时提供的是隔离,而非安全边界:模型代码可以访问 Node API,权限与 bash 工具相当。`worker.terminate()` 停止线程但不停止它 spawn 的 OS 进程。Code Mode 使用与 bash 相同的 `tools/pre-execute` 策略门禁,并额外提供空环境、堆限制、独立 isolate 和对程序本身的硬终止。需要硬多租户边界的部署需要为代码和 bash 都使用容器级后端;运行时的 isolation 描述符让它们能区分该后端。
### 模型看到的内容
@@ -95,7 +93,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw
## 测试
- **运行时实现:** 真实 worker 测试套件覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界与逐跳帧边界、compute 和 wall 预算、恶意绑定流量、空环境、后代进程生命周期清理以及 dispose 至完全停稳。构建后包测试会在纯 Node 下分别运行直接 worker 入口与子进程组合;后者另有一个由 Loader 驱动的 `cordis.yml` 测试。
- **Worker 运行时:** 真实 worker 测试覆盖类型化的绑定值与失败、每一种无损 JSON 根类型的完成值、无效和超限输出、精确的组合账本边界、compute 和 wall 预算、恶意绑定流量、空环境以及 dispose 至完全停稳。一个构建后包测试在纯 Node 下运行 worker 入口。
- **注册表集成:** 测试覆盖代码生成、所有呈现模式、保留名称和限制规则、scoped 可见性、权威组装重写、`toolOrder`、运行时兼容性失败、完整流水线子分发、parent-token 关联、序列化、取消和队列排空、JSON 规范化、错误传播、日志事件、成功与失败程序中的有序上下文延后、外层阻止抑制以及 HMR(热模块替换)清理。
- **带密钥 e2e:** 真实模型在一个程序中组合两次 bash 调用;另一个模型通过 Code Mode fs 分发发现嵌套的工作区指令。测试验证折叠的请求头、关联的分发事件、结果文件、延后上下文和模型行为。
- **快照:** `code-mode-turn`、`both-mode-turn` 和 `code-mode-workspace-context` fixture(测试前置数据)固定 SDK 文本、请求头工具列表、分发事件、延后上下文和结果卡片。
@@ -120,7 +118,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw
## 风险
**worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,门禁使用相同的 seam。子进程实现可以在容器级执行环境中运行,但其 worker 描述符本身不声称具备该边界。
**Worker 不是硬安全边界。** 有意为之且已文档化(§信任姿态):姿态等同于既有的 bash 工具,隔离程度超过它,门禁使用相同的 seam。需要更强隔离的部署需要未来的 `isolation: 'container'` 后端——作为 seam 设计的扩展点跟踪,而非本设计的 TODO。
**`stripTypeScriptTypes` 标记为 experimental。** 它与 Node 自身原生 `.ts` 执行背后的引擎(amaro/swc)相同,在本仓库的整个引擎范围内作为 API 暴露。缓解措施:运行时的单元测试套件固定了所依赖的行为(位置保持、可擦除限制的拒绝消息形状宽松匹配),调用位于一个私有函数之后,且 `amaro`/`sucrase` 是 API 变化时的直接替代品。仅可擦除子集是面向模型的契约线,错误路径是一个可工作的反馈循环,而非死胡同。
@@ -128,7 +126,7 @@ SDK 指示模型编写一个异步的可擦除 TypeScript 函数体,通过 `aw
**注册表 scope 增长。** `dsh-tools` 吸收了代码生成、一个工具、一个桥和一个事件。通过包内的模块边界(`ts-types.ts`、`code-mode.ts` 与 `schema.ts`/`json-schema.ts`/`presentation.ts` 并列)和 seam 约束:所有基底相关的内容都在 `ctx.codeRuntime` 之后。
**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。直接 worker 运行时不对单次绑定设置字节数上限;子进程运行时使用 `maxFrameBytes` 限制每个传输帧,但重复或并发调用仍可能消耗进程或 worker 内存。包含日志、完成值和失败诊断的组合外层输出账本另由 `maxOutputBytes` 限制。
**大型无损 JSON 值可能耗尽内存。** 工具绑定会在分发前对无损 JSON 创建快照,并完整返回规范 JSON 解析值。运行时会校验 worker 端口两侧,但不对单次绑定设置字节数上限;结构化克隆成本以及进程或 worker 内存构成实际边界。只有包含日志、完成值和失败诊断的组合外层输出账本受字节数上限约束。
**仅序列化的子分发。** `Promise.all` 尚未获得挂钟并行性,仅减少往返次数;模型可能过度期望。说明中已声明;解除此限制与原生并行分发 TODO 所需的并发安全元数据绑定。