refactor(e2b): keep provider layer substrate-only

This commit is contained in:
Tianyi Cui
2026-07-29 19:07:11 +08:00
parent 65b345c96a
commit 917a7493f7
28 changed files with 280 additions and 578 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md
2026-07-28-portable-execution-world-consumers.md: 402aa580255a5bd6aa0d046e3bcc16f712da520d
2026-07-28-portable-execution-world-consumers.zh.md: 26d493a79f8adb9e729ff69b0673f2f2775868b0
2026-07-28-portable-execution-world-consumers.md: 250445cfed6a4fe2dd188562813d586df29992c1
2026-07-28-portable-execution-world-consumers.zh.md: 361c4de4e813aa7f57a7eb5acc75087b2630db43

View File

@@ -8,6 +8,8 @@ English | [中文](2026-07-28-portable-execution-world-consumers.zh.md)
The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations.
A remote coding world is useful only when file operations, commands, terminals, language servers, and model-written programs share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment.
Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract.
## Decision
@@ -24,10 +26,26 @@ Generic consumers use that execution world:
- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged.
- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation.
## E2B POC boundary
The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates or reconnects one sandbox and owns kill/pause/leave disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole sandbox identity from the owner and never create private sandboxes.
E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, subprocess Code Runtime processes, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, Code Runtime program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace.
The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement.
Retaining a sandbox preserves remote files and unmanaged state only. Reconnect does not reconstruct host process or terminal handles, protocol connections, pending calls, output cursors, timers, or locks. The POC adds no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution.
## Verification
Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, post-rename version reread plus guarded edit, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP document bounds, Code Runtime bindings/limits/cleanup, host-workspace isolation, and final sandbox deletion.
## Alternatives considered
**Keep one PTY and LSP package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it.
**Create a separate sandbox per capability or tool.** Rejected because file and process operations would not share identity or state, defeating the coding use case and multiplying lifecycle owners.
**Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches.
**Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means.
@@ -38,6 +56,12 @@ Generic consumers use that execution world:
**Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop.
**Put every provider operation in one shared owner package.** Rejected because sandbox identity and lifecycle are the owner's only concerns. Filesystem and subprocess retain distinct contracts, tests, and consumers without turning the owner into a capability grab bag.
**Implement remote filesystem operations only through shell commands.** Rejected because that discards structured filesystem identity, errors, streaming, version guards, and atomic mutation semantics already consumed by the file tools.
**Add a generic distributed-runtime abstraction or reconnect live handles.** Rejected because the existing capability seams carry the demonstrated contracts, while remote identity alone cannot reconstruct callbacks, pending promises, authority, protocol state, or output cursors. A new layer would speculate about persistence and synchronization beyond the POC.
## Consequences
A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, and LSP compose above them, so fixes to those capabilities remain provider-neutral.
@@ -45,3 +69,5 @@ A remote execution provider implements only its shared sandbox owner plus filesy
The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers.
The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer.
The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, retained artifacts accumulate, and escaped processes or reconnect state are not recovered. These are provider constraints, not justification for compatibility shims or more E2B packages.

View File

@@ -6,7 +6,9 @@ Status: implemented
## 问题
文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但 PTY 和 LSP 仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY 与 LSP 包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。
文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性,但若干上层能力仍直接调用宿主 Node API。因此,即使领域行为没有变化,远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包(package)。这些包只会成为浅层适配器:每个包都仅为替换文件与进程操作而复制一个现有消费方。
只有文件操作、命令、终端、语言服务器和模型编写的程序共享同一个沙箱身份时,远程编码世界才有用。若把完整 harness 移入该沙箱,还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。
普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号,以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作,最终不是泄漏提供方内部细节,就是削弱其生命周期契约。
@@ -16,16 +18,36 @@ Status: implemented
文件系统接口负责其他能力需要的路径事实,同时不公开其不透明目标身份:规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责;协议消费方在消费流时执行各自的保留上限。
进程管理接口负责可执行文件查找与进程原语:以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。
进程管理接口负责进程运行坐标与原语:规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程,以及 `spawnTerminal()`。终端操作是一项深层原语,其句柄负责文本 I/O、前台进程组、信号发送,以及一项须等待的 TERM→KILL 操作;该操作会结算所有在途句柄调用,并使整个会话完全停稳。其信号只取消分配;句柄一经发布,便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。
通用消费方使用该执行世界:
- `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。
- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系,通过 `ctx.subprocess` 解析和启动语言服务器,并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作,包括取得队列所有权之前的工作区查找;其 JSON-RPC、池化、同步和规范化保持不变。
- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`;受限模式要求同一执行世界中存在沙箱提供方,未挂载时会在 spawn 前失败。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;在途就绪检查无法释放该预留,写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。启动取消会立即开始终端回滚,而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号,并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。
- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`;其他进程管理提供方则提供相同原语。提供方开始写入时,系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留,随后向前台进程组发送信号,因此延迟字节和该信号都无法落到后续发送;写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后,会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号,并把完整会话的完全停稳委托给句柄上须等待的终止操作。
- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner,并通过 `ctx.subprocess` 启动它,从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施,因此其写入携带显式 `danger-full-access` 策略,而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制,而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找,使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧;每个外层转发环节都会在转发前执行相同的上限检查;原始子进程管道承载以换行符分隔的 UTF-8 JSON,无需冗余的 base64 表示;launcher 会在回收 controller 前发布已接纳的终态帧,使继承 controller 管道的后代进程无法阻止完成;宿主仍会等待进程组完全停稳。
`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端,可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统/进程组合选择 `dsh-code-runtime-subprocess`;它们不需要提供方专用的代码运行时包。
## E2B POC 边界
可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包:`dsh-e2b` 创建或重新连接一个沙箱,并负责 kill、pause 或 leave 资源释放;`dsh-fs-e2b` 实现 `ctx.fs`;`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的沙箱身份,绝不创建私有沙箱。
E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取、子进程代码运行时进程,以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop(智能体循环)、agent(智能体)状态、会话状态与目标状态、会话日志与持久化、LLM(大语言模型)调用、提示词与工具、权限、skill(技能)、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、代码运行时程序/绑定/输出策略,以及 E2B SDK/网络缓冲区。该叠加层既不上传,也不同步宿主工作区。
适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输;流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64,避免 SDK 分片解码丢失字节;私有控制 shell 隔离 profile,后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组,并在结算前证明完全停稳。
保留沙箱只会保留远程文件与非托管状态。重新连接不会重建宿主进程或终端句柄、协议连接、待处理调用、输出游标、计时器或锁。该 POC 不会新增会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄,也不会在其中运行整个 harness。
## 验证
聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布/回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合,其中包括 FS/Bash 可见性、重命名后的版本重读与带保护编辑、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 文档上限、代码运行时绑定/上限/清理、宿主工作区隔离,以及最终沙箱删除。
## 考虑过的替代方案
**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。
**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳,因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题:删除这些适配器不应使领域行为散落到远程提供方中;通用消费方本已负责这些行为。
**为每项能力或工具创建独立沙箱。** 不予采纳,因为文件与进程操作将无法共享身份或状态,从而破坏编码用例,并增加生命周期所有者的数量。
**把终端建模为普通的管道子进程。** 不予采纳,因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小,也更能如实表达契约。
@@ -35,12 +57,22 @@ Status: implemented
**在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳,因为只有 LSP 需要完整文档字节上限,而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制,远程提供方甚至需要辅助协议,却没有已观察到的并发替换缺陷。
**在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop(智能体循环)。
**删除 worker 线程代码运行时。** 不予采纳,因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化,而 worker 后端两者都不需要,并且仍是受支持的单进程路径。
**在远程环境中运行整个 harness。** 不予采纳,因为这是另一种部署模型。让执行能力可移植,并不意味着移动模型调用、会话状态、插件状态或 agent loop。
**把所有提供方操作都放进一个共享所有者包。** 不予采纳,因为沙箱身份与生命周期是所有者唯一的关注点。文件系统与进程管理保留各自独立的契约、测试和消费方,同时避免把所有者变成无边界的能力集合。
**只通过 shell 命令实现远程文件系统操作。** 不予采纳,因为这会丢弃现有文件工具已消费的结构化文件系统身份、错误、流式输出、版本保护和原子变更语义。
**新增通用分布式运行时抽象,或重新连接活跃句柄。** 不予采纳,因为现有能力 seam 已承载经证实的契约,而仅凭远程身份无法重建回调、待处理 promise、权限、协议状态或输出游标。新增一层只会推测 POC 边界之外的持久化与同步问题。
## 后果
远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。
远程执行提供方只需实现共享沙箱所有者,以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上,因此对这些能力的修复仍与提供方无关。
基础接口更宽,一对文件系统/进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制;模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。
本地实现承接 `node-pty` 和平台进程检查,因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸:dispose(资源释放)会在终止顶层 shell 前后清理后代进程,等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程,并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话,因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制,而不是把进程机制移回 PTY 消费方的理由。
E2B 组合证明,共享沙箱所有者加上文件系统与进程管理适配器,就足以在保持上层能力与提供方无关的同时,把可变编码世界移出宿主。其 POC 限制仍明确在案:SDK 会把完整命令传输内容保留在宿主内存中;远程启动无法同步发布 PID;无法获得精确的终端 stdin 等待状态与独立信号事实;基于数值 PID/PGID 的操作没有身份围栏;初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret;保留的产物会累积;也不会恢复逃逸进程或重连状态。这些是提供方限制,不是引入兼容性 shim 或更多 E2B 包的理由。