refactor(e2b): keep provider layer substrate-only
This commit is contained in:
Tianyi Cui
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-28-portable-execution-world-consumers.md
|
||||
2026-07-28-portable-execution-world-consumers.md: 402aa580255a5bd6aa0d046e3bcc16f712da520d
|
||||
2026-07-28-portable-execution-world-consumers.zh.md: 26d493a79f8adb9e729ff69b0673f2f2775868b0
|
||||
2026-07-28-portable-execution-world-consumers.md: 250445cfed6a4fe2dd188562813d586df29992c1
|
||||
2026-07-28-portable-execution-world-consumers.zh.md: 361c4de4e813aa7f57a7eb5acc75087b2630db43
|
||||
|
||||
@@ -8,6 +8,8 @@ English | [中文](2026-07-28-portable-execution-world-consumers.zh.md)
|
||||
|
||||
The filesystem and subprocess seams made file and ordinary process access replaceable, but PTY and LSP still reached host Node APIs directly. A remote execution provider therefore appeared to need separate PTY and LSP packages even though their domain behavior did not change. Those packages would be shallow adapters: each would duplicate an existing consumer merely to replace its file and process operations.
|
||||
|
||||
A remote coding world is useful only when file operations, commands, terminals, language servers, and model-written programs share one sandbox identity. Moving the complete harness into that sandbox would also entangle provider experimentation with plugin loading, credentials, model transport, session durability, supervision, and deployment.
|
||||
|
||||
Ordinary pipes do not cover one requirement. A persistent terminal needs PTY allocation, foreground-process-group inspection and signalling, and cleanup of the complete terminal session. Pretending those operations can be rebuilt in `dsh-pty-local` from an ordinary `spawn()` handle would either leak provider internals or weaken its lifecycle contract.
|
||||
|
||||
## Decision
|
||||
@@ -24,10 +26,26 @@ Generic consumers use that execution world:
|
||||
- `dsh-lsp-local` reads and contains source through `ctx.fs`, resolves and launches language servers through `ctx.subprocess`, and carries provider-owned file URIs through initialization and result rendering. One provider-lifetime signal aborts filesystem and protocol work during disposal, including workspace lookup before queue ownership; its JSON-RPC, pooling, synchronization, and normalization stay unchanged.
|
||||
- `dsh-pty-local` maps persistent-shell semantics onto `ctx.subprocess.spawnTerminal()`. The local `node-pty` and process-inspection implementation moves into `dsh-subprocess-local`; another subprocess provider supplies the same primitive. `danger-full-access` needs no `ctx.sandbox`; a confined mode requires a same-world sandbox provider and fails before spawn when none is mounted. Prompt and silence evidence collected during asynchronous pre-write inspection is discarded when the provider write begins. Cancellation retains the send reservation while an in-flight write settles and then signals the foreground group, so late bytes or the signal cannot target a successor; an in-flight readiness poll cannot release that reservation, and a rejected write sends no signal. The absolute deadline remains armed throughout cancellation. A signal failure becomes terminal transport failure. Completion of a stale inspection resumes polling for the current send. Startup cancellation begins terminal rollback without waiting for a stalled readiness or signalling call. Close rejects new public signals and delegates provider-observable session quiescence to the handle's awaited termination operation.
|
||||
|
||||
## E2B POC boundary
|
||||
|
||||
The opt-in E2B realization has exactly three provider-specific packages under `packages/e2b/`: `dsh-e2b` creates or reconnects one sandbox and owns kill/pause/leave disposal, `dsh-fs-e2b` implements `ctx.fs`, and `dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, PTYs, and remote Linux process groups. The two adapters obtain the sole sandbox identity from the owner and never create private sandboxes.
|
||||
|
||||
E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session groups, language-server processes and source reads, subprocess Code Runtime processes, and adapter-private files under `.dsh-e2b`. The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority, skills, subagent orchestration, PTY buffers and readiness, LSP protocol state, Code Runtime program/binding/output policy, and E2B SDK/network buffers. The overlay neither uploads nor synchronizes the host workspace.
|
||||
|
||||
The adapters retain only substrate mechanics. Filesystem canonicalization crosses the SDK's decoded command transport as strict base64-encoded NUL framing; streamed reads leave byte ceilings with consumers. Subprocess command output and environment snapshots use ASCII/base64 where SDK chunk decoding would otherwise lose bytes, while private control shells isolate profiles and later launches blank discovered credential-shaped names. Process and terminal cleanup uses remote groups and proves quiescence before settlement.
|
||||
|
||||
Retaining a sandbox preserves remote files and unmanaged state only. Reconnect does not reconstruct host process or terminal handles, protocol connections, pending calls, output cursors, timers, or locks. The POC adds no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution.
|
||||
|
||||
## Verification
|
||||
|
||||
Focused package suites pin sandbox lifecycle, canonical path framing, filesystem metadata and atomic versions, subprocess publication/rollback, terminal text I/O and session cleanup, output limits, cancellation, disposal, and invariant registration. A credential-gated Loader composition exercises the same three-package provider through source imports and built exports, including FS/Bash visibility, post-rename version reread plus guarded edit, hostile login profiles, byte-split UTF-8 output, process and terminal cleanup, LSP document bounds, Code Runtime bindings/limits/cleanup, host-workspace isolation, and final sandbox deletion.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep one PTY and LSP package per remote provider.** Rejected because provider mechanics would be repeated above the existing seams. The deletion test exposes the problem: deleting those adapters should not scatter domain behavior into the remote provider; the generic consumers already own it.
|
||||
|
||||
**Create a separate sandbox per capability or tool.** Rejected because file and process operations would not share identity or state, defeating the coding use case and multiplying lifecycle owners.
|
||||
|
||||
**Model a terminal as an ordinary piped subprocess.** Rejected because pipes cannot allocate a controlling terminal, resolve the current foreground process group, or prove complete terminal-session cleanup. One terminal primitive is smaller and more honest than exposing substrate-specific escape hatches.
|
||||
|
||||
**Move PTY readiness and session policy into the subprocess service.** Rejected because those are persistent-terminal consumer semantics, not OS process mechanics. A subprocess provider owns what only its substrate can do; `dsh-pty-local` owns what a Harness terminal means.
|
||||
@@ -38,6 +56,12 @@ Generic consumers use that execution world:
|
||||
|
||||
**Run the whole harness inside the remote environment.** Rejected as a different deployment model. Making execution capabilities portable does not move model calls, session state, plugin state, or the agent loop.
|
||||
|
||||
**Put every provider operation in one shared owner package.** Rejected because sandbox identity and lifecycle are the owner's only concerns. Filesystem and subprocess retain distinct contracts, tests, and consumers without turning the owner into a capability grab bag.
|
||||
|
||||
**Implement remote filesystem operations only through shell commands.** Rejected because that discards structured filesystem identity, errors, streaming, version guards, and atomic mutation semantics already consumed by the file tools.
|
||||
|
||||
**Add a generic distributed-runtime abstraction or reconnect live handles.** Rejected because the existing capability seams carry the demonstrated contracts, while remote identity alone cannot reconstruct callbacks, pending promises, authority, protocol state, or output cursors. A new layer would speculate about persistence and synchronization beyond the POC.
|
||||
|
||||
## Consequences
|
||||
|
||||
A remote execution provider implements only its shared sandbox owner plus filesystem and subprocess adapters. Bash, PTY, and LSP compose above them, so fixes to those capabilities remain provider-neutral.
|
||||
@@ -45,3 +69,5 @@ A remote execution provider implements only its shared sandbox owner plus filesy
|
||||
The fundamental interfaces are wider, and a filesystem/subprocess pair must agree on one execution world. The added operations are limited to facts and lifecycle mechanics that current generic consumers require; model schemas, protocol framing, readiness policy, and presentation do not leak into the providers.
|
||||
|
||||
The local implementation absorbs `node-pty` and platform process inspection because it owns local terminal mechanics. This moves code without weakening terminal teardown: disposal sweeps descendants before and after terminating the top-level shell, waits for exact PID-identity-fenced descendants retained during foreground inspection, and retains Linux session members that survive top-level exit. macOS cannot enumerate a POSIX session after its leader exits, so a child that reparents between inspection snapshots remains an explicit local-provider limitation rather than a reason to move process mechanics back into the PTY consumer.
|
||||
|
||||
The E2B composition demonstrates that a shared sandbox owner plus filesystem and subprocess adapters are sufficient to move the mutable coding world off-host while leaving higher capabilities provider-neutral. Its POC limits remain explicit: the SDK retains complete command transport in host memory, remote startup cannot publish a PID synchronously, exact terminal stdin-wait and independent signal facts are unavailable, numeric PID/PGID operations are not identity-fenced, the initial environment probe cannot hide unknown sandbox-default secrets from already-running same-UID processes, retained artifacts accumulate, and escaped processes or reconnect state are not recovered. These are provider constraints, not justification for compatibility shims or more E2B packages.
|
||||
|
||||
@@ -6,7 +6,9 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性，但 PTY 和 LSP 仍直接调用宿主 Node API。因此，即使领域行为没有变化，远程执行提供方看起来仍需要独立的 PTY 与 LSP 包（package）。这些包只会成为浅层适配器：每个包都仅为替换文件与进程操作而复制一个现有消费方。
|
||||
文件系统与进程管理 seam 使文件访问和普通进程访问具备可替换性，但若干上层能力仍直接调用宿主 Node API。因此，即使领域行为没有变化，远程执行提供方看起来仍需要独立的 PTY、LSP 与代码运行时包（package）。这些包只会成为浅层适配器：每个包都仅为替换文件与进程操作而复制一个现有消费方。
|
||||
|
||||
只有文件操作、命令、终端、语言服务器和模型编写的程序共享同一个沙箱身份时，远程编码世界才有用。若把完整 harness 移入该沙箱，还会把提供方实验与插件加载、凭据、模型传输、会话持久性、监督和部署纠缠在一起。
|
||||
|
||||
普通管道无法满足其中一项要求。持久终端需要分配 PTY、检查前台进程组并发送信号，以及清理完整的终端会话。如果假设可以在 `dsh-pty-local` 中基于普通 `spawn()` 句柄重建这些操作，最终不是泄漏提供方内部细节，就是削弱其生命周期契约。
|
||||
|
||||
@@ -16,16 +18,36 @@ Status: implemented
|
||||
|
||||
文件系统接口负责其他能力需要的路径事实，同时不公开其不透明目标身份：规范化进程路径、规范化 `file:` URI 和包含关系。现有完整文本与流式文本操作仍归文件系统负责；协议消费方在消费流时执行各自的保留上限。
|
||||
|
||||
进程管理接口负责可执行文件查找与进程原语：以原始或收集模式 spawn 普通进程，以及 `spawnTerminal()`。终端操作是一项深层原语，其句柄负责文本 I/O、前台进程组、信号发送，以及一项须等待的 TERM→KILL 操作；该操作会结算所有在途句柄调用，并使提供方仍可观察到的每个会话成员完全停稳。其信号只取消分配；句柄一经发布，便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。
|
||||
进程管理接口负责进程运行坐标与原语：规范化 cwd、私有运行时存储、可执行文件查找、以原始或收集模式 spawn 普通进程，以及 `spawnTerminal()`。终端操作是一项深层原语，其句柄负责文本 I/O、前台进程组、信号发送，以及一项须等待的 TERM→KILL 操作；该操作会结算所有在途句柄调用，并使整个会话完全停稳。其信号只取消分配；句柄一经发布，便负责自身生命周期。提示符检测、空闲推断、scrollback、沙箱策略和所有者生命周期仍由 PTY 消费方负责。
|
||||
|
||||
通用消费方使用该执行世界：
|
||||
|
||||
- `dsh-bash-local` 继续把 Bash 语义映射到普通的 `ctx.subprocess.spawn()`。
|
||||
- `dsh-lsp-local` 通过 `ctx.fs` 读取源文件并验证包含关系，通过 `ctx.subprocess` 解析和启动语言服务器，并让由提供方负责的文件 URI 贯穿初始化与结果渲染。一个提供方生命周期信号会在资源释放期间中止文件系统与协议操作，包括取得队列所有权之前的工作区查找；其 JSON-RPC、池化、同步和规范化保持不变。
|
||||
- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`；其他进程管理提供方则提供相同原语。`danger-full-access` 不需要 `ctx.sandbox`；受限模式要求同一执行世界中存在沙箱提供方，未挂载时会在 spawn 前失败。提供方开始写入时，系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留，随后向前台进程组发送信号，因此延迟字节和该信号都无法落到后续发送；在途就绪检查无法释放该预留，写入被拒绝时也不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后，会针对当前发送恢复轮询。启动取消会立即开始终端回滚，而不等待停滞的就绪检查或信号发送调用。关闭操作会拒绝新的公开信号，并把提供方可观察会话成员的完全停稳委托给句柄上须等待的终止操作。
|
||||
- `dsh-pty-local` 把持久 shell 语义映射到 `ctx.subprocess.spawnTerminal()`。本地 `node-pty` 与进程检查实现移入 `dsh-subprocess-local`；其他进程管理提供方则提供相同原语。提供方开始写入时，系统会丢弃异步写入前检查期间收集的提示符与静默证据。取消会在在途写入结算期间保留发送预留，随后向前台进程组发送信号，因此延迟字节和该信号都无法落到后续发送；写入被拒绝时不会发送信号。绝对截止时间会在整个取消期间保持启用。信号发送失败会成为终结性传输失败。陈旧检查完成后，会针对当前发送恢复轮询。关闭操作会拒绝新的公开信号，并把完整会话的完全停稳委托给句柄上须等待的终止操作。
|
||||
- `dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化无依赖 runner，并通过 `ctx.subprocess` 启动它，从而在本地或远程执行世界中保留代码运行时的绑定与输出契约。固定 runner 是位于 `ctx.subprocess.runtimeRoot` 下的适配器自有基础设施，因此其写入携带显式 `danger-full-access` 策略，而不继承面向模型的文件系统模式。它通过非插件子路径 `dsh-code-runtime-worker/runtime-host` 共享宿主侧 worker 机制，而不是复制这些机制。准备阶段让同一个生命周期信号贯穿文件系统解析、物化和可执行文件查找，使资源释放能够中止停滞的提供方操作。受堆上限约束的 worker 会在传输前拒绝过大的绑定帧；每个外层转发环节都会在转发前执行相同的上限检查；原始子进程管道承载以换行符分隔的 UTF-8 JSON，无需冗余的 base64 表示；launcher 会在回收 controller 前发布已接纳的终态帧，使继承 controller 管道的后代进程无法阻止完成；宿主仍会等待进程组完全停稳。
|
||||
|
||||
`dsh-code-runtime-worker` 仍是独立实现。它是较小的进程内后端，可用于无法假定已安装 Node 可执行文件的单文件分发。远程文件系统／进程组合选择 `dsh-code-runtime-subprocess`；它们不需要提供方专用的代码运行时包。
|
||||
|
||||
## E2B POC 边界
|
||||
|
||||
可选启用的 E2B 实现在 `packages/e2b/` 下恰好只有三个提供方专用包：`dsh-e2b` 创建或重新连接一个沙箱，并负责 kill、pause 或 leave 资源释放；`dsh-fs-e2b` 实现 `ctx.fs`；`dsh-subprocess-e2b` 基于 E2B Commands、PTY 和远程 Linux 进程组实现 `ctx.subprocess`。两个适配器都从所有者取得唯一的沙箱身份，绝不创建私有沙箱。
|
||||
|
||||
E2B 负责可变文件系统、受管命令与 Bash 进程、终端分配与终端会话组、语言服务器进程与源文件读取、子进程代码运行时进程，以及 `.dsh-e2b` 下的适配器私有文件。宿主负责 Cordis 与插件对象、agent loop（智能体循环）、agent（智能体）状态、会话状态与目标状态、会话日志与持久化、LLM（大语言模型）调用、提示词与工具、权限、skill（技能）、subagent 编排、PTY 缓冲区与就绪状态、LSP 协议状态、代码运行时程序／绑定／输出策略，以及 E2B SDK／网络缓冲区。该叠加层既不上传，也不同步宿主工作区。
|
||||
|
||||
适配器只保留执行基底机制。文件系统规范化以严格的 base64 加 NUL 分帧穿过 SDK 已解码的命令传输；流式读取把字节上限留给消费方执行。进程管理命令输出与环境快照采用 ASCII/base64，避免 SDK 分片解码丢失字节；私有控制 shell 隔离 profile，后续启动会把已发现且名称呈凭据特征的环境变量置空。进程与终端清理使用远程进程组，并在结算前证明完全停稳。
|
||||
|
||||
保留沙箱只会保留远程文件与非托管状态。重新连接不会重建宿主进程或终端句柄、协议连接、待处理调用、输出游标、计时器或锁。该 POC 不会新增会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄，也不会在其中运行整个 harness。
|
||||
|
||||
## 验证
|
||||
|
||||
聚焦的包测试套件锁定了沙箱生命周期、规范化路径分帧、文件系统元数据与原子版本、进程管理发布／回滚、终端文本 I/O 与会话清理、输出上限、取消、资源释放和不变式注册。一项受凭据门控的 Loader 组合通过源代码导入与构建后导出运行同一套三包提供方组合，其中包括 FS/Bash 可见性、重命名后的版本重读与带保护编辑、恶意登录 profile、跨字节边界拆分的 UTF-8 输出、进程与终端清理、LSP 文档上限、代码运行时绑定／上限／清理、宿主工作区隔离，以及最终沙箱删除。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**为每个远程提供方分别保留 PTY 与 LSP 包。** 不予采纳，因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题：删除这些适配器不应使领域行为散落到远程提供方中；通用消费方本已负责这些行为。
|
||||
**为每个远程提供方分别保留 PTY、LSP 和代码运行时包。** 不予采纳，因为这会在现有 seam 之上重复实现提供方机制。删除检验揭示了这一问题：删除这些适配器不应使领域行为散落到远程提供方中；通用消费方本已负责这些行为。
|
||||
|
||||
**为每项能力或工具创建独立沙箱。** 不予采纳，因为文件与进程操作将无法共享身份或状态，从而破坏编码用例，并增加生命周期所有者的数量。
|
||||
|
||||
**把终端建模为普通的管道子进程。** 不予采纳，因为管道无法分配控制终端、确定当前前台进程组或证明完整终端会话已清理。一项终端原语比公开特定于执行基底的逃生口更小，也更能如实表达契约。
|
||||
|
||||
@@ -35,12 +57,22 @@ Status: implemented
|
||||
|
||||
**在文件系统 seam 中新增稳定的有界读取原语。** 不予采纳，因为只有 LSP 需要完整文档字节上限，而它可以在消费现有文本流时执行该上限。第二项原语会迫使每个提供方实现稳定句柄和不跟随符号链接的机制，远程提供方甚至需要辅助协议，却没有已观察到的并发替换缺陷。
|
||||
|
||||
**在远程环境中运行整个 harness。** 不予采纳，因为这是另一种部署模型。让执行能力可移植，并不意味着移动模型调用、会话状态、插件状态或 agent loop（智能体循环）。
|
||||
**删除 worker 线程代码运行时。** 不予采纳，因为可移植性不会消除其当前部署需求。进程管理后端需要 Node 可执行文件和文件系统物化，而 worker 后端两者都不需要，并且仍是受支持的单进程路径。
|
||||
|
||||
**在远程环境中运行整个 harness。** 不予采纳，因为这是另一种部署模型。让执行能力可移植，并不意味着移动模型调用、会话状态、插件状态或 agent loop。
|
||||
|
||||
**把所有提供方操作都放进一个共享所有者包。** 不予采纳，因为沙箱身份与生命周期是所有者唯一的关注点。文件系统与进程管理保留各自独立的契约、测试和消费方，同时避免把所有者变成无边界的能力集合。
|
||||
|
||||
**只通过 shell 命令实现远程文件系统操作。** 不予采纳，因为这会丢弃现有文件工具已消费的结构化文件系统身份、错误、流式输出、版本保护和原子变更语义。
|
||||
|
||||
**新增通用分布式运行时抽象，或重新连接活跃句柄。** 不予采纳，因为现有能力 seam 已承载经证实的契约，而仅凭远程身份无法重建回调、待处理 promise、权限、协议状态或输出游标。新增一层只会推测 POC 边界之外的持久化与同步问题。
|
||||
|
||||
## 后果
|
||||
|
||||
远程执行提供方只需实现共享沙箱所有者，以及文件系统与进程管理适配器。Bash、PTY 与 LSP 组合在这些适配器之上，因此对这些能力的修复仍与提供方无关。
|
||||
远程执行提供方只需实现共享沙箱所有者，以及文件系统与进程管理适配器。Bash、PTY、LSP 和基于进程管理的代码运行时组合在这些适配器之上，因此对这些能力的修复仍与提供方无关。
|
||||
|
||||
基础接口更宽，一对文件系统／进程管理提供方必须在同一个执行世界上保持一致。新增操作仅限当前通用消费方所需的事实与生命周期机制；模型 schema、协议分帧、就绪策略和呈现不会渗入提供方。
|
||||
|
||||
本地实现承接 `node-pty` 和平台进程检查，因为它负责本地终端机制。这种代码迁移不会削弱终端拆卸：dispose（资源释放）会在终止顶层 shell 前后清理后代进程，等待前台检查期间保留下来且受精确 PID 身份围栏保护的后代进程，并继续追踪在顶层进程退出后仍存活的 Linux 会话成员。macOS 无法在 POSIX 会话 leader 退出后枚举该会话，因此在两次检查快照之间重新设定父进程的子进程仍是明确的本地提供方限制，而不是把进程机制移回 PTY 消费方的理由。
|
||||
|
||||
E2B 组合证明，共享沙箱所有者加上文件系统与进程管理适配器，就足以在保持上层能力与提供方无关的同时，把可变编码世界移出宿主。其 POC 限制仍明确在案：SDK 会把完整命令传输内容保留在宿主内存中；远程启动无法同步发布 PID；无法获得精确的终端 stdin 等待状态与独立信号事实；基于数值 PID/PGID 的操作没有身份围栏；初始环境探测无法向已在运行的同 UID 进程隐藏未知的沙箱默认 secret；保留的产物会累积；也不会恢复逃逸进程或重连状态。这些是提供方限制，不是引入兼容性 shim 或更多 E2B 包的理由。
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md
|
||||
2026-07-27-e2b-remote-runtime-poc.md: 79c5bfe2dfa11cd695c7134e6ceffb639b449688
|
||||
2026-07-27-e2b-remote-runtime-poc.zh.md: 729a10ec7efa4dd55f041dc804d2252102b13c01
|
||||
@@ -1,65 +0,0 @@
|
||||
# Agent Note: Shared E2B remote runtime POC
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
A remote coding-agent backend is useful only when filesystem operations, one-shot commands, persistent terminals, language servers, and model-written programs observe one coherent world. Attaching E2B independently at individual tools would let those capabilities address different sandboxes, while retaining host PTY, LSP, or worker backends would split state across machines even when the cwd strings match.
|
||||
|
||||
Moving the complete harness into a remote VM would unify that state but also couple provider experimentation to plugin loading, credentials, model transport, agent/session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns.
|
||||
|
||||
## Decision
|
||||
|
||||
The E2B integration is an opt-in provider-composition POC. Its three E2B-specific packages live together under `packages/e2b/`:
|
||||
|
||||
- `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal.
|
||||
- `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API.
|
||||
- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, byte PTYs, and remote Linux process groups.
|
||||
|
||||
The higher capabilities use provider-neutral implementations. `dsh-bash-local` delegates command mechanics to `ctx.subprocess`; `dsh-pty-local` delegates terminal allocation and signalling to `ctx.subprocess.spawnTerminal()`; `dsh-lsp-local` reads through `ctx.fs` and launches through `ctx.subprocess`; `dsh-code-runtime-subprocess` materializes its runner through `ctx.fs` and starts it through `ctx.subprocess`. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns those generic interfaces and consumers.
|
||||
|
||||
The E2B owner is the sole source of sandbox identity. Its two adapters never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces, generic implementations, model-facing tools, and agent loop.
|
||||
|
||||
## POC boundary
|
||||
|
||||
E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`.
|
||||
|
||||
The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories.
|
||||
|
||||
The fundamental adapters carry the substrate-specific mechanics. Adapter-internal E2B command and PTY login shells use a fresh randomized root-level `HOME`; `dsh-subprocess-e2b` also gives them empty overrides for scrubbed credential names before user profiles run. The subprocess adapter consumes E2B's byte PTY callback, transports environment snapshots and command bytes as ASCII/base64 across decoded SDK callbacks, and uses one cancellation controller plus one retryable group-cleanup transaction that reports success only after proving quiescence. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated newline-delimited UTF-8 JSON and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes.
|
||||
|
||||
Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox.
|
||||
|
||||
The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution.
|
||||
|
||||
## Verification
|
||||
|
||||
Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior.
|
||||
|
||||
A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, hostile command and PTY login-profile isolation, default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**A separate E2B sandbox per capability or tool** — rejected because file and command operations would not share identity or state, defeating the coding-agent use case and multiplying lifecycle ownership.
|
||||
|
||||
**Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams.
|
||||
|
||||
**Put every E2B operation in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem and subprocess retain separate provider contracts, tests, and consumers; the owner exposes one shared SDK handle without becoming a capability grab bag.
|
||||
|
||||
**Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools.
|
||||
|
||||
**Keep E2B-specific PTY, LSP, and Code Runtime packages** — rejected because their domain behavior does not vary with E2B. They were shallow adapters that duplicated existing consumers to replace filesystem and process operations; moving those operations behind the fundamental seams gives every provider one implementation of readiness, protocol, binding, and presentation behavior.
|
||||
|
||||
**Call E2B Filesystem, Commands, or PTY APIs directly from higher capabilities** — rejected because it bypasses the `ctx.fs` and `ctx.subprocess` contracts, duplicates execution-world policy in each consumer, and forks model-facing behavior. The subprocess seam includes the irreducible terminal primitive because ordinary pipes cannot supply foreground groups or whole-session cleanup.
|
||||
|
||||
**Add a generic distributed-runtime abstraction first** — rejected because the existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC.
|
||||
|
||||
**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not.
|
||||
|
||||
## Consequences
|
||||
|
||||
The three-package composition demonstrates that filesystem and subprocess are the sufficient provider seams for moving an agent's mutable coding world off-host without changing the loop, higher capability implementations, or model-facing tool packages. Fixes to Bash, PTY, LSP, and Code Runtime remain provider-neutral. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy.
|
||||
|
||||
The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains the complete base64 command transport in SDK memory, exact terminal stdin-wait inspection is unavailable, E2B supplies no independent signal fact, and reconnect cannot restore handles or protocol state. The adapter reports only its own requested TERM/KILL as signals and preserves every unrequested SDK exit as an exit code. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions.
|
||||
@@ -1,65 +0,0 @@
|
||||
# Agent Note: 共享 E2B 远程运行时 POC
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-27-e2b-remote-runtime-poc.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
远程 coding agent（智能体）后端只有在文件系统操作、一次性命令、持久终端、语言服务器和模型编写的程序观察到同一个一致环境时才有用。若在各工具上分别接入 E2B，这些功能可能访问不同的沙箱；即使 cwd 字符串相同，保留宿主 PTY、LSP 或 worker 后端也会让状态分散在不同机器上。
|
||||
|
||||
把完整 harness 迁入远程 VM 可以统一这些状态，但也会把提供方实验与插件加载、凭据、模型传输、agent／会话持久性、监管和部署耦合在一起。这个 POC 只需测试现有功能边界，不应把这些彼此独立的问题纳入范围。
|
||||
|
||||
## 决策
|
||||
|
||||
E2B 集成是一个选择性启用的提供方组合 POC。它的 3 个 E2B 专用包（package）集中位于 `packages/e2b/` 下：
|
||||
|
||||
- `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱，创建其工作目录与私有运行时目录，并拥有 kill/pause/leave 资源释放操作。
|
||||
- `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。
|
||||
- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands、字节 PTY 和远程 Linux 进程组之上实现 `ctx.subprocess`。
|
||||
|
||||
上层功能使用提供方无关的实现。`dsh-bash-local` 把命令机制委托给 `ctx.subprocess`；`dsh-pty-local` 把终端分配与信号发送委托给 `ctx.subprocess.spawnTerminal()`；`dsh-lsp-local` 通过 `ctx.fs` 读取，并通过 `ctx.subprocess` 启动；`dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化 runner，再通过 `ctx.subprocess` 启动它。这些通用接口与消费方由[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义。
|
||||
|
||||
E2B 所有者是沙箱身份的唯一真源。其两个适配器绝不创建私有沙箱，因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录，同时保留现有功能接口、通用实现、面向模型的工具与 agent loop（智能体循环）。
|
||||
|
||||
## POC 边界
|
||||
|
||||
E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker，以及 `.dsh-e2b` 下的适配器私有文件。
|
||||
|
||||
宿主拥有 Cordis 与插件对象、agent loop、agent／会话／goal 状态、会话日志及持久化、LLM（大语言模型）调用、提示词与工具、权限决策、skill（技能）、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id／队列／协议状态、Code Runtime 类型剥离／输出计量／绑定分发，以及 E2B SDK／网络编排。该 overlay 不会上传、挂载或同步宿主工作区；拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。
|
||||
|
||||
基础适配器承载基底专用机制。适配器内部的 E2B 命令与 PTY 登录 shell 使用位于根目录下、全新随机生成的 `HOME`；在用户 profile 脚本运行前，`dsh-subprocess-e2b` 还会为它们设置已清理凭据名称的空值覆盖。该子进程适配器消费 E2B 的字节 PTY 回调，以 ASCII/base64 跨越已解码的 SDK 回调传输环境快照与命令字节，并使用一个取消控制器与一个可重试的进程组清理事务，后者只有在证明完全停稳后才报告成功。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取，该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 以经过验证、由换行符分隔的 UTF-8 JSON 承载 controller/worker 协议，并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。
|
||||
|
||||
保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose（资源释放）时终止并等待退出，之后共享所有者才会暂停、脱离或终止沙箱。
|
||||
|
||||
本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄或完整 harness 执行。
|
||||
|
||||
## 验证
|
||||
|
||||
聚焦包测试套件固定所有者生命周期清理、文件系统路径／containment／有界描述符读取与提交元数据、子进程可执行文件查找／进程组／发布回滚、终端字节 I/O／信号身份／默认环境清理／会话清理、输出上限、中止顺序、等待完全停稳的资源释放，以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信，以及 worker／后代进程清理行为。
|
||||
|
||||
凭据门控的 Loader 组合会创建真实 E2B 沙箱，并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、恶意命令与 PTY 登录 profile 隔离、默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离，以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**每项功能或每个工具使用独立的 E2B 沙箱。** 不予采纳，因为文件操作和命令操作将无法共享身份或状态，既违背 coding agent 用例，也会增加生命周期所有者的数量。
|
||||
|
||||
**在 E2B 内运行完整 harness 进程。** 不予采纳，因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam，并不需要同时回答这些彼此独立的问题。
|
||||
|
||||
**把所有 E2B 操作放入共享所有者包。** 不予采纳，因为生命周期身份是该所有者唯一负责的事项。文件系统与进程管理各自保留独立的提供方契约、测试和消费方；所有者只公开一个共享 SDK 句柄，不会因此包揽各类功能。
|
||||
|
||||
**仅通过 shell 命令实现文件系统操作。** 不予采纳，因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。
|
||||
|
||||
**保留 E2B 专用的 PTY、LSP 与 Code Runtime 包。** 不予采纳，因为它们的领域行为不会随 E2B 改变。这些浅层适配器为了替换文件系统与进程操作而重复现有消费方；把这些操作移到基础 seam 之后，可让所有提供方共享同一套就绪判定、协议、绑定与呈现行为实现。
|
||||
|
||||
**从上层功能直接调用 E2B Filesystem、Commands 或 PTY API。** 不予采纳，因为这会绕过 `ctx.fs` 与 `ctx.subprocess` 契约，在每个消费方中重复执行环境策略，并使面向模型的行为产生分叉。进程管理 seam 纳入不可约简的终端原语，因为普通管道无法提供前台进程组或全会话清理。
|
||||
|
||||
**先添加通用分布式运行时抽象。** 不予采纳，因为现有功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。
|
||||
|
||||
**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳，因为只有远程身份，无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性，就会让陈旧的远程进程看似仍受管理，实际并非如此。
|
||||
|
||||
## 后果
|
||||
|
||||
这个由 3 个包组成的组合证明，文件系统与进程管理这两个提供方 seam 足以把 agent 的可变 coding 环境移出宿主，而无需改变循环、上层功能实现或面向模型的工具包。Bash、PTY、LSP 与 Code Runtime 的修复仍与提供方无关。`sandboxId` 与 `pause`／`leave` 允许实验手动保留远程文件，演示仍以 `kill` 作为清理策略。
|
||||
|
||||
这些适配器并不能对所有消费方与本地后端互换：远程启动无法同步公开 PID，E2B 会在 SDK 内存中保留完整的 base64 命令传输，无法精确检查终端 stdin 等待状态，E2B 不提供独立的信号事实，重新连接也无法恢复句柄或协议状态。适配器只会把自己请求的 TERM/KILL 报告为信号，其他未请求的 SDK 退出都保留为退出码。保留沙箱后会累积远程进程／spill 产物，模型程序与 Node worker 内部机制共享一个 JavaScript realm，有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录，而不会引入兼容垫片或新的跨领域抽象。
|
||||
Reference in New Issue
Block a user