refactor(e2b): keep provider layer substrate-only
This commit is contained in:
Tianyi Cui
@@ -1,6 +0,0 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-27-e2b-remote-runtime-poc.md
|
||||
2026-07-27-e2b-remote-runtime-poc.md: 79c5bfe2dfa11cd695c7134e6ceffb639b449688
|
||||
2026-07-27-e2b-remote-runtime-poc.zh.md: 729a10ec7efa4dd55f041dc804d2252102b13c01
|
||||
@@ -1,65 +0,0 @@
|
||||
# Agent Note: Shared E2B remote runtime POC
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-27-e2b-remote-runtime-poc.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
A remote coding-agent backend is useful only when filesystem operations, one-shot commands, persistent terminals, language servers, and model-written programs observe one coherent world. Attaching E2B independently at individual tools would let those capabilities address different sandboxes, while retaining host PTY, LSP, or worker backends would split state across machines even when the cwd strings match.
|
||||
|
||||
Moving the complete harness into a remote VM would unify that state but also couple provider experimentation to plugin loading, credentials, model transport, agent/session durability, supervision, and deployment. The POC needs to test the existing capability boundaries without taking on those independent concerns.
|
||||
|
||||
## Decision
|
||||
|
||||
The E2B integration is an opt-in provider-composition POC. Its three E2B-specific packages live together under `packages/e2b/`:
|
||||
|
||||
- `@deepseek-ai/dsh-e2b` creates or reconnects one secure E2B sandbox, creates its working and private runtime directories, and owns kill/pause/leave disposal.
|
||||
- `@deepseek-ai/dsh-fs-e2b` implements `ctx.fs` over that sandbox's Filesystem API.
|
||||
- `@deepseek-ai/dsh-subprocess-e2b` implements `ctx.subprocess` over E2B Commands, byte PTYs, and remote Linux process groups.
|
||||
|
||||
The higher capabilities use provider-neutral implementations. `dsh-bash-local` delegates command mechanics to `ctx.subprocess`; `dsh-pty-local` delegates terminal allocation and signalling to `ctx.subprocess.spawnTerminal()`; `dsh-lsp-local` reads through `ctx.fs` and launches through `ctx.subprocess`; `dsh-code-runtime-subprocess` materializes its runner through `ctx.fs` and starts it through `ctx.subprocess`. The [portable execution-world decision](../architecture/2026-07-28-portable-execution-world-consumers.md) owns those generic interfaces and consumers.
|
||||
|
||||
The E2B owner is the sole source of sandbox identity. Its two adapters never create private sandboxes, so filesystem tools, Bash, interactive shells, language servers, and code workers share one remote cwd, process namespace, and adapter-private directory while preserving the existing capability interfaces, generic implementations, model-facing tools, and agent loop.
|
||||
|
||||
## POC boundary
|
||||
|
||||
E2B owns the mutable filesystem, managed command and Bash processes, terminal allocation and terminal-session process groups, language-server processes and source reads, the Code Runtime launcher, controller, and worker, and adapter-private files under `.dsh-e2b`.
|
||||
|
||||
The host owns Cordis and plugin objects, the agent loop, agent/session/goal state, session logs and persistence, LLM calls, prompts and tools, authority decisions, skills, subagent orchestration, PTY buffers and readiness state, LSP JSON-RPC ids/queues/protocol state, Code Runtime type stripping/output accounting/binding dispatch, and E2B SDK/network orchestration. The overlay does not upload, mount, or synchronize the host workspace; identical cwd strings name independent host and remote directories.
|
||||
|
||||
The fundamental adapters carry the substrate-specific mechanics. Adapter-internal E2B command and PTY login shells use a fresh randomized root-level `HOME`; `dsh-subprocess-e2b` also gives them empty overrides for scrubbed credential names before user profiles run. The subprocess adapter consumes E2B's byte PTY callback, transports environment snapshots and command bytes as ASCII/base64 across decoded SDK callbacks, and uses one cancellation controller plus one retryable group-cleanup transaction that reports success only after proving quiescence. `dsh-fs-e2b` performs bounded source reads through a dependency-free helper that walks no-follow directory descriptors beneath the canonical target. Generic Code Runtime keeps its controller/worker protocol on validated newline-delimited UTF-8 JSON and kills the provider-owned process group before inherited pipes drain. Generic LSP uses byte-faithful UTF-8 JSON over command pipes.
|
||||
|
||||
Retaining a sandbox preserves remote files and unmanaged remote state only. Reconnect does not reconstruct host PTY sessions, buffers, process handles, LSP connections or requests, code workers, binding calls, timers, output cursors, or locks. Managed groups terminate and join when their provider disposes before the shared owner pauses, leaves, or kills the sandbox.
|
||||
|
||||
The POC has no session-persistence backend, template builder, volume, snapshot, network-policy layer, sandbox catalog, workspace synchronization, durable remote handles, or whole-harness execution.
|
||||
|
||||
## Verification
|
||||
|
||||
Focused package suites pin owner lifecycle cleanup, filesystem paths/containment/bounded descriptor reads and commit metadata, subprocess executable lookup/process groups/publication rollback, terminal byte I/O/signal identity/default-environment scrubbing/session cleanup, output limits, abort ordering, disposal to quiescence, and package-owned invariant registrations. The generic PTY, LSP, and subprocess Code Runtime suites pin their provider-neutral readiness, cross-namespace `processId`, binding bridge, descriptor isolation, hostile traffic, and worker/descendant cleanup behavior.
|
||||
|
||||
A credential-gated Loader composition creates real E2B sandboxes and exercises FS-to-Bash and Bash-to-FS visibility, real-directory runtime-state setup, process-publication rollback, byte-split UTF-8 command output, bounded raw spill and inherited-output draining, hostile command and PTY login-profile isolation, default-secret scrubbing, stale-interrupt identity, and process-tree cleanup, parent-swap-safe bounded LSP source reads, Code Runtime host bindings, descriptor-isolated output accounting, descendant-held pipe cleanup, wall timeout, abort, runner cleanup, host-workspace isolation, and final sandbox deletion. The same composition runs through source imports and built package exports.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**A separate E2B sandbox per capability or tool** — rejected because file and command operations would not share identity or state, defeating the coding-agent use case and multiplying lifecycle ownership.
|
||||
|
||||
**Run the entire harness process inside E2B** — rejected because it changes deployment, credential flow, model transport, session durability, plugin loading, and supervision at once. Those questions are independent of proving the provider seams.
|
||||
|
||||
**Put every E2B operation in the shared owner package** — rejected because lifecycle identity is the owner's only concern. Filesystem and subprocess retain separate provider contracts, tests, and consumers; the owner exposes one shared SDK handle without becoming a capability grab bag.
|
||||
|
||||
**Implement filesystem operations through shell commands only** — rejected because that bypasses `ctx.fs` identity, structured errors, version guards, streaming reads, and atomic mutation semantics already consumed by the file tools.
|
||||
|
||||
**Keep E2B-specific PTY, LSP, and Code Runtime packages** — rejected because their domain behavior does not vary with E2B. They were shallow adapters that duplicated existing consumers to replace filesystem and process operations; moving those operations behind the fundamental seams gives every provider one implementation of readiness, protocol, binding, and presentation behavior.
|
||||
|
||||
**Call E2B Filesystem, Commands, or PTY APIs directly from higher capabilities** — rejected because it bypasses the `ctx.fs` and `ctx.subprocess` contracts, duplicates execution-world policy in each consumer, and forks model-facing behavior. The subprocess seam includes the irreducible terminal primitive because ordinary pipes cannot supply foreground groups or whole-session cleanup.
|
||||
|
||||
**Add a generic distributed-runtime abstraction first** — rejected because the existing capability seams already carry the required contracts. A new cross-cutting interface would speculate about persistence, synchronization, and reconnect semantics beyond the POC.
|
||||
|
||||
**Restore live capability handles after `sandboxId` reconnect** — rejected because remote identity alone cannot reconstruct host callbacks, pending promises, authority, protocol state, or output cursors. Claiming continuity would make stale remote processes appear managed when they are not.
|
||||
|
||||
## Consequences
|
||||
|
||||
The three-package composition demonstrates that filesystem and subprocess are the sufficient provider seams for moving an agent's mutable coding world off-host without changing the loop, higher capability implementations, or model-facing tool packages. Fixes to Bash, PTY, LSP, and Code Runtime remain provider-neutral. `sandboxId` plus pause/leave permits manual remote-file retention for experiments, while kill remains the demo's cleanup policy.
|
||||
|
||||
The adapters are not interchangeable with local backends for every consumer: remote startup cannot synchronously expose a PID, E2B retains the complete base64 command transport in SDK memory, exact terminal stdin-wait inspection is unavailable, E2B supplies no independent signal fact, and reconnect cannot restore handles or protocol state. The adapter reports only its own requested TERM/KILL as signals and preserves every unrequested SDK exit as an exit code. Remote process/spill artifacts accumulate in a retained sandbox, Code programs share a JavaScript realm with Node worker internals, and a process that deliberately escapes a managed process group or terminal session does not become reconnectable or owned. These gaps remain documented POC constraints rather than compatibility shims or new cross-cutting abstractions.
|
||||
@@ -1,65 +0,0 @@
|
||||
# Agent Note: 共享 E2B 远程运行时 POC
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-27-e2b-remote-runtime-poc.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
远程 coding agent（智能体）后端只有在文件系统操作、一次性命令、持久终端、语言服务器和模型编写的程序观察到同一个一致环境时才有用。若在各工具上分别接入 E2B，这些功能可能访问不同的沙箱；即使 cwd 字符串相同，保留宿主 PTY、LSP 或 worker 后端也会让状态分散在不同机器上。
|
||||
|
||||
把完整 harness 迁入远程 VM 可以统一这些状态，但也会把提供方实验与插件加载、凭据、模型传输、agent／会话持久性、监管和部署耦合在一起。这个 POC 只需测试现有功能边界，不应把这些彼此独立的问题纳入范围。
|
||||
|
||||
## 决策
|
||||
|
||||
E2B 集成是一个选择性启用的提供方组合 POC。它的 3 个 E2B 专用包（package）集中位于 `packages/e2b/` 下：
|
||||
|
||||
- `@deepseek-ai/dsh-e2b` 创建或重新连接一个安全的 E2B 沙箱，创建其工作目录与私有运行时目录，并拥有 kill/pause/leave 资源释放操作。
|
||||
- `@deepseek-ai/dsh-fs-e2b` 在该沙箱的 Filesystem API 之上实现 `ctx.fs`。
|
||||
- `@deepseek-ai/dsh-subprocess-e2b` 在 E2B Commands、字节 PTY 和远程 Linux 进程组之上实现 `ctx.subprocess`。
|
||||
|
||||
上层功能使用提供方无关的实现。`dsh-bash-local` 把命令机制委托给 `ctx.subprocess`；`dsh-pty-local` 把终端分配与信号发送委托给 `ctx.subprocess.spawnTerminal()`；`dsh-lsp-local` 通过 `ctx.fs` 读取，并通过 `ctx.subprocess` 启动；`dsh-code-runtime-subprocess` 通过 `ctx.fs` 物化 runner，再通过 `ctx.subprocess` 启动它。这些通用接口与消费方由[可移植执行环境决策](../architecture/2026-07-28-portable-execution-world-consumers.md)负责定义。
|
||||
|
||||
E2B 所有者是沙箱身份的唯一真源。其两个适配器绝不创建私有沙箱，因此文件系统工具、Bash、交互式 shell、语言服务器和代码 worker 会共享一个远程 cwd、进程命名空间和适配器私有目录，同时保留现有功能接口、通用实现、面向模型的工具与 agent loop（智能体循环）。
|
||||
|
||||
## POC 边界
|
||||
|
||||
E2B 拥有可变文件系统、受管命令与 Bash 进程、终端分配与终端会话进程组、语言服务器进程及源码读取、Code Runtime launcher、controller 和 worker，以及 `.dsh-e2b` 下的适配器私有文件。
|
||||
|
||||
宿主拥有 Cordis 与插件对象、agent loop、agent／会话／goal 状态、会话日志及持久化、LLM（大语言模型）调用、提示词与工具、权限决策、skill（技能）、subagent 编排、PTY 缓冲与就绪状态、LSP JSON-RPC id／队列／协议状态、Code Runtime 类型剥离／输出计量／绑定分发，以及 E2B SDK／网络编排。该 overlay 不会上传、挂载或同步宿主工作区；拼写相同的 cwd 字符串分别指向彼此独立的宿主与远程目录。
|
||||
|
||||
基础适配器承载基底专用机制。适配器内部的 E2B 命令与 PTY 登录 shell 使用位于根目录下、全新随机生成的 `HOME`；在用户 profile 脚本运行前，`dsh-subprocess-e2b` 还会为它们设置已清理凭据名称的空值覆盖。该子进程适配器消费 E2B 的字节 PTY 回调，以 ASCII/base64 跨越已解码的 SDK 回调传输环境快照与命令字节，并使用一个取消控制器与一个可重试的进程组清理事务，后者只有在证明完全停稳后才报告成功。`dsh-fs-e2b` 通过无依赖辅助程序执行有界源码读取，该程序会在规范化目标下逐级遍历不跟随符号链接的目录描述符。通用 Code Runtime 以经过验证、由换行符分隔的 UTF-8 JSON 承载 controller/worker 协议，并在继承的管道排空前终止提供方拥有的进程组。通用 LSP 通过命令管道使用字节保真的 UTF-8 JSON。
|
||||
|
||||
保留沙箱只会保存远程文件与未受管的远程状态。重新连接不会重建宿主 PTY 会话、缓冲、进程句柄、LSP 连接或请求、代码 worker、绑定调用、定时器、输出游标或锁。受管进程组会在所属提供方 dispose（资源释放）时终止并等待退出，之后共享所有者才会暂停、脱离或终止沙箱。
|
||||
|
||||
本 POC 没有会话持久化后端、模板构建器、卷、快照、网络策略层、沙箱目录、工作区同步、持久远程句柄或完整 harness 执行。
|
||||
|
||||
## 验证
|
||||
|
||||
聚焦包测试套件固定所有者生命周期清理、文件系统路径／containment／有界描述符读取与提交元数据、子进程可执行文件查找／进程组／发布回滚、终端字节 I/O／信号身份／默认环境清理／会话清理、输出上限、中止顺序、等待完全停稳的资源释放，以及包自有不变式注册。通用 PTY、LSP 与子进程 Code Runtime 测试套件固定其提供方无关的就绪判定、跨命名空间 `processId`、绑定桥接、描述符隔离、恶意通信，以及 worker／后代进程清理行为。
|
||||
|
||||
凭据门控的 Loader 组合会创建真实 E2B 沙箱，并演练 FS-to-Bash 与 Bash-to-FS 可见性、真实目录形式的运行时状态设置、进程发布回滚、按字节切分的 UTF-8 命令输出、有界原始 spill 与继承输出排空、恶意命令与 PTY 登录 profile 隔离、默认秘密清理、陈旧中断身份与进程树清理、可抵御父目录替换的有界 LSP 源码读取、Code Runtime 宿主绑定、描述符隔离的输出记账、后代进程所持管道的清理、墙钟超时、中止、runner 清理、宿主工作区隔离，以及最终删除沙箱。同一组合分别通过源代码导入与已构建包导出运行。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**每项功能或每个工具使用独立的 E2B 沙箱。** 不予采纳，因为文件操作和命令操作将无法共享身份或状态，既违背 coding agent 用例，也会增加生命周期所有者的数量。
|
||||
|
||||
**在 E2B 内运行完整 harness 进程。** 不予采纳，因为这会同时改变部署、凭据流、模型传输、会话持久性、插件加载和监管方式。要证明提供方 seam，并不需要同时回答这些彼此独立的问题。
|
||||
|
||||
**把所有 E2B 操作放入共享所有者包。** 不予采纳，因为生命周期身份是该所有者唯一负责的事项。文件系统与进程管理各自保留独立的提供方契约、测试和消费方；所有者只公开一个共享 SDK 句柄，不会因此包揽各类功能。
|
||||
|
||||
**仅通过 shell 命令实现文件系统操作。** 不予采纳，因为这会绕过文件工具已经使用的 `ctx.fs` 身份、结构化错误、版本防护、流式读取和原子变更语义。
|
||||
|
||||
**保留 E2B 专用的 PTY、LSP 与 Code Runtime 包。** 不予采纳，因为它们的领域行为不会随 E2B 改变。这些浅层适配器为了替换文件系统与进程操作而重复现有消费方；把这些操作移到基础 seam 之后，可让所有提供方共享同一套就绪判定、协议、绑定与呈现行为实现。
|
||||
|
||||
**从上层功能直接调用 E2B Filesystem、Commands 或 PTY API。** 不予采纳，因为这会绕过 `ctx.fs` 与 `ctx.subprocess` 契约，在每个消费方中重复执行环境策略，并使面向模型的行为产生分叉。进程管理 seam 纳入不可约简的终端原语，因为普通管道无法提供前台进程组或全会话清理。
|
||||
|
||||
**先添加通用分布式运行时抽象。** 不予采纳，因为现有功能 seam 已承载所需契约。新的跨领域接口会预先假定 POC 范围之外的持久化、同步与重连语义。
|
||||
|
||||
**在 `sandboxId` 重连后恢复活动功能句柄。** 不予采纳，因为只有远程身份，无法重建宿主回调、待处理 promise、权限、协议状态或输出游标。若声称保持连续性，就会让陈旧的远程进程看似仍受管理，实际并非如此。
|
||||
|
||||
## 后果
|
||||
|
||||
这个由 3 个包组成的组合证明，文件系统与进程管理这两个提供方 seam 足以把 agent 的可变 coding 环境移出宿主，而无需改变循环、上层功能实现或面向模型的工具包。Bash、PTY、LSP 与 Code Runtime 的修复仍与提供方无关。`sandboxId` 与 `pause`／`leave` 允许实验手动保留远程文件，演示仍以 `kill` 作为清理策略。
|
||||
|
||||
这些适配器并不能对所有消费方与本地后端互换：远程启动无法同步公开 PID，E2B 会在 SDK 内存中保留完整的 base64 命令传输，无法精确检查终端 stdin 等待状态，E2B 不提供独立的信号事实，重新连接也无法恢复句柄或协议状态。适配器只会把自己请求的 TERM/KILL 报告为信号，其他未请求的 SDK 退出都保留为退出码。保留沙箱后会累积远程进程／spill 产物，模型程序与 Node worker 内部机制共享一个 JavaScript realm，有意逃离受管理进程组或终端会话的进程也不会因此变得可重新连接或由该组合管理。这些缺口作为 POC 约束明确记录，而不会引入兼容垫片或新的跨领域抽象。
|
||||
Reference in New Issue
Block a user