fix(fs): observe absence before guarded recreation

This commit is contained in:
Tianyi Cui
2026-08-09 15:22:50 +08:00
parent 9aa2d07353
commit ceba53edd7
68 changed files with 694 additions and 249 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md
README.md: b346b4c5bc7bc2888ccd3dc210bb90e0f1cc5965
README.zh.md: b84b2382c03dc6e422228b19a58aff927e8a5db7
README.md: 039ad72a9651a1e8907c2c63b8c83d113aedbb4e
README.zh.md: be97bbd92edb528f9560518398a0cb70f36b992b

View File

@@ -9,8 +9,8 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide
- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; GNU `realpath -mz` supplies canonical target identity without requiring the final file to exist, and ASCII/base64 plus strict NUL framing preserves newline and multibyte paths across the decoded SDK transport. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam; listings reuse returned metadata and resolve symbolic-link entries sequentially. Versions are opaque hashes of E2B metadata plus a per-write extended attribute.
- **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules.
- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing.
- **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, preserve an existing file's POSIX mode, and publish the staged file through E2B's same-filesystem atomic rename. The rename response supplies the committed version, so no fallible metadata request follows the commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the filesystem seam's observed-state semantics.
- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before rename. The signal is not forwarded into the rename RPC, so cancellation cannot interrupt the atomic commit; a successful rename is the commit point.
- **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, and preserve an existing file's POSIX mode. Replacements publish through E2B's same-filesystem atomic rename. A guarded `createIfAbsent` publishes with remote `ln` instead, making the commit atomically no-replace; metadata read from the staged file before that commit is projected to the target path for the returned version, so no fallible metadata request follows either commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process.
- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before publication. The signal is not forwarded into the rename or guarded-link commit, so cancellation cannot interrupt atomic publication or turn a committed write into a reported failure.
The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only.
@@ -25,7 +25,7 @@ No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, or external process populates it; local files are neither uploaded nor reflected back.
- **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B.
- **Mutation coordination is host-process-local** — `createIfAbsent` preserves a remote creator racing publication, but another harness connection or command can still race replacement; version guards detect only metadata changes represented by E2B.
- **Reads reopen canonical targets by path** — a concurrent remote path replacement between resolution and stream opening is not fenced by a stable file handle; no observed product defect justifies a provider-specific bounded-read protocol in this POC.
- **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency.
- **The POC targets E2B's default Linux image** — it relies on GNU `realpath`/`base64`/`chmod`, same-filesystem rename, streaming reads, and metadata extended attributes; custom templates are outside this POC.

View File

@@ -9,8 +9,8 @@
- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析GNU `realpath -mz` 提供规范化目标身份且不要求最终文件存在ASCII/base64 加严格 NUL 分帧会在已解码的 SDK 传输中保留含换行符和多字节字符的路径。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam目录列表会复用已返回的元数据并依次解析符号链接条目。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。
- **执行世界路径**:规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI以及由提供方负责的包含关系检查因此通用进程管理消费方无需解析 E2B 目标 ID也不会套用宿主路径规则。
- **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。
- **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`,保留现有文件的 POSIX mode,并通过 E2B 的同一文件系统原子重命名发布暂存文件。重命名响应会提供已提交的版本因此提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF存储时恢复占主导的 CRLF并在宿主进程内按规范化目标串行执行变更。可选的创建/版本防护会保留文件系统 seam 的已观察状态语义。
- **失败与取消**E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义,并在 rename 前立即检查。信号不会传入 rename RPC,因此取消无法中断原子提交;成功 rename 是提交点
- **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`保留现有文件的 POSIX mode。替换操作通过 E2B 的同一文件系统原子重命名发布。带防护的 `createIfAbsent` 改用远程 `ln` 发布,使提交具备原子且不替换的语义;系统会把提交前从暂存文件读取的元数据投影到目标路径,以生成返回的版本,因此任何一类提交点之后不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF存储时恢复占主导的 CRLF并在宿主进程内按规范化目标串行执行变更。
- **失败与取消**E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义,并在发布前立即检查。信号不会传入 rename 或防护链接提交,因此取消无法中断原子发布,也不会把已提交的写入报告为失败
该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。
@@ -25,7 +25,7 @@
## 已知限制与延后工作
- **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令或外部进程填充它;本地文件既不会上传,也不会同步回本地。
- **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。
- **变更协调仅限宿主进程内**`createIfAbsent` 会保留与发布发生竞态的远程创建者所写入的文件,但另一个 harness 连接或命令可能与替换操作发生竞态;版本防护只能检测 E2B 元数据所体现的变更。
- **读取会按路径重新打开规范化目标**:在解析与打开流之间若并发替换远程路径,该操作没有稳定文件句柄提供围栏;在该 POC 中,没有已观察到的产品缺陷能够证明提供方专用的有界读取协议值得引入。
- **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。
- **该 POC 面向 E2B 默认 Linux 镜像**:它依赖 GNU `realpath``base64``chmod`、同一文件系统内的 rename、流式读取和元数据扩展属性自定义模板不在该 POC 范围内。

View File

@@ -332,7 +332,13 @@ export class E2BFileSystem extends FileSystem {
}
this.checkWriteIntent(existing, expected, target)
const before = existing === undefined ? null : await this.readForDiff(target, signal)
const version = await this.writeAtomic(target, content, existing, signal)
const version = await this.writeAtomic(
target,
content,
existing,
expected?.kind === 'createIfAbsent',
signal,
)
return {
operation: existing === undefined ? 'create' : 'update',
version,
@@ -363,7 +369,7 @@ export class E2BFileSystem extends FileSystem {
const before = normalizeLineEndings(raw)
const after = literalEdit(before, edit, target.displayPath)
const storage = restoreLineEndings(after, detectsCrlf(raw))
const version = await this.writeAtomic(target, storage, existing, signal)
const version = await this.writeAtomic(target, storage, existing, false, signal)
return { version, before, after }
})
}
@@ -450,6 +456,7 @@ export class E2BFileSystem extends FileSystem {
target: FsTarget,
content: string,
existing: EntryInfo | undefined,
createIfAbsent: boolean,
signal?: AbortSignal,
): Promise<ReturnType<typeof FsVersion>> {
assertNotAborted(signal, 'write')
@@ -476,7 +483,28 @@ export class E2BFileSystem extends FileSystem {
commandOpts(signal),
)
assertNotAborted(signal, 'write')
const committed = await sandbox.files.rename(temporary, targetPath)
let committed: EntryInfo
if (createIfAbsent) {
const staged = await sandbox.files.getInfo(temporary, signalOpts(signal))
assertNotAborted(signal, 'write')
const targetArg = quoteE2BShellArg(targetPath)
const publication = await sandbox.commands.run(
`if ln -- ${quoteE2BShellArg(temporary)} ${targetArg}; then printf created; elif test -e ${targetArg} || test -L ${targetArg}; then printf exists; else exit 1; fi`,
commandOpts(undefined),
)
if (publication.stdout === 'exists') {
throw new FsError(
`cannot overwrite existing "${target.displayPath}" without reading it first`,
'FS_NOT_OBSERVED',
)
}
if (publication.stdout !== 'created') {
throw new Error('guarded create returned an invalid publication result')
}
committed = { ...staged, name: posix.basename(targetPath), path: targetPath }
} else {
committed = await sandbox.files.rename(temporary, targetPath)
}
try {
await sandbox.files.remove(stagingDirectory)
} catch (_committedStagingCleanupFailure) {

View File

@@ -37,6 +37,7 @@ class FakeRemote {
readonly writes: Array<{ path: string; data: string; metadata?: Record<string, string> }> = []
readonly writeParentModes: number[] = []
readonly renames: Array<{ from: string; to: string }> = []
readonly links: Array<{ from: string; to: string }> = []
readonly removals: string[] = []
readonly commands: string[] = []
streamChunks: Uint8Array[] | undefined
@@ -51,6 +52,8 @@ class FakeRemote {
nextRemoveError: unknown
canonicalOutput: string | undefined
abortAfterRename: AbortController | undefined
competitorBeforeLink: { path: string; data: string } | undefined
guardedLinkOutput: string | undefined
disappearOnInfo = new Set<string>()
private clock = 1
@@ -253,6 +256,28 @@ class FakeRemote {
}
const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command)
if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8)
const guardedLink = new RegExp(
"^if ln -- '([^']+)' '([^']+)'; then printf created; "
+ "elif test -e '[^']+' \\|\\| test -L '[^']+'; then printf exists; else exit 1; fi$",
).exec(command)
if (guardedLink !== null) {
const from = guardedLink[1]!
const to = guardedLink[2]!
if (this.guardedLinkOutput !== undefined) {
const stdout = this.guardedLinkOutput
this.guardedLinkOutput = undefined
return { exitCode: 0, stdout, stderr: '' }
}
if (this.competitorBeforeLink?.path === to) {
this.file(to, this.competitorBeforeLink.data)
this.competitorBeforeLink = undefined
}
if (this.nodes.has(to)) return { exitCode: 0, stdout: 'exists', stderr: '' }
this.nodes.set(to, this.required(from))
this.links.push({ from, to })
this.abortAfterRename?.abort('after commit')
return { exitCode: 0, stdout: 'created', stderr: '' }
}
const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command)
if (move !== null) {
if (this.nextRenameError !== undefined) {
@@ -475,6 +500,7 @@ describe('E2BFileSystem atomic writes and edits', () => {
expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600)
expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined()
expect(remote.writeParentModes).toEqual([0o700])
expect(remote.links).toHaveLength(1)
const stagingDirectory = posix.dirname(remote.writes[0]!.path)
expect(posix.dirname(stagingDirectory)).toBe('/workspace')
expect(remote.removals).toContain(stagingDirectory)
@@ -527,6 +553,33 @@ describe('E2BFileSystem atomic writes and edits', () => {
await expectCode(fs.writeText(await fs.resolve('dir'), 'x'), 'FS_NOT_REGULAR_FILE')
})
it('preserves a competitor created after the guarded-create probe', async () => {
const remote = new FakeRemote()
remote.competitorBeforeLink = { path: '/workspace/race.txt', data: 'competitor' }
const { fs } = await setup(remote)
await expectCode(
fs.writeText(await fs.resolve('race.txt'), 'ours', { kind: 'createIfAbsent' }),
'FS_NOT_OBSERVED',
)
expect(new TextDecoder().decode(remote.nodes.get('/workspace/race.txt')?.data)).toBe('competitor')
expect(remote.links).toHaveLength(0)
expect(remote.removals).toHaveLength(1)
})
it('rejects an invalid guarded-create publication response before claiming success', async () => {
const remote = new FakeRemote()
remote.guardedLinkOutput = 'unexpected'
const { fs } = await setup(remote)
await expectCode(
fs.writeText(await fs.resolve('invalid.txt'), 'ours', { kind: 'createIfAbsent' }),
'FS_IO_ERROR',
)
expect(remote.nodes.has('/workspace/invalid.txt')).toBe(false)
expect(remote.removals).toHaveLength(1)
})
it('does not turn an abort observed after a successful move into a failed write', async () => {
const remote = new FakeRemote()
const controller = new AbortController()
@@ -537,6 +590,20 @@ describe('E2BFileSystem atomic writes and edits', () => {
expect(controller.signal.aborted).toBe(true)
})
it('does not turn an abort observed after a guarded create into a failed write', async () => {
const remote = new FakeRemote()
const controller = new AbortController()
remote.abortAfterRename = controller
const { fs } = await setup(remote)
await expect(fs.writeText(
await fs.resolve('committed-create'),
'yes',
{ kind: 'createIfAbsent' },
controller.signal,
)).resolves.toMatchObject({ operation: 'create' })
expect(controller.signal.aborted).toBe(true)
})
it('does not turn post-commit staging cleanup failure into a failed write', async () => {
const remote = new FakeRemote()
remote.nextRemoveError = new Error('empty staging cleanup failed')