fix(review): bound E2B readBytes at the seam, prove conditional-registration disposal, and align read_image contracts
- E2BFileSystem.readBytes now short-circuits on the stat size before any content transfer and streams the remote object, cancelling at the first chunk past the cap, honoring the seam's bounded-buffering contract; the fs-e2b README pair documents the new primitive. - The tool-fs HMR test now proves the attachments-scoped registration: disposing the store withdraws read_image while read/write/edit stay, remounting restores it, and disposing the plugin withdraws everything. - read_image caps reads at the smaller of maxImageBytes and maxMessageImageBytes, records an absent observation for a missing target like its sibling read, widens the mismatch remedy to cover out-of-family formats, and renames the gate's parameter to requestedPath; module/apply/registration JSDoc now match the shipped composition. zh terminology aligned; the examples manifest keeps its literal arrow.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md
|
||||
README.md: 1b66e84defb56cbfaa4a91d6ba6b48377fb52ca9
|
||||
README.zh.md: d9cd3ce1e109bf6b0b7fae02157d1ec6be51e575
|
||||
README.md: 9171989f968f144593107eb918fe75cd12de7768
|
||||
README.zh.md: 9f50bbe4c37bbbfb641690a690be45dbb5158258
|
||||
|
||||
@@ -9,6 +9,7 @@ E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provide
|
||||
- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; GNU `realpath -mz` supplies canonical target identity without requiring the final file to exist, and ASCII/base64 plus strict NUL framing preserves newline and multibyte paths across the decoded SDK transport. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam; listings reuse returned metadata and resolve symbolic-link entries sequentially. Versions are opaque hashes of E2B metadata plus a per-write extended attribute.
|
||||
- **Execution-world paths** — canonical targets expose absolute POSIX process paths, percent-encoded `file:` URIs, and provider-owned containment checks, so generic subprocess consumers never parse E2B target ids or apply host path rules.
|
||||
- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing.
|
||||
- **Bounded raw-byte reads** — `readBytes` short-circuits on the stat size before any content transfer, then streams the remote object and cancels the stream at the first chunk past `maxBytes` (`FS_TOO_LARGE`), so neither an at-rest oversized file nor a post-stat grower is buffered whole in host memory. The empty-file quirk of the pinned SDK (content-length 0 returns `''` in stream format) yields an empty result.
|
||||
- **Atomic mutations** — writes create a random sibling staging directory, change it to mode `0700` before uploading content, and preserve an existing file's POSIX mode. Replacements publish through E2B's same-filesystem atomic rename. A guarded `createIfAbsent` publishes with remote `ln -T` instead, making the commit atomically no-replace even when a directory appears at the destination; metadata read from the staged file before that commit is projected to the target path for the returned version, so no fallible metadata request follows either commit point. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process.
|
||||
- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at earlier SDK request boundaries and checked immediately before publication. The signal is not forwarded into the rename or guarded-link commit, so cancellation cannot interrupt atomic publication or turn a committed write into a reported failure.
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;GNU `realpath -mz` 提供规范化目标身份,且不要求最终文件存在;ASCII/base64 加严格 NUL 分帧会在已解码的 SDK 传输中保留含换行符和多字节字符的路径。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam;目录列表会复用已返回的元数据,并依次解析符号链接条目。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。
|
||||
- **执行世界路径**:规范化目标公开绝对 POSIX 进程路径、百分号编码的 `file:` URI,以及由提供方负责的包含关系检查,因此通用进程管理消费方无需解析 E2B 目标 ID,也不会套用宿主路径规则。
|
||||
- **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8,并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。
|
||||
- **有界原始字节读取**:`readBytes` 在任何内容传输之前先按 stat 大小短路,然后流式读取远程对象,并在第一个超过 `maxBytes` 的分片处取消流(`FS_TOO_LARGE`),因此静态超限文件和 stat 后增长的文件都不会被完整缓冲进宿主内存。所钉版本 SDK 的空文件怪癖(content-length 为 0 时 stream 格式返回 `''`)产生空结果。
|
||||
- **原子变更**:写入会创建随机的同级暂存目录,在上传内容前将其 mode 改为 `0700`,并保留现有文件的 POSIX mode。替换操作通过 E2B 的同一文件系统原子重命名发布。带防护的 `createIfAbsent` 改用远程 `ln -T` 发布,即使目标位置出现目录,也能使提交具备原子且不替换的语义;系统会把提交前从暂存文件读取的元数据投影到目标路径,以生成返回的版本,因此任何一类提交点之后都不会再进行可能失败的元数据请求。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF,存储时恢复占主导的 CRLF,并在宿主进程内按规范化目标串行执行变更。
|
||||
- **失败与取消**:E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在更早的 SDK 请求边界上采用尽力而为语义,并在发布前立即检查。信号不会传入 rename 或防护链接提交,因此取消无法中断原子发布,也不会把已提交的写入报告为失败。
|
||||
|
||||
|
||||
@@ -229,20 +229,59 @@ export class E2BFileSystem extends FileSystem {
|
||||
|
||||
override async readBytes(target: FsTarget, signal: AbortSignal | undefined, maxBytes: number): Promise<Uint8Array> {
|
||||
const sandbox = await this.ctx.e2b.getSandbox()
|
||||
await this.requireRegular(target, signal)
|
||||
let bytes: Uint8Array
|
||||
const info = await this.requireRegular(target, signal)
|
||||
if (info.size !== undefined && info.size > maxBytes) {
|
||||
throw new FsError(`cannot read "${target.displayPath}": ${info.size} bytes exceeds the ${maxBytes}-byte limit`, 'FS_TOO_LARGE')
|
||||
}
|
||||
let stream: ReadableStream<Uint8Array>
|
||||
try {
|
||||
// The E2B files API returns the whole object; the remote sandbox owns
|
||||
// that buffering, so the seam bound is enforced on the complete result.
|
||||
bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) })
|
||||
// Same pinned-SDK quirk as streamText: content-length 0 returns ''
|
||||
// instead of a ReadableStream.
|
||||
const read = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) }) as
|
||||
ReadableStream<Uint8Array> | string
|
||||
stream = typeof read === 'string'
|
||||
? new ReadableStream<Uint8Array>({ start(controller) { controller.close() } })
|
||||
: read
|
||||
} catch (error: unknown) {
|
||||
throw mapError(error, 'read', target.displayPath, signal)
|
||||
}
|
||||
assertNotAborted(signal, 'read')
|
||||
if (bytes.byteLength > maxBytes) {
|
||||
throw new FsError(`cannot read "${target.displayPath}": ${bytes.byteLength} bytes exceeds the ${maxBytes}-byte limit`, 'FS_TOO_LARGE')
|
||||
const reader = stream.getReader()
|
||||
const chunks: Uint8Array[] = []
|
||||
let bytes = 0
|
||||
let completed = false
|
||||
try {
|
||||
while (true) {
|
||||
assertNotAborted(signal, 'read')
|
||||
const next = await reader.read()
|
||||
if (next.done) break
|
||||
// The stat preflight covers the at-rest case; this streamed bound stops
|
||||
// a post-stat grower without transferring past the first overflowing chunk.
|
||||
bytes += next.value.byteLength
|
||||
if (bytes > maxBytes) {
|
||||
throw new FsError(`cannot read "${target.displayPath}": content exceeds the ${maxBytes}-byte limit`, 'FS_TOO_LARGE')
|
||||
}
|
||||
chunks.push(next.value)
|
||||
}
|
||||
completed = true
|
||||
} catch (error: unknown) {
|
||||
throw mapError(error, 'read', target.displayPath, signal)
|
||||
} finally {
|
||||
if (!completed) {
|
||||
try {
|
||||
await reader.cancel()
|
||||
} catch (_streamCancellationFailure) {
|
||||
// The read already failed; a cancellation failure on the abandoned
|
||||
// remote stream adds nothing actionable for the caller.
|
||||
}
|
||||
}
|
||||
}
|
||||
return bytes
|
||||
const whole = new Uint8Array(bytes)
|
||||
let offset = 0
|
||||
for (const chunk of chunks) {
|
||||
whole.set(chunk, offset)
|
||||
offset += chunk.byteLength
|
||||
}
|
||||
return whole
|
||||
}
|
||||
|
||||
override async streamText(target: FsTarget, signal?: AbortSignal): Promise<AsyncIterable<string>> {
|
||||
@@ -430,10 +469,11 @@ export class E2BFileSystem extends FileSystem {
|
||||
}
|
||||
}
|
||||
|
||||
private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise<void> {
|
||||
private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise<FsInfo> {
|
||||
const info = await this.stat(target, signal)
|
||||
if (info === undefined) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND')
|
||||
if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
return info
|
||||
}
|
||||
|
||||
private checkWriteIntent(existing: EntryInfo | undefined, expected: FsWriteIntent | undefined, target: FsTarget): void {
|
||||
|
||||
@@ -488,6 +488,22 @@ describe('E2BFileSystem identity, metadata, and reads', () => {
|
||||
await expectCode(fs.readBytes(target, undefined, 4), 'FS_ABORTED')
|
||||
})
|
||||
|
||||
it('readBytes bounds a post-stat grower mid-stream and reads an empty file through the SDK quirk', async () => {
|
||||
const remote = new FakeRemote()
|
||||
remote.file('/workspace/grow.bin', [1, 1, 1, 1])
|
||||
remote.file('/workspace/empty.bin', '')
|
||||
const { fs } = await setup(remote)
|
||||
|
||||
remote.streamChunks = [bytes([1, 1, 1]), bytes([1, 2, 2])]
|
||||
remote.streamKeepOpen = true
|
||||
await expectCode(fs.readBytes(await fs.resolve('grow.bin'), undefined, 4), 'FS_TOO_LARGE')
|
||||
expect(remote.streamCancel).toHaveBeenCalledOnce()
|
||||
|
||||
remote.streamChunks = undefined
|
||||
remote.streamKeepOpen = false
|
||||
expect((await fs.readBytes(await fs.resolve('empty.bin'), undefined, 4)).byteLength).toBe(0)
|
||||
})
|
||||
|
||||
it('honors aborts before and during remote reads', async () => {
|
||||
const remote = new FakeRemote()
|
||||
remote.file('/workspace/a', 'a')
|
||||
|
||||
Reference in New Issue
Block a user