refactor(e2b): group remote providers

This commit is contained in:
Tianyi Cui
2026-07-28 14:52:37 +08:00
parent 6667102890
commit e64d40837c
81 changed files with 171 additions and 249 deletions

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/e2b/fs-e2b/README.md
README.md: d8f3915fa281e8b9c062e3412c92bc6358616284
README.zh.md: 93d27bcdd514ccbf87e07d0eff5958edf3c726c1

View File

@@ -0,0 +1,29 @@
# @deepseek-ai/dsh-fs-e2b
English | [中文](README.zh.md)
E2B implementation of the [`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) provider seam. It has no config: load [`@deepseek-ai/dsh-e2b`](../e2b/README.md) first, then this service in place of `dsh-fs-local`. The provider uses the owner's remote cwd and SDK handle, so file tools observe the same world as E2B-backed Bash processes.
## Behavior
- **Remote identity and metadata** — relative paths resolve as POSIX paths against the caller cwd or `ctx.e2b.cwd`; `realpath -m` supplies canonical target identity without requiring the final file to exist. `stat`, no-follow `lstat`, and stable one-level directory listings project E2B metadata into the filesystem seam. Versions are opaque hashes of E2B metadata plus a per-write extended attribute.
- **UTF-8 reads** — whole reads and streamed reads preserve cross-chunk decoding, reject invalid UTF-8, and use the seam's 8192-byte NUL sample for binary detection. The model-facing tool still owns size selection and line windowing.
- **Atomic mutations** — writes upload a mode-`0600` temporary sibling, preserve an existing file's POSIX mode, and publish through same-directory Linux `mv -f`. E2B creates missing parent directories. Literal edits LF-normalize for matching, restore dominant CRLF storage, and serialize mutations per canonical target within the host process. Optional create/version guards keep the base seam's observed-state semantics.
- **Failures and cancellation** — E2B not-found, permission, abort, and other controller failures map to the existing `FsError` vocabulary. Cancellation is best-effort at SDK request boundaries; a successful rename is the commit point.
The provider does not copy, mount, or reconcile the host workspace. Giving it a host path as `cwd` creates a remote directory with the same spelling only.
## Model Experience
Indirectly, through [`dsh-tool-fs`](../../fs/tool-fs/README.md), which renders remote UTF-8 content, directory results, mutation acknowledgements, and provider errors while E2B identity and transport remain internal.
#### KV Cache effect
No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **No host synchronization** — an empty E2B cwd stays empty until a tool, command, template, or external process populates it; local files are neither uploaded nor reflected back.
- **Mutation coordination is host-process-local** — another harness connection or remote command can race the adapter; version guards detect only metadata changes represented by E2B.
- **Whole-file mutation costs remain** — overwrite diffs and literal edits read complete files into host memory, and every operation incurs E2B controller latency.
- **Custom templates must support the used Linux and envd features** — `realpath`, `chmod`, `mv`, same-filesystem POSIX rename, streaming reads, and file metadata extended attributes are required; unsupported templates fail rather than degrade silently.

View File

@@ -0,0 +1,29 @@
# @deepseek-ai/dsh-fs-e2b
[English](README.md) | 中文
[`@deepseek-ai/dsh-fs`](../../fs/fs/README.md) 提供方 seam 的 E2B 实现。它没有配置:先加载 [`@deepseek-ai/dsh-e2b`](../e2b/README.md),再用本服务取代 `dsh-fs-local`。该提供方使用所有者的远程 cwd 和 SDK 句柄,因此文件工具观察到的环境与 E2B 后端 Bash 进程相同。
## 行为
- **远程身份与元数据**:相对路径以调用方 cwd 或 `ctx.e2b.cwd` 为基准,按照 POSIX 路径解析;`realpath -m` 提供规范化目标身份,且不要求最终文件存在。`stat`、不跟随链接的 `lstat` 和稳定的单层目录列表会把 E2B 元数据投影到文件系统 seam。版本是 E2B 元数据与每次写入设置的扩展属性所组成的不透明哈希。
- **UTF-8 读取**:完整读取和流式读取会保留跨分片解码、拒绝无效 UTF-8并使用 seam 的 8192 字节 NUL 样本检测二进制内容。面向模型的工具仍负责选择大小和行窗口。
- **原子变更**:写入会上传 mode 为 `0600` 的同级临时文件,保留现有文件的 POSIX mode并通过同目录 Linux `mv -f` 发布。E2B 会创建缺失的父目录。字面量编辑匹配时会规范化为 LF存储时恢复占主导的 CRLF并在宿主进程内按规范化目标串行执行变更。可选的创建版本防护会保留基础 seam 的已观察状态语义。
- **失败与取消**E2B 的未找到、权限、中止及其他控制器故障会映射到现有 `FsError` 词汇。取消在 SDK 请求边界上采用尽力而为语义;成功 rename 是提交点。
该提供方不会复制、挂载或协调宿主工作区。把宿主路径用作 `cwd`,只会在远程创建一个拼写相同的目录。
## 模型体验
通过 [`dsh-tool-fs`](../../fs/tool-fs/README.md) 间接影响模型;该工具会渲染远程 UTF-8 内容、目录结果、变更确认和提供方错误,而 E2B 身份及传输保持内部实现。
#### KV Cache 影响
不会直接失效;请求前缀变更由具名消费方负责。
## 已知限制与延后工作
- **不提供宿主同步**:空的 E2B cwd 会一直为空,直到工具、命令、模板或外部进程填充它;本地文件既不会上传,也不会同步回本地。
- **变更协调仅限宿主进程内**:另一个 harness 连接或远程命令可能与适配器发生竞态;版本防护只能检测 E2B 元数据所体现的变更。
- **仍需承担完整文件变更成本**:覆盖差异和字面量编辑会把完整文件读入宿主内存,每项操作也都会产生 E2B 控制器延迟。
- **自定义模板必须支持所用的 Linux 与 envd 功能**:必须支持 `realpath``chmod``mv`、同一文件系统内的 POSIX rename、流式读取和文件元数据扩展属性不支持的模板会失败而不会静默降级。

View File

@@ -0,0 +1,41 @@
{
"name": "@deepseek-ai/dsh-fs-e2b",
"description": "E2B filesystem implementation for DeepSeek Harness",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./invariant": {
"types": "./lib/types/invariant.d.ts",
"default": "./lib/invariant.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/invariant.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-e2b": "^0.0.1",
"@deepseek-ai/dsh-fs": "^0.0.1",
"@deepseek-ai/dsh-invariants": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"@deepseek-ai/dsh-e2b": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,423 @@
/**
* E2B implementation of the filesystem provider seam. Paths, contents, and
* atomic staging files remain inside the shared remote sandbox.
* @module @deepseek-ai/dsh-fs-e2b
*/
import { createHash, randomUUID } from 'node:crypto'
import { posix } from 'node:path'
import { FileSystem, FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs'
import type {
FsDirEntry,
FsEditOutcome,
FsEditRequest,
FsInfo,
FsPathInfo,
FsTarget,
FsWriteIntent,
FsWriteOutcome,
} from '@deepseek-ai/dsh-fs'
import {
CommandExitError,
FileNotFoundError,
FileType,
quoteE2BShellArg,
} from '@deepseek-ai/dsh-e2b'
import type { EntryInfo, Sandbox } from '@deepseek-ai/dsh-e2b'
const VERSION_METADATA_KEY = 'dsh-version'
const BINARY_SAMPLE_BYTES = 8192
function assertNotAborted(signal: AbortSignal | undefined, operation: string): void {
if (signal?.aborted === true) throw new FsError(`${operation} aborted`, 'FS_ABORTED')
}
function normalizeLineEndings(value: string): string {
return value.replaceAll('\r\n', '\n')
}
function detectsCrlf(value: string): boolean {
const sample = value.slice(0, 4096)
const crlf = sample.split('\r\n').length - 1
const lf = sample.split('\n').length - 1 - crlf
return crlf > lf
}
function restoreLineEndings(value: string, crlf: boolean): string {
return crlf ? normalizeLineEndings(value).replaceAll('\n', '\r\n') : value
}
function decodeText(bytes: Uint8Array, displayPath: string, binarySampleBytes: number): string {
if (bytes.subarray(0, binarySampleBytes).includes(0)) {
throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT')
}
try {
return new TextDecoder('utf-8', { fatal: true }).decode(bytes)
} catch (error: unknown) {
throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error })
}
}
function signalOpts(signal: AbortSignal | undefined): { signal?: AbortSignal } {
return signal === undefined ? {} : { signal }
}
function entryType(entry: EntryInfo): FsInfo['type'] {
switch (entry.type) {
case FileType.FILE:
return 'file'
case FileType.DIR:
return 'directory'
default:
return 'other'
}
}
function entryVersion(entry: EntryInfo): ReturnType<typeof FsVersion> {
const facts = JSON.stringify([
entry.metadata?.[VERSION_METADATA_KEY],
entry.path,
entry.type,
entry.size,
entry.mode,
entry.modifiedTime?.toISOString(),
entry.symlinkTarget,
])
return FsVersion(`e2b:${createHash('sha256').update(facts).digest('hex')}`)
}
function mapError(error: unknown, operation: string, displayPath: string, signal?: AbortSignal): FsError {
if (error instanceof FsError) return error
if (signal?.aborted === true || (error instanceof DOMException && error.name === 'AbortError')) {
return new FsError(`${operation} aborted`, 'FS_ABORTED', { cause: error })
}
if (error instanceof FileNotFoundError) {
return new FsError(`cannot ${operation} "${displayPath}": not found`, 'FS_NOT_FOUND', { cause: error })
}
if (/permission denied|operation not permitted/i.test(String(error))) {
return new FsError(`cannot ${operation} "${displayPath}": permission denied`, 'FS_PERMISSION_DENIED', { cause: error })
}
return new FsError(`cannot ${operation} "${displayPath}": ${String(error)}`, 'FS_IO_ERROR', { cause: error })
}
function literalEdit(content: string, request: FsEditRequest, displayPath: string): string {
const oldString = normalizeLineEndings(request.oldString)
const newString = normalizeLineEndings(request.newString)
if (oldString.length === 0) {
throw new FsError(`cannot edit "${displayPath}": old_string must be non-empty`, 'FS_EDIT_NOT_FOUND')
}
let matches = 0
let offset = 0
while (true) {
const found = content.indexOf(oldString, offset)
if (found < 0) break
matches += 1
offset = found + oldString.length
}
if (matches === 0) throw new FsError(`cannot edit "${displayPath}": old_string was not found`, 'FS_EDIT_NOT_FOUND')
if (!request.replaceAll && matches !== 1) {
throw new FsError(`cannot edit "${displayPath}": old_string matched ${matches} times`, 'FS_AMBIGUOUS_EDIT')
}
return request.replaceAll ? content.split(oldString).join(newString) : content.replace(oldString, newString)
}
/** Remote filesystem backend sharing the sandbox owned by `ctx.e2b`. */
export class E2BFileSystem extends FileSystem {
static inject = ['e2b']
private readonly locks = new Map<string, Promise<unknown>>()
override async resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise<FsTarget> {
assertNotAborted(opts?.signal, 'resolve')
if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND')
const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path)
try {
const sandbox = await this.ctx.e2b.getSandbox()
const targetKey = await this.canonicalPath(sandbox, displayPath, opts?.signal)
assertNotAborted(opts?.signal, 'resolve')
return { targetKey: FsTargetKey(targetKey), displayPath }
} catch (error: unknown) {
throw mapError(error, 'resolve', displayPath, opts?.signal)
}
}
override async stat(target: FsTarget, signal?: AbortSignal): Promise<FsInfo | undefined> {
assertNotAborted(signal, 'stat')
const entry = await this.probe(String(target.targetKey), target.displayPath, signal)
if (entry === undefined) return undefined
return {
version: entryVersion(entry),
type: entryType(entry),
...(entry.type === FileType.FILE ? { size: entry.size } : {}),
}
}
override async lstat(path: string, opts?: { cwd?: string }, signal?: AbortSignal): Promise<FsPathInfo | undefined> {
assertNotAborted(signal, 'lstat')
if (path.trim().length === 0) throw new FsError('file_path must be a non-empty string', 'FS_NOT_FOUND')
const displayPath = posix.resolve(opts?.cwd ?? this.ctx.e2b.cwd, path)
const entry = await this.probe(displayPath, displayPath, signal)
if (entry === undefined) return undefined
const type = entry.symlinkTarget !== undefined
? 'symlink' as const
: entry.type === FileType.FILE
? 'file' as const
: entry.type === FileType.DIR
? 'directory' as const
: 'other' as const
return {
version: entryVersion(entry),
type,
...(entry.type === FileType.FILE ? { size: entry.size } : {}),
}
}
override async readText(target: FsTarget, signal?: AbortSignal): Promise<string> {
const sandbox = await this.ctx.e2b.getSandbox()
await this.requireRegular(target, signal)
try {
const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) })
assertNotAborted(signal, 'read')
return decodeText(bytes, target.displayPath, BINARY_SAMPLE_BYTES)
} catch (error: unknown) {
throw mapError(error, 'read', target.displayPath, signal)
}
}
override async streamText(target: FsTarget, signal?: AbortSignal): Promise<AsyncIterable<string>> {
const sandbox = await this.ctx.e2b.getSandbox()
await this.requireRegular(target, signal)
let stream: ReadableStream<Uint8Array>
try {
stream = await sandbox.files.read(String(target.targetKey), { format: 'stream', ...signalOpts(signal) })
} catch (error: unknown) {
throw mapError(error, 'read', target.displayPath, signal)
}
const displayPath = target.displayPath
return {
async *[Symbol.asyncIterator](): AsyncGenerator<string> {
const reader = stream.getReader()
const decoder = new TextDecoder('utf-8', { fatal: true })
let sampledBytes = 0
try {
while (true) {
assertNotAborted(signal, 'read')
const next = await reader.read()
if (next.done) break
if (sampledBytes < BINARY_SAMPLE_BYTES) {
const sample = next.value.subarray(0, BINARY_SAMPLE_BYTES - sampledBytes)
if (sample.includes(0)) throw new FsError(`cannot read "${displayPath}": binary file`, 'FS_NOT_TEXT')
sampledBytes += sample.length
}
let text: string
try {
text = decoder.decode(next.value, { stream: true })
} catch (error: unknown) {
throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error })
}
if (text.length > 0) yield text
}
try {
decoder.decode()
} catch (error: unknown) {
throw new FsError(`cannot read "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT', { cause: error })
}
} catch (error: unknown) {
throw mapError(error, 'read', displayPath, signal)
} finally {
reader.releaseLock()
}
},
}
}
override async listDir(target: FsTarget, signal?: AbortSignal): Promise<FsDirEntry[]> {
const info = await this.stat(target, signal)
if (info === undefined) throw new FsError(`cannot list "${target.displayPath}": not found`, 'FS_NOT_FOUND')
if (info.type !== 'directory') throw new FsError(`cannot list "${target.displayPath}": not a directory`, 'FS_NOT_DIRECTORY')
try {
const sandbox = await this.ctx.e2b.getSandbox()
const listed = await sandbox.files.list(String(target.targetKey), { depth: 1, ...signalOpts(signal) })
const entries = await Promise.all(listed.map(async (entry): Promise<FsDirEntry> => {
const displayPath = posix.join(target.displayPath, entry.name)
const canonical = await this.canonicalPath(sandbox, entry.path, signal)
const resolved = await this.probe(canonical, displayPath, signal)
return {
name: entry.name,
type: resolved === undefined ? 'other' : entryType(resolved),
target: { targetKey: FsTargetKey(canonical), displayPath },
...(resolved !== undefined ? { version: entryVersion(resolved) } : {}),
...(resolved?.type === FileType.FILE ? { size: resolved.size } : {}),
}
}))
return entries.sort((left, right) => left.name.localeCompare(right.name))
} catch (error: unknown) {
throw mapError(error, 'list', target.displayPath, signal)
}
}
override async writeText(
target: FsTarget,
content: string,
expected?: FsWriteIntent,
signal?: AbortSignal,
): Promise<FsWriteOutcome> {
return this.withLock(String(target.targetKey), async () => {
const existing = await this.probe(String(target.targetKey), target.displayPath, signal)
if (existing !== undefined && entryType(existing) !== 'file') {
throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
}
this.checkWriteIntent(existing, expected, target)
const before = existing === undefined ? null : await this.readForDiff(target, signal)
const version = await this.writeAtomic(target, content, existing, signal)
return {
operation: existing === undefined ? 'create' : 'update',
version,
before,
after: normalizeLineEndings(content),
}
})
}
override async editText(
target: FsTarget,
edit: FsEditRequest,
expected?: { version: ReturnType<typeof FsVersion> },
signal?: AbortSignal,
): Promise<FsEditOutcome> {
return this.withLock(String(target.targetKey), async () => {
const existing = await this.probe(String(target.targetKey), target.displayPath, signal)
if (existing === undefined) {
throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
}
if (entryType(existing) !== 'file') {
throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
}
if (expected !== undefined && entryVersion(existing) !== expected.version) {
throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
}
const raw = await this.readForEdit(target, signal)
const before = normalizeLineEndings(raw)
const after = literalEdit(before, edit, target.displayPath)
const storage = restoreLineEndings(after, detectsCrlf(raw))
const version = await this.writeAtomic(target, storage, existing, signal)
return { version, before, after }
})
}
private async withLock<T>(targetKey: string, operation: () => Promise<T>): Promise<T> {
const prior = this.locks.get(targetKey) ?? Promise.resolve()
const run = prior.then(operation, operation)
const tail = run.then(() => undefined, () => undefined)
this.locks.set(targetKey, tail)
try {
return await run
} finally {
if (this.locks.get(targetKey) === tail) this.locks.delete(targetKey)
}
}
private async canonicalPath(sandbox: Sandbox, path: string, signal?: AbortSignal): Promise<string> {
try {
const result = await sandbox.commands.run(`realpath -m -- ${quoteE2BShellArg(path)}`, signalOpts(signal))
return result.stdout.replace(/\n$/, '')
} catch (error: unknown) {
if (error instanceof CommandExitError) throw new Error(error.stderr || error.message, { cause: error })
throw error
}
}
private async probe(path: string, displayPath: string, signal?: AbortSignal): Promise<EntryInfo | undefined> {
assertNotAborted(signal, 'stat')
try {
const sandbox = await this.ctx.e2b.getSandbox()
const entry = await sandbox.files.getInfo(path, signalOpts(signal))
assertNotAborted(signal, 'stat')
return entry
} catch (error: unknown) {
if (error instanceof FileNotFoundError) return undefined
throw mapError(error, 'stat', displayPath, signal)
}
}
private async requireRegular(target: FsTarget, signal?: AbortSignal): Promise<void> {
const info = await this.stat(target, signal)
if (info === undefined) throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND')
if (info.type !== 'file') throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
}
private checkWriteIntent(existing: EntryInfo | undefined, expected: FsWriteIntent | undefined, target: FsTarget): void {
if (expected?.kind === 'createIfAbsent' && existing !== undefined) {
throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED')
}
if (expected?.kind === 'replaceIfVersion') {
if (existing === undefined || entryVersion(existing) !== expected.version) {
throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
}
}
}
private async readForDiff(target: FsTarget, signal?: AbortSignal): Promise<string | null> {
try {
const sandbox = await this.ctx.e2b.getSandbox()
const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) })
assertNotAborted(signal, 'read')
return normalizeLineEndings(decodeText(bytes, target.displayPath, bytes.length))
} catch (error: unknown) {
if (error instanceof FsError && error.code === 'FS_NOT_TEXT') return null
throw mapError(error, 'read', target.displayPath, signal)
}
}
private async readForEdit(target: FsTarget, signal?: AbortSignal): Promise<string> {
try {
const sandbox = await this.ctx.e2b.getSandbox()
const bytes = await sandbox.files.read(String(target.targetKey), { format: 'bytes', ...signalOpts(signal) })
assertNotAborted(signal, 'edit')
return decodeText(bytes, target.displayPath, bytes.length)
} catch (error: unknown) {
throw mapError(error, 'edit', target.displayPath, signal)
}
}
private async writeAtomic(
target: FsTarget,
content: string,
existing: EntryInfo | undefined,
signal?: AbortSignal,
): Promise<ReturnType<typeof FsVersion>> {
assertNotAborted(signal, 'write')
const sandbox = await this.ctx.e2b.getSandbox()
const targetPath = String(target.targetKey)
const versionId = randomUUID()
const temporary = posix.join(posix.dirname(targetPath), `.${posix.basename(targetPath)}.dsh-${randomUUID()}.tmp`)
try {
await sandbox.files.write(temporary, content, {
metadata: { [VERSION_METADATA_KEY]: versionId },
...signalOpts(signal),
})
assertNotAborted(signal, 'write')
const mode = existing === undefined ? 0o600 : existing.mode & 0o777
await sandbox.commands.run(
`chmod ${mode.toString(8)} -- ${quoteE2BShellArg(temporary)}`,
signalOpts(signal),
)
assertNotAborted(signal, 'write')
await sandbox.commands.run(
`mv -f -- ${quoteE2BShellArg(temporary)} ${quoteE2BShellArg(targetPath)}`,
signalOpts(signal),
)
const committed = await sandbox.files.getInfo(targetPath)
return entryVersion(committed)
} catch (error: unknown) {
try {
await sandbox.files.remove(temporary)
} catch (_temporaryAlreadyAbsent) {
// Only the private staging path is swallowed; the original failure owns the operation.
}
throw mapError(error, 'write', target.displayPath, signal)
}
}
}
export default E2BFileSystem

View File

@@ -0,0 +1,27 @@
/** Package-owned invariant companion for `@deepseek-ai/dsh-fs-e2b`. */
/* jscpd:ignore-start */
import type { Context } from 'cordis'
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
const PACKAGE_NAME = '@deepseek-ai/dsh-fs-e2b'
/** Cordis companion plugin name. */
export const name = 'fs-e2b-invariant'
/** Service required before reserving package ownership. */
export const inject = ['invariants']
/**
* No runtime invariant: each operation returns the E2B controller's committed
* result directly, with no independent event or cache to cross-check.
*/
const install: InvariantInstaller = () => {}
/**
* Register this package's invariant companion.
* @param ctx - Cordis context carrying the invariant service.
* @returns the installed registration's disposer after setup succeeds.
*/
export const apply = (ctx: Context): Promise<() => void> =>
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
/* jscpd:ignore-end */

View File

@@ -0,0 +1,537 @@
import { dirname, posix } from 'node:path'
import { Context } from 'cordis'
import {
CommandExitError,
FileNotFoundError,
FileType,
type EntryInfo,
type Sandbox,
} from '@deepseek-ai/dsh-e2b'
import type E2BSandboxService from '@deepseek-ai/dsh-e2b'
import { FsVersion } from '@deepseek-ai/dsh-fs'
import E2BFileSystem from '@deepseek-ai/dsh-fs-e2b'
import * as E2BFsInvariant from '../src/invariant.ts'
import InvariantService from '@deepseek-ai/dsh-invariants'
import { describe, expect, it } from 'vitest'
interface RemoteNode {
type: FileType
data: Uint8Array
mode: number
modified: number
metadata?: Record<string, string>
symlinkTarget?: string
}
function bytes(value: string | readonly number[]): Uint8Array {
return typeof value === 'string' ? new TextEncoder().encode(value) : Uint8Array.from(value)
}
function commandError(exitCode: number, stderr = ''): CommandExitError {
return new CommandExitError({ exitCode, stdout: '', stderr, error: stderr })
}
class FakeRemote {
readonly nodes = new Map<string, RemoteNode>()
readonly writes: Array<{ path: string; data: string; metadata?: Record<string, string> }> = []
readonly renames: Array<{ from: string; to: string }> = []
readonly removals: string[] = []
readonly commands: string[] = []
streamChunks: Uint8Array[] | undefined
nextCommandError: unknown
nextInfoError: unknown
nextListError: unknown
nextReadError: unknown
nextRenameError: unknown
nextRemoveError: unknown
abortAfterRename: AbortController | undefined
disappearOnInfo = new Set<string>()
private clock = 1
constructor() {
this.dir('/')
this.dir('/workspace')
}
dir(path: string): void {
this.nodes.set(path, { type: FileType.DIR, data: bytes(''), mode: 0o755, modified: this.clock++ })
}
file(path: string, data: string | readonly number[], mode = 0o644): void {
this.nodes.set(path, { type: FileType.FILE, data: bytes(data), mode, modified: this.clock++ })
}
other(path: string): void {
this.nodes.set(path, { type: 'other' as FileType, data: bytes(''), mode: 0o600, modified: this.clock++ })
}
symlink(path: string, target: string): void {
this.nodes.set(path, {
type: FileType.FILE,
data: bytes(''),
mode: 0o777,
modified: this.clock++,
symlinkTarget: target,
})
}
mutate(path: string, data: string): void {
const node = this.required(path)
node.data = bytes(data)
node.modified = this.clock++
}
private required(path: string): RemoteNode {
const node = this.nodes.get(path)
if (node === undefined) throw new FileNotFoundError(`missing: ${path}`)
return node
}
private followed(path: string): { path: string; node: RemoteNode; link?: RemoteNode } {
const node = this.required(path)
if (node.symlinkTarget === undefined) return { path, node }
return { path: node.symlinkTarget, node: this.required(node.symlinkTarget), link: node }
}
private info(path: string): EntryInfo {
if (this.disappearOnInfo.delete(path)) throw new FileNotFoundError(`missing: ${path}`)
return this.rawInfo(path)
}
private rawInfo(path: string): EntryInfo {
const followed = this.followed(path)
const node = followed.node
return {
name: posix.basename(path),
path,
type: node.type,
size: node.data.byteLength,
mode: node.mode,
permissions: 'rw-------',
owner: 'user',
group: 'user',
modifiedTime: new Date(node.modified),
...(node.metadata !== undefined ? { metadata: { ...node.metadata } } : {}),
...(followed.link?.symlinkTarget !== undefined ? { symlinkTarget: followed.link.symlinkTarget } : {}),
}
}
private checkAbort(options: { signal?: AbortSignal } | undefined): void {
if (options?.signal?.aborted === true) throw new DOMException('aborted', 'AbortError')
}
readonly sandbox = {
sandboxId: 'fake',
files: {
makeDir: async (path: string): Promise<boolean> => {
if (this.nodes.has(path)) return false
this.dir(path)
return true
},
getInfo: async (path: string, options?: { signal?: AbortSignal }): Promise<EntryInfo> => {
this.checkAbort(options)
if (this.nextInfoError !== undefined) {
const error = this.nextInfoError
this.nextInfoError = undefined
throw error
}
return this.info(path)
},
read: async (path: string, options: { format: 'bytes' | 'stream'; signal?: AbortSignal }): Promise<Uint8Array | ReadableStream<Uint8Array>> => {
this.checkAbort(options)
if (this.nextReadError !== undefined) {
const error = this.nextReadError
this.nextReadError = undefined
throw error
}
const data = this.followed(path).node.data
if (options.format === 'bytes') return data.slice()
const chunks = this.streamChunks ?? [data.slice()]
return new ReadableStream<Uint8Array>({
start(controller) {
for (const chunk of chunks) controller.enqueue(chunk)
controller.close()
},
})
},
list: async (path: string, options?: { depth?: number; signal?: AbortSignal }): Promise<EntryInfo[]> => {
this.checkAbort(options)
if (this.nextListError !== undefined) {
const error = this.nextListError
this.nextListError = undefined
throw error
}
this.required(path)
return [...this.nodes.keys()]
.filter(candidate => candidate !== path && dirname(candidate) === path)
.map(candidate => this.rawInfo(candidate))
},
write: async (path: string, data: string, options?: { metadata?: Record<string, string>; signal?: AbortSignal }): Promise<object> => {
this.checkAbort(options)
const parent = dirname(path)
if (!this.nodes.has(parent)) this.dir(parent)
this.nodes.set(path, {
type: FileType.FILE,
data: bytes(data),
mode: 0o644,
modified: this.clock++,
...(options?.metadata !== undefined ? { metadata: { ...options.metadata } } : {}),
})
this.writes.push({ path, data, ...(options?.metadata !== undefined ? { metadata: options.metadata } : {}) })
return {}
},
rename: async (from: string, to: string, options?: { signal?: AbortSignal }): Promise<EntryInfo> => {
this.checkAbort(options)
if (this.nextRenameError !== undefined) {
const error = this.nextRenameError
this.nextRenameError = undefined
throw error
}
const node = this.required(from)
this.nodes.delete(from)
this.nodes.set(to, node)
this.renames.push({ from, to })
this.abortAfterRename?.abort('after commit')
return this.info(to)
},
remove: async (path: string): Promise<void> => {
this.removals.push(path)
if (this.nextRemoveError !== undefined) {
const error = this.nextRemoveError
this.nextRemoveError = undefined
throw error
}
this.nodes.delete(path)
},
},
commands: {
run: async (command: string, options?: { signal?: AbortSignal }): Promise<{ exitCode: number; stdout: string; stderr: string }> => {
this.checkAbort(options)
this.commands.push(command)
if (this.nextCommandError !== undefined) {
const error = this.nextCommandError
this.nextCommandError = undefined
throw error
}
if (command.startsWith('realpath -m -- ')) {
const input = command.slice('realpath -m -- '.length).slice(1, -1)
const node = this.nodes.get(input)
return { exitCode: 0, stdout: `${node?.symlinkTarget ?? input}\n`, stderr: '' }
}
const chmod = /^chmod ([0-7]+) -- '([^']+)'$/.exec(command)
if (chmod !== null) this.required(chmod[2]!).mode = Number.parseInt(chmod[1]!, 8)
const move = /^mv -f -- '([^']+)' '([^']+)'$/.exec(command)
if (move !== null) {
if (this.nextRenameError !== undefined) {
const error = this.nextRenameError
this.nextRenameError = undefined
throw error
}
const node = this.required(move[1]!)
this.nodes.delete(move[1]!)
this.nodes.set(move[2]!, node)
this.renames.push({ from: move[1]!, to: move[2]! })
this.abortAfterRename?.abort('after commit')
}
return { exitCode: 0, stdout: '', stderr: '' }
},
},
} as unknown as Sandbox
}
async function setup(remote = new FakeRemote()): Promise<{ ctx: Context; fs: E2BFileSystem; remote: FakeRemote }> {
const ctx = new Context()
const runtime = {
cwd: '/workspace',
runtimeRoot: '/workspace/.dsh-e2b',
disposeMode: 'kill',
getSandbox: async () => remote.sandbox,
} as unknown as E2BSandboxService
ctx.provide('e2b', runtime)
await ctx.plugin(E2BFileSystem)
return { ctx, fs: ctx.fs as E2BFileSystem, remote }
}
async function expectCode(promise: Promise<unknown>, code: string): Promise<void> {
await expect(promise).rejects.toMatchObject({ code })
}
describe('E2BFileSystem identity, metadata, and reads', () => {
it('resolves remote paths, reports symlinks, and lists direct children in stable order', async () => {
const remote = new FakeRemote()
remote.file('/workspace/z.txt', 'z')
remote.file('/workspace/a.txt', 'a')
remote.dir('/workspace/dir')
remote.other('/workspace/special')
remote.file('/workspace/dir/nested.txt', 'nested')
remote.symlink('/workspace/link.txt', '/workspace/a.txt')
const { fs } = await setup(remote)
const link = await fs.resolve('link.txt')
expect(link).toEqual({ targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' })
await expect(fs.lstat('link.txt')).resolves.toMatchObject({ type: 'symlink', size: 1 })
await expect(fs.lstat('a.txt')).resolves.toMatchObject({ type: 'file', size: 1 })
await expect(fs.lstat('dir')).resolves.toEqual(expect.objectContaining({ type: 'directory' }))
await expect(fs.lstat('special')).resolves.toEqual(expect.objectContaining({ type: 'other' }))
await expect(fs.lstat('missing')).resolves.toBeUndefined()
await expect(fs.stat(link)).resolves.toMatchObject({ type: 'file', size: 1 })
const directory = await fs.resolve('.')
const listed = await fs.listDir(directory)
expect(listed.map(entry => entry.name)).toEqual(['a.txt', 'dir', 'link.txt', 'special', 'z.txt'])
expect(listed.find(entry => entry.name === 'dir')).toMatchObject({ type: 'directory' })
expect(listed.find(entry => entry.name === 'link.txt')).toMatchObject({
type: 'file',
target: { targetKey: '/workspace/a.txt', displayPath: '/workspace/link.txt' },
})
expect(listed.some(entry => entry.name === 'nested.txt')).toBe(false)
})
it('reads whole and streamed UTF-8 across chunk boundaries', async () => {
const remote = new FakeRemote()
remote.file('/workspace/text.txt', 'A€B')
remote.streamChunks = [bytes([65, 0xe2]), bytes([0x82, 0xac, 66])]
const { fs } = await setup(remote)
const target = await fs.resolve('text.txt')
await expect(fs.readText(target)).resolves.toBe('A€B')
let streamed = ''
for await (const chunk of await fs.streamText(target)) streamed += chunk
expect(streamed).toBe('A€B')
remote.streamChunks = [bytes([0xe2]), bytes([0x82, 0xac])]
let initiallyBuffered = ''
for await (const chunk of await fs.streamText(target)) initiallyBuffered += chunk
expect(initiallyBuffered).toBe('€')
})
it('matches local binary sampling while edits still reject any NUL byte', async () => {
const remote = new FakeRemote()
remote.file('/workspace/late-nul.txt', `${'a'.repeat(8192)}\0tail`)
const { fs } = await setup(remote)
const target = await fs.resolve('late-nul.txt')
await expect(fs.readText(target)).resolves.toContain('\0tail')
remote.streamChunks = [bytes('a'.repeat(8192)), bytes([0, 116])]
let streamed = ''
for await (const chunk of await fs.streamText(target)) streamed += chunk
expect(streamed).toBe(`${'a'.repeat(8192)}\0t`)
await expectCode(fs.editText(target, { oldString: 'tail', newString: 'end', replaceAll: false }), 'FS_NOT_TEXT')
})
it('maps binary, invalid UTF-8, missing, and non-regular read failures', async () => {
const remote = new FakeRemote()
remote.file('/workspace/binary', [0, 1])
remote.file('/workspace/invalid', [0xff])
remote.dir('/workspace/directory')
const { fs } = await setup(remote)
await expectCode(fs.readText(await fs.resolve('binary')), 'FS_NOT_TEXT')
await expectCode(fs.readText(await fs.resolve('invalid')), 'FS_NOT_TEXT')
await expectCode(fs.readText(await fs.resolve('missing')), 'FS_NOT_FOUND')
await expectCode(fs.readText(await fs.resolve('directory')), 'FS_NOT_REGULAR_FILE')
remote.streamChunks = [bytes([0xff])]
const invalid = await fs.streamText(await fs.resolve('invalid'))
await expect((async () => { for await (const _chunk of invalid) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
remote.streamChunks = [bytes([0])]
const binary = await fs.streamText(await fs.resolve('binary'))
await expect((async () => { for await (const _chunk of binary) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
remote.streamChunks = [bytes([0xe2])]
const incomplete = await fs.streamText(await fs.resolve('invalid'))
await expect((async () => { for await (const _chunk of incomplete) void _chunk })()).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
const raced = await fs.resolve('invalid')
remote.nextReadError = new FileNotFoundError('gone after stat')
await expectCode(fs.streamText(raced), 'FS_NOT_FOUND')
})
it('honors aborts before and during remote reads', async () => {
const remote = new FakeRemote()
remote.file('/workspace/a', 'a')
const { fs } = await setup(remote)
await expectCode(fs.resolve('a', { signal: AbortSignal.abort() }), 'FS_ABORTED')
await expectCode(fs.lstat('a', undefined, AbortSignal.abort()), 'FS_ABORTED')
await expectCode(fs.stat(await fs.resolve('a'), AbortSignal.abort()), 'FS_ABORTED')
remote.nextReadError = new DOMException('aborted', 'AbortError')
await expectCode(fs.readText(await fs.resolve('a')), 'FS_ABORTED')
})
it('rejects empty paths and directory-listing type errors', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file', 'x')
const { fs } = await setup(remote)
await expectCode(fs.resolve(' '), 'FS_NOT_FOUND')
await expectCode(fs.lstat(''), 'FS_NOT_FOUND')
await expectCode(fs.listDir(await fs.resolve('missing')), 'FS_NOT_FOUND')
await expectCode(fs.listDir(await fs.resolve('/workspace/file')), 'FS_NOT_DIRECTORY')
remote.nextListError = new Error('listing transport failed')
await expectCode(fs.listDir(await fs.resolve('/workspace')), 'FS_IO_ERROR')
})
})
describe('E2BFileSystem atomic writes and edits', () => {
it('creates owner-only files and returns metadata after the committed move', async () => {
const { fs, remote } = await setup()
const target = await fs.resolve('new.txt')
const outcome = await fs.writeText(target, 'one\r\ntwo\rthree', { kind: 'createIfAbsent' })
expect(outcome).toMatchObject({ operation: 'create', before: null, after: 'one\ntwo\rthree' })
expect(remote.nodes.get('/workspace/new.txt')?.mode).toBe(0o600)
expect(remote.nodes.get('/workspace/new.txt')?.metadata?.['dsh-version']).toBeDefined()
await expect(fs.stat(target)).resolves.toMatchObject({ version: outcome.version, size: 14 })
})
it('preserves replacement mode, normalizes only CRLF for diffs, and changes version on external writes', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'old\r\nline\rlone', 0o640)
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
const before = (await fs.stat(target))!.version
const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: before })
expect(outcome).toMatchObject({ operation: 'update', before: 'old\nline\rlone', after: 'new' })
expect(remote.nodes.get('/workspace/file.txt')?.mode).toBe(0o640)
const committed = outcome.version
remote.mutate('/workspace/file.txt', 'external')
expect((await fs.stat(target))!.version).not.toBe(committed)
})
it('returns null as the overwrite diff basis for binary or invalid prior content', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', [0xff])
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
await expect(fs.writeText(target, 'valid')).resolves.toMatchObject({ before: null, after: 'valid' })
})
it('fails an overwrite when reading its text diff basis fails for another reason', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'prior')
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
remote.nextReadError = new Error('read transport failed')
await expectCode(fs.writeText(target, 'replacement'), 'FS_IO_ERROR')
expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('prior')
})
it('enforces create and version intents before publication', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'v1')
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
const version = (await fs.stat(target))!.version
await expectCode(fs.writeText(target, 'blind', { kind: 'createIfAbsent' }), 'FS_NOT_OBSERVED')
remote.mutate('/workspace/file.txt', 'v2')
await expectCode(fs.writeText(target, 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION')
await expectCode(fs.writeText(await fs.resolve('missing'), 'stale', { kind: 'replaceIfVersion', version }), 'FS_STALE_VERSION')
remote.dir('/workspace/dir')
await expectCode(fs.writeText(await fs.resolve('dir'), 'x'), 'FS_NOT_REGULAR_FILE')
})
it('does not turn an abort observed after a successful move into a failed write', async () => {
const remote = new FakeRemote()
const controller = new AbortController()
remote.abortAfterRename = controller
const { fs } = await setup(remote)
await expect(fs.writeText(await fs.resolve('committed'), 'yes', undefined, controller.signal))
.resolves.toMatchObject({ operation: 'create' })
expect(controller.signal.aborted).toBe(true)
})
it('cleans staging files and maps command, permission, and abort failures', async () => {
const remote = new FakeRemote()
const { fs } = await setup(remote)
const commandTarget = await fs.resolve('command')
remote.nextCommandError = commandError(1, 'chmod failed')
await expectCode(fs.writeText(commandTarget, 'x'), 'FS_IO_ERROR')
expect(remote.removals).toHaveLength(1)
remote.nextRenameError = new Error('permission denied')
await expectCode(fs.writeText(await fs.resolve('permission'), 'x'), 'FS_PERMISSION_DENIED')
remote.nextRemoveError = new Error('cleanup also failed')
remote.nextRenameError = new DOMException('aborted', 'AbortError')
await expectCode(fs.writeText(await fs.resolve('abort'), 'x'), 'FS_ABORTED')
})
it('applies literal edits atomically and restores the detected CRLF style', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'one\r\ntwo\r\nthree\n')
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
const version = (await fs.stat(target))!.version
const outcome = await fs.editText(
target,
{ oldString: 'two\r\n', newString: 'TWO\r\n', replaceAll: false },
{ version },
)
expect(outcome).toMatchObject({ before: 'one\ntwo\nthree\n', after: 'one\nTWO\nthree\n' })
expect(new TextDecoder().decode(remote.nodes.get('/workspace/file.txt')?.data)).toBe('one\r\nTWO\r\nthree\r\n')
})
it('reports stale and literal-match failures with stable codes', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'a a')
remote.dir('/workspace/dir')
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
await expectCode(fs.editText(target, { oldString: '', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND')
await expectCode(fs.editText(target, { oldString: 'z', newString: 'x', replaceAll: false }), 'FS_EDIT_NOT_FOUND')
await expectCode(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: false }), 'FS_AMBIGUOUS_EDIT')
await expect(fs.editText(target, { oldString: 'a', newString: 'x', replaceAll: true }))
.resolves.toMatchObject({ after: 'x x' })
await expectCode(fs.editText(target, { oldString: 'x', newString: 'y', replaceAll: false }, { version: FsVersion('stale') }), 'FS_STALE_VERSION')
await expectCode(fs.editText(await fs.resolve('missing'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_STALE_VERSION')
await expectCode(fs.editText(await fs.resolve('dir'), { oldString: 'x', newString: 'y', replaceAll: false }), 'FS_NOT_REGULAR_FILE')
})
it('serializes guarded mutations so only one stale version can win', async () => {
const remote = new FakeRemote()
remote.file('/workspace/file.txt', 'base')
const { fs } = await setup(remote)
const target = await fs.resolve('file.txt')
const version = (await fs.stat(target))!.version
const results = await Promise.allSettled([
fs.writeText(target, 'one', { kind: 'replaceIfVersion', version }),
fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }),
])
expect(results.filter(result => result.status === 'fulfilled')).toHaveLength(1)
expect(results.filter(result => result.status === 'rejected')).toHaveLength(1)
})
})
describe('E2B filesystem adapter integration edges', () => {
it('maps canonicalization, permission, and generic provider failures', async () => {
const remote = new FakeRemote()
const { fs } = await setup(remote)
remote.nextCommandError = commandError(1, 'not a directory')
await expectCode(fs.resolve('bad'), 'FS_IO_ERROR')
remote.nextCommandError = commandError(1)
await expectCode(fs.resolve('bad-again'), 'FS_IO_ERROR')
remote.nextCommandError = new Error('canonical transport failed')
await expectCode(fs.resolve('bad-transport'), 'FS_IO_ERROR')
remote.file('/workspace/a', 'a')
const target = await fs.resolve('a')
remote.nextInfoError = new Error('metadata transport failed')
await expectCode(fs.stat(target), 'FS_IO_ERROR')
remote.nextReadError = new Error('operation not permitted')
await expectCode(fs.readText(target), 'FS_PERMISSION_DENIED')
remote.nextReadError = 'transport vanished'
await expectCode(fs.readText(target), 'FS_IO_ERROR')
})
it('keeps a listed child whose metadata disappears as an other entry', async () => {
const remote = new FakeRemote()
remote.file('/workspace/a', 'a')
remote.disappearOnInfo.add('/workspace/a')
const { fs } = await setup(remote)
const listed = await fs.listDir(await fs.resolve('/workspace'))
expect(listed).toEqual([{
name: 'a',
type: 'other',
target: { targetKey: '/workspace/a', displayPath: '/workspace/a' },
}])
})
it('registers the package-owned empty invariant installer', async () => {
const ctx = new Context()
await ctx.plugin(InvariantService, { enabled: true })
const fiber = await ctx.plugin(E2BFsInvariant).await()
await fiber.dispose()
})
})

View File

@@ -0,0 +1,25 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": ["src"],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../e2b"
},
{
"path": "../../fs/fs"
},
{
"path": "../../support/invariants"
}
]
}