Merge remote-tracking branch 'origin/master' into feat/read-image-context
# Conflicts: # docs/config-catalog.i18n.yaml # docs/config-catalog.md # docs/config-catalog.zh.md # packages/support/llm-replay/src/index.ts # packages/support/llm-replay/tests/llm-replay.spec.ts
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md
|
||||
README.md: 4b65bc3c5a5b31a8b4f96ec86e2239c2da680897
|
||||
README.zh.md: 45d9646f89623d2de106cd8e1dab38941c912977
|
||||
README.md: d17dc0747833a0ecb85505260badc79ad739f27f
|
||||
README.zh.md: e13ab2f04b84b59fbb3846c08139e679b2d43be1
|
||||
|
||||
@@ -20,7 +20,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing.
|
||||
- **`readBytes`** — raw whole-file bytes with no decoding or binary rejection (the `read_image` tool validates content through the attachment service). The required byte cap short-circuits on the stat size before any content I/O; the subsequent stream reads at most one byte beyond the cap, so a file growing after stat still fails `FS_TOO_LARGE` without unbounded buffering.
|
||||
- **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`.
|
||||
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a regular file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`, while a non-regular path entry is preserved and rejected with `FS_NOT_REGULAR_FILE`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
|
||||
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a regular file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`, while a non-regular path entry is preserved and rejected with `FS_NOT_REGULAR_FILE`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). An overwrite returns the prior text as its contextual diff basis only when both the opened prior file and UTF-8 replacement are strictly below `config.diffBasisMaxBytes` (default 10 MiB). The descriptor read enforces that limit even if an external writer replaces or changes the file size after the initial probe. Otherwise the provider returns `before: null`, so presentation uses its whole-file fallback.
|
||||
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
|
||||
|
||||
The package-root SDK surface is the default/named `LocalFileSystem` class plus `Config`. Raw I/O lives in `src/fsio.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the thin service wiring.
|
||||
@@ -36,9 +36,9 @@ No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)).
|
||||
- **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`).
|
||||
- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard.
|
||||
- **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path.
|
||||
- **A sub-limit overwrite still buffers a contextual basis** — `writeText` may retain up to just below `config.diffBasisMaxBytes` of prior text in addition to the caller-owned replacement; the bound does not cap the returned `after` value or presentation's whole-file fallback.
|
||||
- **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits.
|
||||
- **The per-target mutation lock is in-process only** — guarded create still uses an atomic no-replace publication across processes, but replacement writers in another process are caught only when the optional version guard observes their metadata change; they are never serialized.
|
||||
- **Guarded creation requires hard-link support** — filesystems or mounts that reject hard-link publication cannot serve `createIfAbsent`; the provider preserves the missing target and reports `FS_IO_ERROR`.
|
||||
|
||||
@@ -20,7 +20,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片解码,因此超大文件无需整体保存在内存中,消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑。
|
||||
- **`readBytes`**:按原始字节读取整个文件,不做解码或二进制拒绝(`read_image` 工具通过附件服务校验内容)。必填的字节上限在任何内容 I/O 之前先按 stat 大小短路;随后的流最多多读一个字节,因此 stat 之后增长的文件仍会以 `FS_TOO_LARGE` 失败,不会无界缓冲。
|
||||
- **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。
|
||||
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的普通文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;非普通路径条目也会被保留,并以 `FS_NOT_REGULAR_FILE` 拒绝;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
|
||||
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的普通文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;非普通路径条目也会被保留,并以 `FS_NOT_REGULAR_FILE` 拒绝;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。仅当打开后的旧文件和 UTF-8 替换内容都严格低于 `config.diffBasisMaxBytes`(默认 10 MiB)时,覆写才返回旧文本作为上下文 diff 基础。即使外部写入方在初次探测后替换文件或改变文件大小,文件描述符读取仍会强制执行该上限;否则提供方返回 `before: null`,由展示层使用整文件回退。
|
||||
- **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。
|
||||
|
||||
包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。
|
||||
@@ -36,9 +36,9 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
## 已知限制与延期工作
|
||||
|
||||
- **`config.cwd` 不是沙箱**:它是解析默认值,而非约束;绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall(瀑布式事件)上的权限插件实施约束(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences))。
|
||||
- **覆盖会把整个旧文件读入内存**:只用于 UI diff;在大小阈值之上限制这次预读取的工作延期处理(`TODO(overwrite-diff-bound)`)。
|
||||
- **版本 token 依赖文件系统元数据**:它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime;如果存储层在重写时无法更新其中任何一项事实,仍可能绕过陈旧防护。
|
||||
- **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。
|
||||
- **低于上限的覆写仍会缓冲上下文基础**:`writeText` 除调用方持有的替换内容外,最多还会保留略低于 `config.diffBasisMaxBytes` 的旧文本;该上限不限制返回的 `after` 值,也不限制展示层的整文件回退。
|
||||
- **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。
|
||||
- **每目标变更锁仅限进程内**:即使跨进程,带防护的创建仍采用原子且不替换的发布方式;但只有当可选版本防护观察到元数据变化时,系统才能发现其他进程中的替换写入方,且绝不会将其串行化。
|
||||
- **带防护的创建要求支持硬链接**:拒绝硬链接发布的文件系统或挂载点无法支持 `createIfAbsent`;提供方会使目标保持缺失状态并报告 `FS_IO_ERROR`。
|
||||
|
||||
@@ -15,6 +15,8 @@ import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs'
|
||||
import { copyFileDaclWin32, replaceFileWin32 } from './win32.ts'
|
||||
|
||||
const BINARY_SAMPLE_BYTES = 8192
|
||||
// Bound one non-abortable FileHandle.read so cancellation is observed between chunks.
|
||||
const DIFF_BASIS_READ_CHUNK_BYTES = 64 * 1024
|
||||
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
|
||||
@@ -74,9 +76,8 @@ function versionOf(info: BigIntStats): FsVersion {
|
||||
}
|
||||
|
||||
/**
|
||||
* Test hook: lets specs pin the atomic-write temp names (to prove
|
||||
* exclusive-open behavior without a name race) and observe the staged temp
|
||||
* file before it is renamed over the target.
|
||||
* Test hook: lets specs pin the atomic-write temp names (to prove exclusive-open behavior without
|
||||
* a name race), override native boundaries, and observe the staged temp file before publication.
|
||||
*/
|
||||
export interface FsIoInternals {
|
||||
/** Override the host platform for native-publication unit coverage. */
|
||||
@@ -680,21 +681,67 @@ export async function readForEdit(
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort overwrite diff basis. Binary or invalid UTF-8 returns `null` so the write still
|
||||
* succeeds and presentation falls back to a whole-file diff.
|
||||
* @param absolutePath - the file to read (typically a target key); it must exist.
|
||||
* @param signal - aborts the read (`FS_ABORTED`).
|
||||
* @returns the LF-normalized text, or null for a binary or non-UTF-8 file.
|
||||
* Best-effort overwrite diff basis. Binary, invalid UTF-8, a file at/above the byte limit,
|
||||
* or a file deleted/made unreadable after the caller's preflight returns `null` so the write
|
||||
* still succeeds and presentation falls back to a whole-file diff. The bound is enforced on
|
||||
* the opened descriptor rather than a prior path stat, so concurrent external replacement or
|
||||
* size changes cannot make this helper buffer more than `maxBytes`.
|
||||
* @param absolutePath - the file to read (typically a target key).
|
||||
* @param maxBytes - exclusive upper bound for bytes held as the contextual-diff basis.
|
||||
* @param signal - aborts the read (`FS_ABORTED`); cancellation propagates, unlike I/O failure.
|
||||
* @returns the LF-normalized text, or null for a non-regular, at/above-limit, binary, non-UTF-8,
|
||||
* descriptor-size-changed, or unreadable file.
|
||||
*/
|
||||
export async function readTextForDiff(absolutePath: string, signal?: AbortSignal): Promise<string | null> {
|
||||
const buffer = await readFileAbortable(absolutePath, 'read', signal)
|
||||
if (buffer.includes(0)) return null
|
||||
export async function readTextForDiff(
|
||||
absolutePath: string,
|
||||
maxBytes: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<string | null> {
|
||||
throwIfAborted(signal, 'read')
|
||||
try {
|
||||
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(buffer))
|
||||
const handle = await open(absolutePath, 'r')
|
||||
let buffer: Buffer
|
||||
let total = 0
|
||||
let openedSize = 0
|
||||
try {
|
||||
throwIfAborted(signal, 'read')
|
||||
const info = await handle.stat()
|
||||
throwIfAborted(signal, 'read')
|
||||
if (!info.isFile()) return null
|
||||
if (info.size >= maxBytes) return null
|
||||
openedSize = info.size
|
||||
// One extra byte detects growth after stat without retaining per-read backing buffers.
|
||||
buffer = Buffer.allocUnsafe(openedSize + 1)
|
||||
while (total < buffer.length) {
|
||||
throwIfAborted(signal, 'read')
|
||||
const length = Math.min(buffer.length - total, DIFF_BASIS_READ_CHUNK_BYTES)
|
||||
const { bytesRead } = await handle.read(buffer, total, length, null)
|
||||
if (bytesRead === 0) break
|
||||
total += bytesRead
|
||||
}
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
throwIfAborted(signal, 'read')
|
||||
if (total !== openedSize) return null
|
||||
const basis = buffer.subarray(0, total)
|
||||
if (basis.includes(0)) return null
|
||||
try {
|
||||
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(basis))
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes;
|
||||
* any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
return null
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
return null
|
||||
// Cancellation is the caller's intent and still propagates.
|
||||
if (error instanceof FsError) throw error
|
||||
// A descriptor-phase errno — deleted or made unreadable after the caller's
|
||||
// preflight, or a faulted read — costs only the optional basis: a committed
|
||||
// write must not fail for a presentation-only pre-read.
|
||||
if (error instanceof Error && 'code' in error) return null
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import { constants as bufferConstants } from 'node:buffer'
|
||||
import { isAbsolute, relative, resolve, sep } from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
import z from 'schemastery'
|
||||
@@ -40,9 +41,19 @@ import type { FsIoInternals } from './fsio.ts'
|
||||
export interface Config {
|
||||
/** Base directory for relative paths. Defaults to `process.cwd()`. */
|
||||
cwd?: string
|
||||
/**
|
||||
* Exclusive UTF-8 byte limit on each overwrite-diff side, capped by the
|
||||
* runtime's safe allocation/decode maximum. Defaults to 10 MiB.
|
||||
*/
|
||||
diffBasisMaxBytes?: number
|
||||
}
|
||||
|
||||
type ResolvedConfig = Required<Config>
|
||||
const DEFAULT_DIFF_BASIS_MAX_BYTES = 10 * 1024 * 1024
|
||||
const MAX_DIFF_BASIS_BYTES = Math.min(
|
||||
bufferConstants.MAX_LENGTH,
|
||||
bufferConstants.MAX_STRING_LENGTH,
|
||||
)
|
||||
|
||||
/**
|
||||
* The host-filesystem backend. Reads resolve relative paths from {@link Config.cwd}
|
||||
@@ -53,11 +64,12 @@ type ResolvedConfig = Required<Config>
|
||||
export class LocalFileSystem extends FileSystem {
|
||||
static Config: z<Config> = z.object({
|
||||
cwd: z.string().default(process.cwd()),
|
||||
diffBasisMaxBytes: z.number().default(DEFAULT_DIFF_BASIS_MAX_BYTES),
|
||||
})
|
||||
|
||||
/** Validated config (schemastery applied the defaults before construction). */
|
||||
readonly config: ResolvedConfig
|
||||
/** Test hook forwarded to fsio (force streaming path, pin temp names). */
|
||||
/** Test hook forwarded to fsio for atomic-publication boundaries. */
|
||||
internals: FsIoInternals = {}
|
||||
/** Per-targetKey tail promise: serializes mutating ops so the read→guard→write
|
||||
* window can't interleave, making concurrent writes/edits deterministically
|
||||
@@ -66,7 +78,13 @@ export class LocalFileSystem extends FileSystem {
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
this.config = config as ResolvedConfig
|
||||
const resolved = config as ResolvedConfig
|
||||
if (!Number.isSafeInteger(resolved.diffBasisMaxBytes)
|
||||
|| resolved.diffBasisMaxBytes <= 0
|
||||
|| resolved.diffBasisMaxBytes > MAX_DIFF_BASIS_BYTES) {
|
||||
throw new Error(`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${MAX_DIFF_BASIS_BYTES}`)
|
||||
}
|
||||
this.config = resolved
|
||||
}
|
||||
|
||||
/** Run `op` with exclusive access to `targetKey` (FIFO per key). */
|
||||
@@ -169,9 +187,16 @@ export class LocalFileSystem extends FileSystem {
|
||||
}
|
||||
// No expectation means an unconditional but still atomic write.
|
||||
|
||||
// Preserve prior text for contextual diffs; null falls back to a whole-file diff.
|
||||
// TODO(overwrite-diff-bound): cap this UI-only pre-read for large files.
|
||||
const before = existing ? await readTextForDiff(target.targetKey, signal) : null
|
||||
// Capture an optional contextual-diff basis before the write. The bounded
|
||||
// reader checks the opened file itself, so an external replacement after
|
||||
// `probe()` cannot turn this best-effort presentation read into an
|
||||
// unbounded allocation. Either side at/above the configured limit yields
|
||||
// `before: null`; consumers retain their whole-file fallback.
|
||||
const diffable = existing !== null
|
||||
&& Buffer.byteLength(content, 'utf8') < this.config.diffBasisMaxBytes
|
||||
const before = diffable
|
||||
? await readTextForDiff(target.targetKey, this.config.diffBasisMaxBytes, signal)
|
||||
: null
|
||||
await writeFileAtomic(
|
||||
target.targetKey,
|
||||
content,
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { constants as bufferConstants } from 'node:buffer'
|
||||
import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -43,13 +44,39 @@ async function versionOf(target: FsTarget): Promise<FsVersion> {
|
||||
return info.version
|
||||
}
|
||||
|
||||
async function remountWithDiffLimit(diffBasisMaxBytes: number): Promise<void> {
|
||||
await fiber.dispose()
|
||||
fiber = await ctx.plugin(LocalFileSystem, { cwd: dir, diffBasisMaxBytes })
|
||||
fs = ctx.fs as LocalFileSystem
|
||||
}
|
||||
|
||||
describe('registration', () => {
|
||||
it('registers LocalFileSystem as ctx.fs with a default cwd', async () => {
|
||||
const bare = new Context()
|
||||
const bareFiber = await bare.plugin(LocalFileSystem)
|
||||
expect((bare.fs as LocalFileSystem).config.cwd).toBe(process.cwd())
|
||||
expect((bare.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(10 * 1024 * 1024)
|
||||
await bareFiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects non-positive, fractional, unsafe, or unallocatable diff-basis limits', async () => {
|
||||
const maxDiffBasisBytes = Math.min(
|
||||
bufferConstants.MAX_LENGTH,
|
||||
bufferConstants.MAX_STRING_LENGTH,
|
||||
)
|
||||
const valid = new Context()
|
||||
const validFiber = await valid.plugin(LocalFileSystem, { diffBasisMaxBytes: maxDiffBasisBytes })
|
||||
expect((valid.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(maxDiffBasisBytes)
|
||||
await validFiber.dispose()
|
||||
|
||||
for (const diffBasisMaxBytes of [0, -1, 1.5, maxDiffBasisBytes + 1, Number.MAX_SAFE_INTEGER + 1]) {
|
||||
const invalid = new Context()
|
||||
await expect(invalid.plugin(LocalFileSystem, { diffBasisMaxBytes })).rejects.toThrow(
|
||||
`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${maxDiffBasisBytes}`,
|
||||
)
|
||||
await invalid.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('resolve', () => {
|
||||
@@ -481,6 +508,54 @@ describe('writeText', () => {
|
||||
expect(outcome.after).toBe('now valid')
|
||||
})
|
||||
|
||||
it('an overwrite of a prior file AT the whole-file bound reports before:null (undiffable), still succeeds', async () => {
|
||||
// The configured bound keeps the fixture small; 8 bytes at a bound of 8
|
||||
// pins the exclusive edge without coupling this provider to a read tool.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'big.txt'), '12345678')
|
||||
const target = await fs.resolve('big.txt')
|
||||
const outcome = await fs.writeText(target, 'tiny')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('tiny')
|
||||
})
|
||||
|
||||
it('an overwrite whose NEW content is at the whole-file bound reports before:null (no huge contextual diff)', async () => {
|
||||
// The bound gates BOTH sides of the diff pair: a small prior file rewritten
|
||||
// with at/above-bound content yields no contextual-hunk basis either, since
|
||||
// a small-to-huge rewrite's hunk is as large as the new content — the
|
||||
// consumer must fall back to the whole-file diff card, exactly like a
|
||||
// create of the same size.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'grow.txt'), 'tiny')
|
||||
const target = await fs.resolve('grow.txt')
|
||||
const outcome = await fs.writeText(target, '12345678')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('12345678')
|
||||
})
|
||||
|
||||
it('gates the NEW content by UTF-8 byte length, not character count', async () => {
|
||||
// Three CJK characters are 9 UTF-8 bytes: below an 8-byte bound by
|
||||
// characters but at/above it by bytes, so the basis must be declined.
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'cjk.txt'), 'tiny')
|
||||
const target = await fs.resolve('cjk.txt')
|
||||
const outcome = await fs.writeText(target, '你好吗')
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(outcome.before).toBeNull()
|
||||
expect(outcome.after).toBe('你好吗')
|
||||
})
|
||||
|
||||
it('an overwrite with BOTH sides below the whole-file bound keeps its contextual before basis', async () => {
|
||||
await remountWithDiffLimit(8)
|
||||
await writeFile(join(dir, 'small.txt'), '1234567')
|
||||
const target = await fs.resolve('small.txt')
|
||||
const outcome = await fs.writeText(target, 'new')
|
||||
expect(outcome.before).toBe('1234567')
|
||||
expect(outcome.after).toBe('new')
|
||||
})
|
||||
|
||||
it('releases per-target mutation locks after success and failure', async () => {
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.writeText(target, 'created', { kind: 'createIfAbsent' })
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* policy and lives in `dsh-fs-policy`, so it is not tested here.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
probe,
|
||||
probeNoFollow,
|
||||
readForEdit,
|
||||
readTextForDiff,
|
||||
readWholeText,
|
||||
resolveLocalTarget,
|
||||
restoreLineEndings,
|
||||
@@ -316,6 +317,261 @@ describe('readWholeText', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('readTextForDiff', () => {
|
||||
it('returns normalized text only when the opened file is strictly below the limit', async () => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'a\r\nb')
|
||||
expect(await readTextForDiff(file, 5)).toBe('a\nb')
|
||||
expect(await readTextForDiff(file, 4)).toBeNull()
|
||||
})
|
||||
|
||||
it('bounds the actual opened file rather than trusting an earlier path size', async () => {
|
||||
const file = join(dir, 'replaced.txt')
|
||||
await writeFile(file, 'tiny')
|
||||
const earlierSize = (await stat(file)).size
|
||||
await writeFile(file, '123456789')
|
||||
expect(earlierSize).toBeLessThan(8)
|
||||
expect(await readTextForDiff(file, 8)).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null when the opened file shrinks after descriptor stat', async () => {
|
||||
const file = join(dir, 'shrinking.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
await writeFile(file, 'abc')
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 8)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null when the opened file grows after descriptor stat', async () => {
|
||||
const file = join(dir, 'growing.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
await writeFile(file, 'abcdef-grown')
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 32)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null when the file vanishes before the basis open (deletion race)', async () => {
|
||||
expect(await readTextForDiff(join(dir, 'deleted-after-preflight.txt'), 32)).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null when the opened descriptor is no longer a regular file', async () => {
|
||||
const file = join(dir, 'swapped.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
const info = await handle.stat(...statArgs)
|
||||
return Object.assign(info, { isFile: () => false })
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
expect(await isolatedReadTextForDiff(file, 32)).toBeNull()
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('propagates a non-errno fault instead of masking it as a null basis', async () => {
|
||||
const file = join(dir, 'faulted.txt')
|
||||
await writeFile(file, 'abcdef')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open() {
|
||||
throw new TypeError('forged programming fault')
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
await expect(isolatedReadTextForDiff(file, 32)).rejects.toThrow('forged programming fault')
|
||||
} finally {
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('returns null for binary and invalid UTF-8 without blocking the caller write', async () => {
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
expect(await readTextForDiff(join(dir, 'bin'), 8)).toBeNull()
|
||||
expect(await readTextForDiff(join(dir, 'bad'), 8)).toBeNull()
|
||||
})
|
||||
|
||||
it('honors a pre-aborted signal', async () => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'text')
|
||||
await expect(readTextForDiff(file, 8, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
|
||||
it.each(['open', 'stat'] as const)('observes cancellation immediately after %s', async (stage) => {
|
||||
const file = join(dir, 'basis.txt')
|
||||
await writeFile(file, 'text')
|
||||
const reached = Promise.withResolvers<undefined>()
|
||||
const release = Promise.withResolvers<undefined>()
|
||||
let statCalls = 0
|
||||
const allocate = vi.spyOn(Buffer, 'allocUnsafe')
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
if (stage === 'open') {
|
||||
reached.resolve(undefined)
|
||||
await release.promise
|
||||
}
|
||||
return {
|
||||
close: handle.close.bind(handle),
|
||||
read: handle.read.bind(handle),
|
||||
async stat(...statArgs: Parameters<typeof handle.stat>) {
|
||||
statCalls += 1
|
||||
const info = await handle.stat(...statArgs)
|
||||
if (stage === 'stat') {
|
||||
reached.resolve(undefined)
|
||||
await release.promise
|
||||
}
|
||||
return info
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
const controller = new AbortController()
|
||||
const pending = isolatedReadTextForDiff(file, 8, controller.signal)
|
||||
await reached.promise
|
||||
const allocationCalls = allocate.mock.calls.length
|
||||
controller.abort()
|
||||
release.resolve(undefined)
|
||||
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
expect(statCalls).toBe(stage === 'open' ? 0 : 1)
|
||||
expect(allocate).toHaveBeenCalledTimes(allocationCalls)
|
||||
} finally {
|
||||
release.resolve(undefined)
|
||||
allocate.mockRestore()
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
|
||||
it('bounds descriptor reads and observes cancellation before the next chunk', async () => {
|
||||
const file = join(dir, 'large-basis.txt')
|
||||
const fileBytes = 200 * 1024
|
||||
await writeFile(file, 'x'.repeat(fileBytes))
|
||||
const firstRead = Promise.withResolvers<undefined>()
|
||||
const releaseFirstRead = Promise.withResolvers<undefined>()
|
||||
const readLengths: number[] = []
|
||||
vi.resetModules()
|
||||
vi.doMock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return {
|
||||
...actual,
|
||||
async open(...args: Parameters<typeof actual.open>) {
|
||||
const handle = await actual.open(...args)
|
||||
return {
|
||||
stat: handle.stat.bind(handle),
|
||||
close: handle.close.bind(handle),
|
||||
async read(buffer: Buffer, offset: number, length: number, position: number | null) {
|
||||
readLengths.push(length)
|
||||
const result = await handle.read(buffer, offset, length, position)
|
||||
if (readLengths.length === 1) {
|
||||
firstRead.resolve(undefined)
|
||||
await releaseFirstRead.promise
|
||||
}
|
||||
return result
|
||||
},
|
||||
}
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
|
||||
const controller = new AbortController()
|
||||
const pending = isolatedReadTextForDiff(file, fileBytes + 1, controller.signal)
|
||||
await firstRead.promise
|
||||
expect(readLengths).toEqual([64 * 1024])
|
||||
controller.abort()
|
||||
releaseFirstRead.resolve(undefined)
|
||||
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
expect(readLengths).toHaveLength(1)
|
||||
} finally {
|
||||
releaseFirstRead.resolve(undefined)
|
||||
vi.doUnmock('node:fs/promises')
|
||||
vi.resetModules()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('streamWholeText', () => {
|
||||
it('streams the whole file as decoded text', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/fs/fs-sandbox/README.md
|
||||
README.md: c40fc7999ab85a70702f65a5675208163f5fc351
|
||||
README.zh.md: 15db5abbfc5307c0570925026ec435d8dbb51bf2
|
||||
README.md: ae1fd746c711a86e308a02e0054ba478e8d913c0
|
||||
README.zh.md: e25a3467c06fbd93de9bb75d6b364e8da5a451fe
|
||||
|
||||
@@ -4,6 +4,8 @@ English | [中文](README.zh.md)
|
||||
|
||||
`SandboxedFileSystem` extends [`LocalFileSystem`](../fs-local/README.md) and registers as `ctx.fs`. It inherits every text-storage mechanic verbatim (resolve, stat, read/stream, list, the atomic write, the read-match-write edit critical section) and adds only a per-call MODE fence on `writeText`/`editText`. Reads always pass through — every mode permits reading.
|
||||
|
||||
Its plugin config is the local backend config unchanged: `cwd` remains the relative-path resolution default, and `diffBasisMaxBytes` bounds the optional overwrite contextual-diff basis.
|
||||
|
||||
Loading it INSTEAD OF `dsh-fs-local`, together with a [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md), is the whole swap; the model-facing tools (`dsh-tool-fs`) are untouched. The tool layer resolves the calling session's mode and cwd into the SAME per-call policy bash receives, so the two families never confine to different roots.
|
||||
|
||||
## The fence
|
||||
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
`SandboxedFileSystem` 扩展 [`LocalFileSystem`](../fs-local/README.md) 并注册为 `ctx.fs`。它逐字继承全部文本存储机制(解析、stat、读取/流式读取、列出、原子写入、按读取、匹配、写入顺序执行的编辑临界区),只为 `writeText`/`editText` 增加按调用的模式围栏。读取始终直接通过:所有模式都允许读取。
|
||||
|
||||
它原样复用本地后端配置:`cwd` 仍是相对路径的解析默认值,`diffBasisMaxBytes` 则限制可选的覆写上下文 diff 基础。
|
||||
|
||||
只需加载它来替代 `dsh-fs-local`,并同时加载 [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md),即可完成替换;面向模型的工具(`dsh-tool-fs`)无需改动。工具层把调用会话的模式和 cwd 解析为与 bash 相同的按调用策略,因此两个能力族绝不会约束到不同根目录。
|
||||
|
||||
## 围栏
|
||||
|
||||
@@ -41,10 +41,10 @@ import type {} from '@deepseek-ai/dsh-sandbox-policy'
|
||||
import { isPathUnder } from './containment.ts'
|
||||
|
||||
/**
|
||||
* Plugin config: the local backend's knobs, verbatim (only `cwd`, the resolve
|
||||
* base for relative paths). The sandbox default (mode + `workspace-write`
|
||||
* fallback root) is NOT here — `ctx.sandboxPolicy` resolves each calling
|
||||
* session for every enforcing capability.
|
||||
* Plugin config: the local backend's knobs verbatim (`cwd` resolution default
|
||||
* and `diffBasisMaxBytes` overwrite-presentation bound). The sandbox default
|
||||
* (mode + `workspace-write` fallback root) is NOT here — `ctx.sandboxPolicy`
|
||||
* resolves each calling session for every enforcing capability.
|
||||
*/
|
||||
export type Config = LocalConfig
|
||||
|
||||
|
||||
@@ -132,10 +132,11 @@ export interface FsWriteOutcome {
|
||||
version: FsVersion
|
||||
/**
|
||||
* The file's content BEFORE the write, or `null` when the file did not exist
|
||||
* (a create) or was undiffable (binary/non-UTF-8). LF-normalized storage text
|
||||
* (the diff basis), never a diff — a consumer computes the result-time
|
||||
* contextual diff from `before`/`after` when `before` is present, else falls
|
||||
* back to a whole-file diff.
|
||||
* (a create) or the backend declined a contextual basis (for example, a
|
||||
* binary/non-UTF-8 prior file or either overwrite side reaching its exclusive limit).
|
||||
* LF-normalized storage text (the diff basis), never a diff — a consumer
|
||||
* computes the result-time contextual diff from `before`/`after` when
|
||||
* `before` is present, else falls back to a whole-file diff.
|
||||
*/
|
||||
before: string | null
|
||||
/** The file's content AFTER the write, LF-normalized to share `before`'s diff basis. */
|
||||
|
||||
Reference in New Issue
Block a user