fix(fs): observe absence before guarded recreation

This commit is contained in:
Tianyi Cui
2026-08-09 15:22:50 +08:00
parent 9aa2d07353
commit ceba53edd7
68 changed files with 694 additions and 249 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md
README.md: ec0cffea1c82ab3bb2485fcb22ac5339d5d2a0d9
README.zh.md: 99adba8c8b58f1f0b6f0f5ab8b672eb074b38f31
README.md: ac2f271ad9651797e1aceeba49e7a52533405f10
README.zh.md: 633d55fe06b2261d9d5b52184987811c87c8f155

View File

@@ -19,7 +19,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence.
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing.
- **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`.
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
The package-root SDK surface is the default/named `LocalFileSystem` class plus `Config`. Raw I/O lives in `src/fsio.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the thin service wiring.
@@ -39,4 +39,4 @@ No direct invalidation; the named consumer owns any request-prefix changes.
- **Version tokens depend on filesystem metadata** — they combine device, inode, size, nanosecond mtime, and nanosecond ctime; a storage layer that cannot update any of those facts for a rewrite can still defeat the stale guard.
- **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path.
- **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits.
- **The per-target mutation lock is in-process only** — a writer in another process is caught only by the optional version guard, never serialized.
- **The per-target mutation lock is in-process only** — guarded create still uses an atomic no-replace publication across processes, but replacement writers in another process are caught only when the optional version guard observes their metadata change; they are never serialized.

View File

@@ -19,7 +19,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此飞行中的中止会报告 `FS_ABORTED`,而非陈旧的不存在结果。
- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片解码,因此超大文件无需整体保存在内存中,消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑。
- **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
- **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。
包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。
@@ -39,4 +39,4 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **版本 token 依赖文件系统元数据**:它们组合设备、inode、大小、纳秒级 mtime 和纳秒级 ctime;如果存储层在重写时无法更新其中任何一项事实,仍可能绕过陈旧防护。
- **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。
- **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。
- **每目标变更锁仅限进程内**:其他进程中的写入方只会被可选版本防护发现,绝不会被串行化。
- **每目标变更锁仅限进程内**:即使跨进程,带防护的创建仍采用原子且不替换的发布方式;但只有当可选版本防护观察到元数据变化时,系统才能发现其他进程中的替换写入方,且绝不会将其串行化。

View File

@@ -1,13 +1,13 @@
/**
* Cordis-free local filesystem mechanics. This provider layer returns validated UTF-8 text,
* streams large files, and rejects binary data; line windows belong to `dsh-tool-fs`. Writes
* stage an exclusive owner-only file in a private sibling directory and atomically rename it.
* stage an exclusive owner-only file in a private sibling directory and atomically publish it.
* @module @deepseek-ai/dsh-fs-local/fsio
*/
import { randomUUID } from 'node:crypto'
import { createReadStream } from 'node:fs'
import { chmod, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises'
import { chmod, link, lstat, mkdir, open, readFile, realpath, readdir, rename, rm, stat } from 'node:fs/promises'
import type { BigIntStats, Dirent, Stats } from 'node:fs'
import { basename, dirname, join, resolve } from 'node:path'
import { TextDecoder } from 'node:util'
@@ -20,6 +20,10 @@ function isENOENT(error: unknown): boolean {
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
}
function isEEXIST(error: unknown): boolean {
return error instanceof Error && 'code' in error && error.code === 'EEXIST'
}
/**
* A path component that is expected to be a directory is a regular file (e.g.
* resolving `afile/child.txt` when `afile` is a file). Like `ENOENT`, the target
@@ -85,7 +89,9 @@ export interface FsIoInternals {
copyFileDacl?: (source: string, destination: string) => Promise<void>
/** Override the Win32 security-preserving replacement boundary. */
replaceFile?: (replaced: string, replacement: string) => Promise<void>
/** Test hook after the temp file is written/synced but before final chmod+rename. */
/** Override the hard-link no-replace publication boundary. */
linkFile?: (existingPath: string, newPath: string) => Promise<void>
/** Test hook after the temp file is written/synced but before final chmod+publication. */
inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise<void>
}
@@ -426,8 +432,10 @@ async function removeStagingDirOrThrow(stagingDir: string, originalError: unknow
* @param content - the full UTF-8 text to write.
* @param mode - existing destination's POSIX mode to preserve, or `undefined` for a new file;
* inert as a mode on Windows but identifies replacement security semantics.
* @param signal - cancellation checked before the final rename.
* @param signal - cancellation checked before final publication.
* @param internals - Test hook for pinning temp names and observing the staged file.
* @param createIfAbsent - publish with a hard-link no-replace primitive; a
* concurrent creator is preserved and rejected with `FS_NOT_OBSERVED`.
*/
export async function writeFileAtomic(
absolutePath: string,
@@ -435,6 +443,7 @@ export async function writeFileAtomic(
mode: number | undefined,
signal: AbortSignal | undefined,
internals: FsIoInternals = {},
createIfAbsent = false,
): Promise<void> {
throwIfAborted(signal, 'write')
const directory = dirname(absolutePath)
@@ -448,6 +457,7 @@ export async function writeFileAtomic(
const platform = internals.platform ?? process.platform
const copyFileDacl = internals.copyFileDacl ?? copyFileDaclWin32
const replaceFile = internals.replaceFile ?? replaceFileWin32
const linkFile = internals.linkFile ?? link
let handle: Awaited<ReturnType<typeof open>> | undefined
let stagingCreated = false
try {
@@ -468,7 +478,18 @@ export async function writeFileAtomic(
handle = undefined
throwIfAborted(signal, 'write')
if (platform === 'win32' && mode !== undefined) {
if (createIfAbsent) {
try {
await linkFile(tempPath, absolutePath)
} catch (error: unknown) {
if (!isEEXIST(error)) throw error
throw new FsError(
`cannot overwrite existing "${absolutePath}" without reading it first`,
'FS_NOT_OBSERVED',
{ cause: error },
)
}
} else if (platform === 'win32' && mode !== undefined) {
try {
await replaceFile(absolutePath, tempPath)
} catch (error: unknown) {

View File

@@ -167,7 +167,14 @@ export class LocalFileSystem extends FileSystem {
// Preserve prior text for contextual diffs; null falls back to a whole-file diff.
// TODO(overwrite-diff-bound): cap this UI-only pre-read for large files.
const before = existing ? await readTextForDiff(target.targetKey, signal) : null
await writeFileAtomic(target.targetKey, content, existing?.mode, signal, this.internals)
await writeFileAtomic(
target.targetKey,
content,
existing?.mode,
signal,
this.internals,
expected?.kind === 'createIfAbsent',
)
const after = await probe(target.targetKey)
return {
operation: existing ? 'update' : 'create',

View File

@@ -298,6 +298,16 @@ describe('writeText', () => {
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('old')
})
it('createIfAbsent preserves a competitor created after the initial probe', async () => {
const path = join(dir, 'a.txt')
const target = await fs.resolve('a.txt')
fs.internals.inspectTemp = async () => { await writeFile(path, 'competitor') }
await expect(fs.writeText(target, 'ours', { kind: 'createIfAbsent' }))
.rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
expect(await readFile(path, 'utf8')).toBe('competitor')
})
it('replaceIfVersion replaces when the version matches', async () => {
await writeFile(join(dir, 'a.txt'), 'old')
const target = await fs.resolve('a.txt')

View File

@@ -496,6 +496,17 @@ describe('writeFileAtomic — temp-file safety', () => {
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it('surfaces a non-collision guarded-create publication failure and cleans staging', async () => {
const file = join(dir, 'a.txt')
const denied = Object.assign(new Error('link denied'), { code: 'EACCES' })
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
linkFile: async () => { throw denied },
}, true)).rejects.toBe(denied)
await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' })
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it.skipIf(!posixModes)('creates new files owner-only by default', async () => {
const file = join(dir, 'a.txt')
await writeFileAtomic(file, 'hello', undefined, undefined)