fix(settings-local): never steal writer locks

This commit is contained in:
Tianyi Cui
2026-07-31 01:14:43 +08:00
parent f9f8148e79
commit afb05b4049
9 changed files with 36 additions and 75 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/settings/settings-local/README.md
README.md: 2c0817afd2f2fd35fda2d22cd7f7ef3772fe2257
README.zh.md: 547abb035368f07d4478a5c3a1793cdaa6743c68
README.md: 39c2254caac720149d2fbf04d067e6154b82a671
README.zh.md: 9fcc1319a35d56a965eb756737175dc89518c0e5

View File

@@ -19,7 +19,7 @@ Defaulting is one explicit `resolveSpec(config)` step; an unsupported extension
- **Boot fails loud, reload keeps last-good.** An existing-but-invalid document fails plugin load; once live, an unreadable or unparsable edit warns and keeps the last good sections. A missing document resolves every namespace from defaults and `base`; deleting it publishes the same empty state.
- **Every write is a read-modify-write.** A persist first re-reads the document and publishes any difference into the seam — an external edit still inside the watcher debounce window, a change the watcher missed, or another process's write — then renders against that fresh text, so a write can never resurrect a stale document or drop an unobserved sibling section. If the on-disk document turned invalid, the write rejects loud instead of overwriting the user's manual edit.
- **Writes hold a cross-process writer lock.** The read-render-rename cycle runs under a `wx`-created `<file>.lock` sibling with exponential backoff, a 2 s acquisition deadline (the write rejects), and stale-lock takeover after 5 s (a crashed holder, broken with a warning). Readers never take the lock: the rename commit is atomic, so reloads are always consistent.
- **Writes hold a cross-process writer lock.** The read-render-rename cycle runs under a `wx`-created `<file>.lock` sibling with exponential backoff and a 2 s acquisition deadline. A contender never removes a lock it does not own; it rejects at the deadline instead. Readers never take the lock: the rename commit is atomic, so reloads are always consistent.
- **Write-back is atomic, owner-only, and symlink-proof.** The render exclusive-creates a random-suffix temp sibling with mode `0600` (`wx` refuses to follow a planted symlink) and renames over the target, cleaning the temp up on failure.
- **YAML edits are leaf-level diffs.** A write sets only the values that changed and deletes only the keys that were removed, so comments, anchors, and formatting survive on every untouched node and on the key of every changed pair; a changed array (or other non-map value) replaces wholesale, taking comments inside it along. JSON re-serializes without comments.
- **Reloads and writes share one operation chain.** Watcher refreshes and persists from every namespace queue run one at a time in queue order; each render sees the text the previous operation committed.
@@ -38,6 +38,7 @@ No direct invalidation; the consuming plugin owns any request-prefix changes.
## Known Limitations and Deferred Work
- **Same-namespace conflicts stay last-write-wins** — the writer lock and read-modify-write keep concurrent writers from dropping each other's namespaces, but two writers editing one namespace still resolve to the later write; there is no per-value merge or revision check.
- **An abandoned writer lock requires operator recovery** — lock age cannot prove ownership, so writers fail closed after 2 s instead of deleting an old lock that may still protect a slow holder; remove `<file>.lock` only after establishing that no writer owns it.
- **A missed watcher event stays unseen until the next signal** — reads never re-stat the file, so a change the watcher fails to report is only folded in by the next event, the next write, or a restart.
- **Comment preservation is YAML-only and map-shaped** — JSON documents re-serialize without comments (JSON has none), and comments inside a changed array (or attached inline to a changed scalar value) go with the value they described.
- **No value indirection** — sections hold literal values; `${env:VAR}`-style references for secrets are a deferred seam-level feature.

View File

@@ -19,7 +19,7 @@
- **启动报错响亮,重载保留最后可用值。** 存在但非法的文档使插件加载失败;运行中不可读或不可解析的编辑只告警并保留最后可用分节。文档缺失时所有 namespace 按默认值与 `base` 解析;删除文档发布同样的空状态。
- **每次写入都是一次读-改-写。** persist 先重读文档并把任何差异发布进 seam——无论是仍在 watcher 防抖窗口内的外部编辑、watcher 漏掉的变更,还是另一个进程的写入——再基于这份新鲜文本渲染,因此写入绝不会复活陈旧文档,也不会丢掉未观察到的同级分节。若磁盘上的文档已变为非法,写入响亮拒绝,而不是覆盖用户的手工编辑。
- **写入持有跨进程写锁。** 读-渲染-rename 流程在 `wx` 创建的 `<file>.lock` 同级文件下运行,带指数退避2 s 的获取期限(到期则写入拒绝)与 5 s 后的陈旧锁接管(持有者已崩溃,破锁并告警)。读取方从不取锁rename 提交是原子的,重载因此始终一致。
- **写入持有跨进程写锁。** 读-渲染-rename 流程在 `wx` 创建的 `<file>.lock` 同级文件下运行,带指数退避2 s 的获取期限。竞争者绝不移除不归自己所有的锁,而会在期限到达时拒绝写入。读取方从不取锁rename 提交是原子的,重载因此始终一致。
- **写回原子、仅属主可读、抗符号链接。** 渲染以 `0600` 权限独占创建随机后缀临时同级文件(`wx` 拒绝跟随预埋符号链接)后 rename 覆盖目标,失败时清理临时文件。
- **YAML 编辑是叶子级 diff。** 写入只设置发生变化的值、只删除被移除的键,因此注释、锚点与排版在每个未触碰的节点上以及每个被改键值对的键上都得以保留;被改的数组(或其他非 map 值整体替换其中的注释随之一同被换掉。JSON 重新序列化,无注释。
- **重载与写入共享一条操作链。** watcher 刷新与来自各 namespace 队列的 persist 按队列顺序逐个执行;每次渲染都基于上一次操作提交后的文本。
@@ -38,6 +38,7 @@
## Known Limitations and Deferred Work
- **同 namespace 冲突仍是后写胜出** — 写锁加读-改-写让并发写入者不会丢掉彼此的 namespace但两个写入者编辑同一个 namespace 时仍以较后的写入为准;没有按值合并,也没有修订检查。
- **遗留的写锁需要操作者恢复** — 锁的存续时间无法证明所有权,因此写入方会在 2 s 后以失败收口,不会删除一把可能仍在保护慢速持有者的旧锁;只有确认没有写入方拥有 `<file>.lock` 后才能将其移除。
- **漏掉的 watcher 事件在下一个信号前保持不可见** — 读取从不重新 stat 文件,因此 watcher 漏报的变更只会在下一个事件、下一次写入或重启时被并入。
- **注释保留仅限 YAML 且仅限 map 形状** — JSON 文档重新序列化无注释JSON 本身没有),且被改数组内部的注释(或行内附着在被改标量值上的注释)随其所描述的值一同被换掉。
- **无值间接引用** — 分节存字面值;面向密钥的 `${env:VAR}` 式引用是 seam 层的延后特性。

View File

@@ -11,7 +11,7 @@ import { Context, Service } from 'cordis'
import z from 'schemastery'
import { watch as chokidarWatch } from 'chokidar'
import { randomBytes } from 'node:crypto'
import { mkdir, readFile, rename, rm, stat, writeFile } from 'node:fs/promises'
import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'
import { dirname, extname, join, resolve } from 'node:path'
import { Document, parseDocument } from 'yaml'
import { resolveDshHome } from '@deepseek-ai/dsh-paths'
@@ -105,7 +105,6 @@ function isEEXIST(error: unknown): boolean {
const LOCK_RETRY_INITIAL_MS = 20
const LOCK_RETRY_MAX_MS = 200
const LOCK_TIMEOUT_MS = 2_000
const LOCK_STALE_MS = 5_000
/** File-backed settings provider (`settings.yaml`/`.json`). */
export class SettingsLocal extends Settings {
@@ -229,15 +228,6 @@ export class SettingsLocal extends Settings {
} catch (error) {
if (!isEEXIST(error)) throw error
}
const ageMs = await this.lockAgeMs(lockPath)
if (ageMs === undefined) continue
if (ageMs > LOCK_STALE_MS) {
// TODO(settings-lock-ownership): Replace age-only takeover with ownership-safe
// acquisition and release so a slow writer cannot remove a successor's lock.
this.ctx.logger.warn('settings-local: breaking a stale writer lock at %s', lockPath)
await rm(lockPath, { force: true })
continue
}
if (Date.now() >= deadline) {
throw new Error(`settings-local: timed out waiting for the writer lock at ${lockPath}`)
}
@@ -251,16 +241,6 @@ export class SettingsLocal extends Settings {
}
}
/** Age of the writer lock, or `undefined` when it vanished after a failed create. */
private async lockAgeMs(lockPath: string): Promise<number | undefined> {
try {
return Date.now() - (await stat(lockPath)).mtimeMs
} catch (error) {
if (!isENOENT(error)) throw error
return undefined
}
}
override async* [Service.init](): AsyncGenerator<() => Promise<void> | void, void, void> {
// The base init loads and publishes; a parse failure there is a boot
// failure: an existing-but-invalid document must fail loud, never be

View File

@@ -70,25 +70,20 @@ describe('writer lock', () => {
expect(await readFile(path, 'utf8')).toContain('value: 7')
})
it('breaks a stale writer lock with a warning and writes through', async () => {
it('does not steal an old writer lock', async () => {
const dir = await tempDir()
const path = join(dir, 'settings.yaml')
await writeFile(path, 'alpha:\n value: 4\n')
const ctx = await boot({ path, watch: false })
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
await writeFile(`${path}.lock`, 'crashed-holder\n')
const lockPath = `${path}.lock`
await writeFile(lockPath, 'slow-holder\n')
const past = (Date.now() - 60_000) / 1000
await utimes(`${path}.lock`, past, past)
await scope.update({ value: 9 })
expect(await readFile(path, 'utf8')).toContain('value: 9')
})
await utimes(lockPath, past, past)
it('times out on a lock a live holder never releases', async () => {
const dir = await tempDir()
const path = join(dir, 'settings.yaml')
const ctx = await boot({ path, watch: false })
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
await writeFile(`${path}.lock`, 'busy-holder\n')
await expect(scope.update({ value: 1 })).rejects.toThrow(/timed out waiting for the writer lock/)
await expect(scope.update({ value: 9 })).rejects.toThrow(/timed out waiting for the writer lock/)
expect(await readFile(path, 'utf8')).toContain('value: 4')
expect(await readFile(lockPath, 'utf8')).toBe('slow-holder\n')
}, 10_000)
it('surfaces a non-contention lock failure as the write error', async () => {

View File

@@ -1,6 +1,6 @@
// Writer-lock races that cannot be timed from outside: a contender whose lock
// vanishes between the failed exclusive create and the stat, a stat failing
// for a reason other than absence, and a temp-file write failing mid-cycle.
// vanishes after the failed exclusive create and a temp-file write failing
// mid-cycle.
// The fs/promises seam is partially mocked to inject exactly one failure at a
// chosen path suffix; everything else passes through to the real filesystem.
import { afterEach, describe, expect, it, vi } from 'vitest'
@@ -14,27 +14,23 @@ import { SettingsLocal } from '../src/index.ts'
const state = vi.hoisted(() => ({
/** One-shot failure injections keyed by operation, matched on a path suffix. */
failures: [] as Array<{ op: 'writeFile' | 'stat'; suffix: string; code: string }>,
failures: [] as Array<{ suffix: string; code: string }>,
}))
vi.mock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs/promises')>()
const inject = (op: 'writeFile' | 'stat', path: unknown): void => {
const index = state.failures.findIndex(f => f.op === op && String(path).endsWith(f.suffix))
const inject = (path: unknown): void => {
const index = state.failures.findIndex(f => String(path).endsWith(f.suffix))
if (index === -1) return
const [failure] = state.failures.splice(index, 1)
throw Object.assign(new Error(`${failure!.code}: injected ${op} failure`), { code: failure!.code })
throw Object.assign(new Error(`${failure!.code}: injected writeFile failure`), { code: failure!.code })
}
return {
...actual,
writeFile: (async (path: unknown, ...rest: never[]) => {
inject('writeFile', path)
inject(path)
return (actual.writeFile as (path: unknown, ...args: never[]) => Promise<void>)(path, ...rest)
}) as typeof actual.writeFile,
stat: (async (path: unknown, ...rest: never[]) => {
inject('stat', path)
return (actual.stat as (path: unknown, ...args: never[]) => Promise<unknown>)(path, ...rest)
}) as typeof actual.stat,
}
})
@@ -62,36 +58,24 @@ async function boot(config: ConstructorParameters<typeof SettingsLocal>[1]): Pro
}
describe('writer-lock races', () => {
it('retries immediately when the contending lock vanished before the stat', async () => {
it('retries when the contending lock vanished after the failed create', async () => {
const dir = await tempDir()
const path = join(dir, 'settings.yaml')
const ctx = await boot({ path, watch: false })
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
// The exclusive create loses to a holder that releases before the stat:
// no lock file actually exists, so the stat sees honest absence and the
// very next attempt takes the lock.
state.failures.push({ op: 'writeFile', suffix: '.lock', code: 'EEXIST' })
// The exclusive create loses once, but no lock remains by the retry.
state.failures.push({ suffix: '.lock', code: 'EEXIST' })
await scope.update({ value: 3 })
expect(await readFile(path, 'utf8')).toContain('value: 3')
})
it('propagates a stat failure that does not mean absence', async () => {
const dir = await tempDir()
const path = join(dir, 'settings.yaml')
const ctx = await boot({ path, watch: false })
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
state.failures.push({ op: 'writeFile', suffix: '.lock', code: 'EEXIST' })
state.failures.push({ op: 'stat', suffix: '.lock', code: 'EACCES' })
await expect(scope.update({ value: 3 })).rejects.toThrow(/EACCES/)
})
it('cleans up the temp file and releases the lock when the write fails mid-cycle', async () => {
const dir = await tempDir()
const path = join(dir, 'settings.yaml')
await writeFile(path, 'alpha:\n value: 1\n')
const ctx = await boot({ path, watch: false })
const scope = ctx.settings.register(settingsNamespace('alpha'), AlphaSchema)
state.failures.push({ op: 'writeFile', suffix: '.tmp', code: 'ENOSPC' })
state.failures.push({ suffix: '.tmp', code: 'ENOSPC' })
await expect(scope.update({ value: 9 })).rejects.toThrow(/ENOSPC/)
// The document is untouched and the writer lock was released on the way out.
expect(await readFile(path, 'utf8')).toContain('value: 1')