Merge remote-tracking branch 'origin/master' into worktree/semantic-session-checkpoints
# Conflicts: # packages/session-persistence/session-persistence-jsonl/README.md
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
# Agent Note: Windows write-permission semantics — inherited DACLs, not mode bits
|
||||
|
||||
Status: implemented
|
||||
|
||||
The replacement-file decision in this record is superseded by [Windows DACL preservation](../bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md).
|
||||
|
||||
## Problem
|
||||
|
||||
`writeFileAtomic` in `@deepseek-ai/dsh-fs-local` protects write-in-progress content with POSIX mode bits: the staging directory is created `0o700`, the temp file is opened `0o600`, and new files default to `0o600`. On POSIX this keeps temporary content owner-only regardless of the parent directory's permissions.
|
||||
|
||||
Windows has no working equivalent behind the same API. Node's `chmod` there drives only the read-only attribute (every mode this package passes carries owner-write, so the calls are benign no-ops), and `stat().mode` reports synthetic `0o666`/`0o444` bits. The real security state is the file's DACL: a newly created file or directory inherits from its parent, while replacement needs the explicit handling owned by the superseding Agent Note.
|
||||
|
||||
## Decision
|
||||
|
||||
New Windows files use directory inheritance rather than synthetic mode bits: the staging directory is created inside the target's parent directory (`dirname(absolutePath)`), so it and the temp file inherit the destination directory's DACL. Replacement files follow the stricter [DACL preservation contract](../bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md).
|
||||
|
||||
Tests assert mode bits on POSIX only. Native Windows coverage pins the package-owned replacement behavior; new-file inheritance remains an operating-system contract rather than a machine-specific ACL allowlist.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Explicit owner-only DACLs for new files.** Rejected because they would break inheritance and surprise users whose project directories are deliberately shared. Replacement writes copy the target's existing DACL rather than inventing an owner-only policy.
|
||||
|
||||
**Test-side ACL verification.** A `Get-Acl` SID allowlist or `icacls` would verify Windows inheritance and the machine's `%TEMP%` ACL rather than package behavior; `icacls` also localizes well-known account names, making parsing locale-fragile.
|
||||
|
||||
**Skip `chmod` on Windows.** Platform-guarding benign no-op calls adds branches without changing behavior.
|
||||
|
||||
## Consequences
|
||||
|
||||
POSIX keeps owner-only temp content regardless of the parent directory. A new Windows target inside a broadly accessible directory inherits that accessibility by design; a replacement retains the target's narrower DACL when one exists.
|
||||
|
||||
Mode preservation across a replace degenerates to a no-op on Windows: a writable file probes as `0o666`, and replaying that through `chmod` leaves the read-only attribute clear. A read-only target cannot be replaced there because publication fails before the synthetic mode would matter.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Agent Note: Windows-native durable JSONL publication
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-session-persistence-jsonl` publishes a session log lazily on the first append. The POSIX protocol writes a temp file, fsyncs it, links it to the final name, fsyncs the parent directory, and then removes the temp link. The parent-directory fsync is part of the durability contract: a crash after the namespace change must not lose the committed final name while leaving callers believing the session log materialized.
|
||||
|
||||
Windows has atomic namespace operations, but Node does not expose a POSIX-equivalent parent-directory fsync contract there. Treating Windows directory sync failures as success would silently weaken a durable backend. The Windows path therefore needs a different publication primitive rather than a conditional inside the POSIX `syncDir` helper.
|
||||
|
||||
## Decision
|
||||
|
||||
The JSONL backend forks inside `materialize()` before any namespace mutation. Shared code computes the session directory, final log path, and encoded header plus initial event batch; POSIX and Windows then run separate publication protocols.
|
||||
|
||||
POSIX keeps the existing protocol: create the root and cwd bucket with parent directory fsyncs, write and fsync a temp file, publish with `link()` so an existing final log is never overwritten, fsync the bucket directory, then remove the redundant temp hard link.
|
||||
|
||||
Windows creates missing directories through a durable staging publish: create a random sibling directory, then publish it to the final directory name with `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` without `MOVEFILE_REPLACE_EXISTING` or `MOVEFILE_COPY_ALLOWED`. File materialization writes and fsyncs the temp log, then publishes that temp file to the final path with the same write-through `MoveFileExW` call and no replacement. `koffi` is the minimal Win32 bridge for this API surface; its install script is allowed in `pnpm-workspace.yaml` because the package ships the native loader and prebuilt platform modules.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Ignore Windows directory-sync failures.** Rejected because it reports a first append as durable without forcing the published namespace entry to stable storage.
|
||||
|
||||
**Use `CreateHardLinkW`.** Rejected because hard links are filesystem-dependent, do not publish directories, and expose no write-through option.
|
||||
|
||||
**Use replacement or transactional APIs.** `ReplaceFileW` has replacement semantics that conflict with same-id collision rejection, and Transactional NTFS is not recommended for new application designs.
|
||||
|
||||
## Consequences
|
||||
|
||||
The backend keeps one external contract across platforms: first append either publishes a complete log at the final name or fails without overwriting an existing log. The platform split is an implementation detail; `SessionPersistence` APIs and the logical JSONL record format do not change. The later [Zstandard encoding decision](2026-07-19-zstandard-jsonl-session-logs.md) applies before either platform publishes the opaque bytes.
|
||||
|
||||
Windows tests exercise the real Win32 publish path on native Windows. Power-loss behavior remains an API-contract property rather than something unit tests can prove; the testable invariants are that directory fsync is not called on Windows materialization, final-path collisions fail, temp logs are fsync'd before publication, and the resulting log loads normally.
|
||||
|
||||
Append and repair still use ordinary file-handle fsyncs on both platforms. A failed append closes its append-only handle, reopens the log read/write, truncates it to the pre-append size, and fsyncs the rollback because Windows rejects `ftruncate` on append-only handles.
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-zstandard-jsonl-session-logs.md: 09d30594fe31eed138a128dabc1947b15857808d
|
||||
2026-07-19-zstandard-jsonl-session-logs.zh.md: 131531d9dba7cb01407191bf937f8b0ee3c6860a
|
||||
2026-07-19-zstandard-jsonl-session-logs.md: ccfc81dd47504e6a9e9b19cda7c4b9fc40accecc
|
||||
2026-07-19-zstandard-jsonl-session-logs.zh.md: de5436a6eaefcb45e52e0ff4fea8592c7efcd127
|
||||
|
||||
@@ -24,7 +24,7 @@ The compressed artifact is a standard concatenation of independent [Zstandard fr
|
||||
|
||||
Compression uses Node's built-in [`zstdCompress` and `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html), available at the repository's Node 22.19 floor. The backend enables `ZSTD_c_checksumFlag`, otherwise accepts Node's defaults, and exposes neither a compression-level knob nor a new dependency. The API is marked experimental by Node, so the Node 22.19, 24, and 26 compatibility gate exercises the exact helper.
|
||||
|
||||
First materialization compresses the two initial frames before opening the temporary file, then keeps the existing write, file `fsync`, collision-safe hard-link publication, and directory `fsync` sequence. Later batches are compressed before opening the destination and appended at EOF. A caught write or file-sync failure truncates to the prior byte length, syncs the rollback, and rethrows so the coordinator can retry the unchanged batch.
|
||||
First materialization compresses the two initial frames before opening the temporary file, then writes and `fsync`s that file. POSIX publishes it through a collision-safe hard link and directory `fsync`; Windows publishes it without replacement through `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)`. Later batches are compressed before opening the destination and appended at EOF. A caught write or file-sync failure closes the append handle, reopens the log read/write, truncates to the prior byte length, syncs the rollback, and rethrows so the coordinator can retry the unchanged batch on both platforms.
|
||||
|
||||
### Read, listing, and crash recovery
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ JSONL 持久化后端会逐字保留每个 `SessionEvent`,其中包括数量
|
||||
|
||||
压缩使用 Node 内置的 [`zstdCompress` 与 `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html),仓库最低支持的 Node 22.19 已提供这些 API。后端启用 `ZSTD_c_checksumFlag`,其余采用 Node 默认值,不公开压缩级别调节项,也不增加依赖。Node 将该 API 标记为实验性,因此 Node 22.19、24 与 26 兼容性门禁会执行同一个辅助实现。
|
||||
|
||||
首次物化会在打开临时文件之前压缩两个初始帧,然后保留既有的写入、文件 `fsync`、避免冲突的硬链接发布与目录 `fsync` 顺序。后续批次也会先压缩,再打开目标并在 EOF 追加。捕获到写入或文件同步失败时,后端会截断到原有字节长度,同步回滚结果,再重新抛出错误,让协调器重试未变化的批次。
|
||||
首次物化会在打开临时文件之前压缩两个初始帧,然后写入该文件并执行 `fsync`。POSIX 通过避免冲突的硬链接和目录 `fsync` 发布该文件;Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 在不替换目标文件的情况下发布。后续批次也会先压缩,再打开目标并在 EOF 追加。捕获到写入或文件同步失败时,后端会关闭追加句柄,以读写方式重新打开日志,截断到原有字节长度,同步回滚结果,再重新抛出错误,让协调器能够在两个平台上重试未变化的批次。
|
||||
|
||||
### 读取、列举与崩溃恢复
|
||||
|
||||
|
||||
Reference in New Issue
Block a user