Merge commit 'refs/private/gh-unblock-approved-prs/master' into HEAD

# Conflicts:
#	packages/session-persistence/session-persistence-jsonl/README.md
#	packages/session-persistence/session-persistence-jsonl/src/index.ts
This commit is contained in:
Tianyi Cui
2026-07-22 10:35:08 +08:00
71 changed files with 2447 additions and 312 deletions

View File

@@ -0,0 +1,31 @@
# Agent Note: Windows write-permission semantics — inherited DACLs, not mode bits
Status: implemented
The replacement-file decision in this record is superseded by [Windows DACL preservation](../bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md).
## Problem
`writeFileAtomic` in `@deepseek-ai/dsh-fs-local` protects write-in-progress content with POSIX mode bits: the staging directory is created `0o700`, the temp file is opened `0o600`, and new files default to `0o600`. On POSIX this keeps temporary content owner-only regardless of the parent directory's permissions.
Windows has no working equivalent behind the same API. Node's `chmod` there drives only the read-only attribute (every mode this package passes carries owner-write, so the calls are benign no-ops), and `stat().mode` reports synthetic `0o666`/`0o444` bits. The real security state is the file's DACL: a newly created file or directory inherits from its parent, while replacement needs the explicit handling owned by the superseding Agent Note.
## Decision
New Windows files use directory inheritance rather than synthetic mode bits: the staging directory is created inside the target's parent directory (`dirname(absolutePath)`), so it and the temp file inherit the destination directory's DACL. Replacement files follow the stricter [DACL preservation contract](../bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md).
Tests assert mode bits on POSIX only. Native Windows coverage pins the package-owned replacement behavior; new-file inheritance remains an operating-system contract rather than a machine-specific ACL allowlist.
## Alternatives considered
**Explicit owner-only DACLs for new files.** Rejected because they would break inheritance and surprise users whose project directories are deliberately shared. Replacement writes copy the target's existing DACL rather than inventing an owner-only policy.
**Test-side ACL verification.** A `Get-Acl` SID allowlist or `icacls` would verify Windows inheritance and the machine's `%TEMP%` ACL rather than package behavior; `icacls` also localizes well-known account names, making parsing locale-fragile.
**Skip `chmod` on Windows.** Platform-guarding benign no-op calls adds branches without changing behavior.
## Consequences
POSIX keeps owner-only temp content regardless of the parent directory. A new Windows target inside a broadly accessible directory inherits that accessibility by design; a replacement retains the target's narrower DACL when one exists.
Mode preservation across a replace degenerates to a no-op on Windows: a writable file probes as `0o666`, and replaying that through `chmod` leaves the read-only attribute clear. A read-only target cannot be replaced there because publication fails before the synthetic mode would matter.

View File

@@ -0,0 +1,33 @@
# Agent Note: Windows-native durable JSONL publication
Status: implemented
## Problem
`dsh-session-persistence-jsonl` publishes a session log lazily on the first append. The POSIX protocol writes a temp file, fsyncs it, links it to the final name, fsyncs the parent directory, and then removes the temp link. The parent-directory fsync is part of the durability contract: a crash after the namespace change must not lose the committed final name while leaving callers believing the session log materialized.
Windows has atomic namespace operations, but Node does not expose a POSIX-equivalent parent-directory fsync contract there. Treating Windows directory sync failures as success would silently weaken a durable backend. The Windows path therefore needs a different publication primitive rather than a conditional inside the POSIX `syncDir` helper.
## Decision
The JSONL backend forks inside `materialize()` before any namespace mutation. Shared code computes the session directory, final log path, and encoded header plus initial event batch; POSIX and Windows then run separate publication protocols.
POSIX keeps the existing protocol: create the root and cwd bucket with parent directory fsyncs, write and fsync a temp file, publish with `link()` so an existing final log is never overwritten, fsync the bucket directory, then remove the redundant temp hard link.
Windows creates missing directories through a durable staging publish: create a random sibling directory, then publish it to the final directory name with `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` without `MOVEFILE_REPLACE_EXISTING` or `MOVEFILE_COPY_ALLOWED`. File materialization writes and fsyncs the temp log, then publishes that temp file to the final path with the same write-through `MoveFileExW` call and no replacement. `koffi` is the minimal Win32 bridge for this API surface; its install script is allowed in `pnpm-workspace.yaml` because the package ships the native loader and prebuilt platform modules.
## Alternatives considered
**Ignore Windows directory-sync failures.** Rejected because it reports a first append as durable without forcing the published namespace entry to stable storage.
**Use `CreateHardLinkW`.** Rejected because hard links are filesystem-dependent, do not publish directories, and expose no write-through option.
**Use replacement or transactional APIs.** `ReplaceFileW` has replacement semantics that conflict with same-id collision rejection, and Transactional NTFS is not recommended for new application designs.
## Consequences
The backend keeps one external contract across platforms: first append either publishes a complete log at the final name or fails without overwriting an existing log. The platform split is an implementation detail; `SessionPersistence` APIs and the logical JSONL record format do not change. The later [Zstandard encoding decision](2026-07-19-zstandard-jsonl-session-logs.md) applies before either platform publishes the opaque bytes.
Windows tests exercise the real Win32 publish path on native Windows. Power-loss behavior remains an API-contract property rather than something unit tests can prove; the testable invariants are that directory fsync is not called on Windows materialization, final-path collisions fail, temp logs are fsync'd before publication, and the resulting log loads normally.
Append and repair still use ordinary file-handle fsyncs on both platforms. A failed append closes its append-only handle, reopens the log read/write, truncates it to the pre-append size, and fsyncs the rollback because Windows rejects `ftruncate` on append-only handles.

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-19-zstandard-jsonl-session-logs.md: 09d30594fe31eed138a128dabc1947b15857808d
2026-07-19-zstandard-jsonl-session-logs.zh.md: 131531d9dba7cb01407191bf937f8b0ee3c6860a
2026-07-19-zstandard-jsonl-session-logs.md: ccfc81dd47504e6a9e9b19cda7c4b9fc40accecc
2026-07-19-zstandard-jsonl-session-logs.zh.md: de5436a6eaefcb45e52e0ff4fea8592c7efcd127

View File

@@ -24,7 +24,7 @@ The compressed artifact is a standard concatenation of independent [Zstandard fr
Compression uses Node's built-in [`zstdCompress` and `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html), available at the repository's Node 22.19 floor. The backend enables `ZSTD_c_checksumFlag`, otherwise accepts Node's defaults, and exposes neither a compression-level knob nor a new dependency. The API is marked experimental by Node, so the Node 22.19, 24, and 26 compatibility gate exercises the exact helper.
First materialization compresses the two initial frames before opening the temporary file, then keeps the existing write, file `fsync`, collision-safe hard-link publication, and directory `fsync` sequence. Later batches are compressed before opening the destination and appended at EOF. A caught write or file-sync failure truncates to the prior byte length, syncs the rollback, and rethrows so the coordinator can retry the unchanged batch.
First materialization compresses the two initial frames before opening the temporary file, then writes and `fsync`s that file. POSIX publishes it through a collision-safe hard link and directory `fsync`; Windows publishes it without replacement through `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)`. Later batches are compressed before opening the destination and appended at EOF. A caught write or file-sync failure closes the append handle, reopens the log read/write, truncates to the prior byte length, syncs the rollback, and rethrows so the coordinator can retry the unchanged batch on both platforms.
### Read, listing, and crash recovery

View File

@@ -24,7 +24,7 @@ JSONL 持久化后端会逐字保留每个 `SessionEvent`,其中包括数量
压缩使用 Node 内置的 [`zstdCompress` 与 `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html),仓库最低支持的 Node 22.19 已提供这些 API。后端启用 `ZSTD_c_checksumFlag`,其余采用 Node 默认值,不公开压缩级别调节项,也不增加依赖。Node 将该 API 标记为实验性,因此 Node 22.19、24 与 26 兼容性门禁会执行同一个辅助实现。
首次物化会在打开临时文件之前压缩两个初始帧,然后保留既有的写入、文件 `fsync`、避免冲突的硬链接发布与目录 `fsync` 顺序。后续批次也会先压缩,再打开目标并在 EOF 追加。捕获到写入或文件同步失败时,后端会截断到原有字节长度,同步回滚结果,再重新抛出错误,让协调器重试未变化的批次。
首次物化会在打开临时文件之前压缩两个初始帧,然后写入该文件并执行 `fsync`。POSIX 通过避免冲突的硬链接和目录 `fsync` 发布该文件;Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 在不替换目标文件的情况下发布。后续批次也会先压缩,再打开目标并在 EOF 追加。捕获到写入或文件同步失败时,后端会关闭追加句柄,以读写方式重新打开日志,截断到原有字节长度,同步回滚结果,再重新抛出错误,让协调器能够在两个平台上重试未变化的批次。
### 读取、列举与崩溃恢复