Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(persistence): reconcile project session layout

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-25 15:11:17 +08:00
parent c09397f833
commit 8a7bb03aab
19 changed files with 41 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-06-18-shared-persistence-write-coordinator.md: ea9c4fb74f7c1bd68fb62efedd3e1657da96ea65
2026-06-18-shared-persistence-write-coordinator.zh.md: 3b4dd7b762c2f39a908eabe23e5d734981b5767b
2026-06-18-shared-persistence-write-coordinator.md: 4632351a6f39c44c9ba8af58d508d4665b9e9279
2026-06-18-shared-persistence-write-coordinator.zh.md: 40a7144038ac0db4ca6cac651c0a3cef5de4afa9

2
.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md
View File

@@ -23,7 +23,7 @@ The coordinator retires a session from `session/disposed`: it waits for the cont
Five required members plus an optional lifecycle hook form the only boundary between the coordinator and storage:
- `name` — backend label for the dispose-failure `AggregateError`.
- `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL cwd bucket; SQLite's id is globally unique). Resume/load, non-mutating inspection, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication.
- `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL project directory; SQLite's id is globally unique). Resume/load, non-mutating inspection, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication.
- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook).
- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`).
- `list()` — list all stored metadata.

2
.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md
View File

@@ -23,7 +23,7 @@ Status: implemented
五个必需成员加一个可选的生命周期钩子，构成协调器与存储之间唯一的边界：
- `name`——后端标签，用于 dispose 失败时的 `AggregateError`。
- `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀（JSONL 的所有 cwd bucket；SQLite 的 id 全局唯一）。恢复/加载、不修改状态的检查、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id，并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。
- `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀（JSONL 的所有项目目录；SQLite 的 id 全局唯一）。恢复/加载、不修改状态的检查、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id，并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。
- `appendBatch(meta, events, isMaterialized)`——持久追加一个连续批次，在尚未物化时原子地惰性物化会话（物化写入与首批事件必须一起提交——崩溃不得留下一个已物化但为空的会话；这就是为什么没有单独的 `materialize` 钩子）。
- `commitRepair(meta, tornMarker, closers)`——使崩溃修复持久化：截断损坏的尾部（当且仅当 `tornMarker !== undefined`）并追加 `closers`。**不要求原子性**——JSONL 合理地分两步 fsync（先截断再追加），SQLite 在一个事务中完成 DELETE+INSERT。用于 `load`（截断 + 合成 closers）和 live-adoption（仅截断，`closers = []`）。
- `list()`——列出所有已存储的元数据。

6
.agents/notes/implemented/architecture/2026-07-05-windows-jsonl-durable-publish.md
View File

@@ -12,9 +12,9 @@ Windows has atomic namespace operations, but Node does not expose a POSIX-equiva
The JSONL backend forks inside `materialize()` before any namespace mutation. Shared code computes the session directory, final log path, and encoded header plus initial event batch; POSIX and Windows then run separate publication protocols.
POSIX keeps the existing protocol: create the root and cwd bucket with parent directory fsyncs, write and fsync a temp file, publish with `link()` so an existing final log is never overwritten, fsync the bucket directory, then remove the redundant temp hard link.
POSIX keeps the existing protocol: create the root, project directory, and session directory with parent directory fsyncs, write and fsync a temp file, publish with `link()` so an existing final log is never overwritten, fsync the session directory, then remove the redundant temp hard link.
Windows creates missing directories through a durable staging publish: create a random sibling directory, then publish it to the final directory name with `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` without `MOVEFILE_REPLACE_EXISTING` or `MOVEFILE_COPY_ALLOWED`. File materialization writes and fsyncs the temp log, then publishes that temp file to the final path with the same write-through `MoveFileExW` call and no replacement. `koffi` is the minimal Win32 bridge for this API surface; its install script is allowed in `pnpm-workspace.yaml` because the package ships the native loader and prebuilt platform modules.
Windows creates missing directories through a durable staging publish: create a random sibling directory under the constant `.dsh-mkdir-` prefix, independent of the target basename, then publish it to the final directory name with `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` without `MOVEFILE_REPLACE_EXISTING` or `MOVEFILE_COPY_ALLOWED`. File materialization writes and fsyncs the temp log, then publishes that temp file to the final path with the same write-through `MoveFileExW` call and no replacement. `koffi` is the minimal Win32 bridge for this API surface; its install script is allowed in `pnpm-workspace.yaml` because the package ships the native loader and prebuilt platform modules.
## Alternatives considered
@@ -28,6 +28,6 @@ Windows creates missing directories through a durable staging publish: create a
The backend keeps one external contract across platforms: first append either publishes a complete log at the final name or fails without overwriting an existing log. The platform split is an implementation detail; `SessionPersistence` APIs and the logical JSONL record format do not change. The later [Zstandard encoding decision](2026-07-19-zstandard-jsonl-session-logs.md) applies before either platform publishes the opaque bytes.
Windows tests exercise the real Win32 publish path on native Windows. Power-loss behavior remains an API-contract property rather than something unit tests can prove; the testable invariants are that directory fsync is not called on Windows materialization, final-path collisions fail, temp logs are fsync'd before publication, and the resulting log loads normally.
Windows tests exercise the real Win32 publish path on native Windows. Power-loss behavior remains an API-contract property rather than something unit tests can prove; the testable invariants are that directory fsync is not called on Windows materialization, final-path collisions fail, maximum-length target components remain materializable, temp logs are fsync'd before publication, and the resulting log loads normally.
Append and repair still use ordinary file-handle fsyncs on both platforms. A failed append closes its append-only handle, reopens the log read/write, truncates it to the pre-append size, and fsyncs the rollback because Windows rejects `ftruncate` on append-only handles.

2
.agents/notes/implemented/architecture/2026-07-24-project-session-directories.i18n.yaml
View File

@@ -3,4 +3,4 @@
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-24-project-session-directories.md: 0aa3f513d5a1bb3e44cf33a0ae1eb791ee3a46c2
2026-07-24-project-session-directories.zh.md: f6bb1bd0ddb1067b68d1389182ce5b3397ad81fd
2026-07-24-project-session-directories.zh.md: 3d8d33fa9fddad010ab319ac4e1f873b69b4e1dd

4
.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md
View File

@@ -25,11 +25,11 @@ JSONL 后端按可读的项目键存储会话，并为每个会话提供独立
项目键有意不带哈希后缀。这遵循 coding agent（编码智能体）常用的易读约定，使规范化后的项目路径本身就是完整的目录名。规范化过程有损：`/a/b-c` 与 `/a-b/c` 等路径，或者保留前缀相同的长路径，会共用同一个项目目录。不同的会话 id 仍会选择不同的会话目录；复用相同的会话 id 仍构成存储冲突，系统会予以拒绝。
在不区分大小写的文件系统上，大小写不同的项目键也可能指向同一个物理目录。只有当文件系统路径规范化将发现路径和预期路径解析为同一个 transcript 时，身份验证才接受这种拼写变体。规范化后的路径如果不同，仍视为存储损坏，因此大小写别名不会让区分大小写的存储放宽同一 id 的冲突检查。
在不区分大小写的文件系统上，大小写不同的项目键也可能指向同一个物理目录。只有当文件系统路径规范化将发现路径和预期路径解析为同一个 transcript（文本记录）时，身份验证才接受这种拼写变体。规范化后的路径如果不同，仍视为存储损坏，因此大小写别名不会让区分大小写的存储放宽同一 id 的冲突检查。
根目录由部署配置决定。这种布局既不选择全局根目录，也不要求项目共享根目录。部署选择集中存储时，目录名仍能让项目路径易于辨认；使用项目本地根目录时，也采用同样的确定性结构。
编码后的会话 id 用于命名归属目录，而不是 transcript（文本记录）文件本身。`SessionPersistence.locate()` 仍返回固定的 transcript 路径，从而保持钩子 `transcript_path` 和 `DSH_SESSION_JSONL` 的语义不变。发现过程会忽略会话目录中的其他条目，因此后端以后添加会话自有产物时无需再次改变布局。
编码后的会话 id 用于命名归属目录，而不是 transcript 文件本身。`SessionPersistence.locate()` 仍返回固定的 transcript 路径，从而保持钩子 `transcript_path` 和 `DSH_SESSION_JSONL` 的语义不变。发现过程会忽略会话目录中的其他条目，因此后端以后添加会话自有产物时无需再次改变布局。
延迟物化仍以 transcript 为界：`create()` 不执行文件系统 I/O，首次追加会先创建项目目录和会话目录，再以无冲突方式发布 transcript。空目录不会被列为会话。后端会显式报告布局错误并拒绝扁平的 `<project>/<id>.jsonl*` 产物；预发布格式不提供自动数据迁移。