Merge remote-tracking branch 'origin/master' into mergebot/pr1667
This commit is contained in:
imccyu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-14-session-persistence.md
|
||||
2026-06-14-session-persistence.md: 137b2b01126214629952812f3dd3b71985a3acda
|
||||
2026-06-14-session-persistence.zh.md: 0f00902f5d6d60073bb56aabaf420bf2042e08fc
|
||||
2026-06-14-session-persistence.md: 00e129e57c7144fd62eec26f5854ee21dec4e964
|
||||
2026-06-14-session-persistence.zh.md: 1c98d5771819e8776d9f5cae4a147d2016ee5978
|
||||
|
||||
@@ -14,23 +14,23 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append
|
||||
|
||||
Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic:
|
||||
|
||||
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
|
||||
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `locate`/`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`/`list`/`listSnapshots`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
|
||||
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session: a `SessionHeader` line followed by storage records that losslessly represent the contiguous `SessionEvent` stream. Eligible `assistant/chunk` delta runs use packed rows by default; [checksummed Zstandard frames](2026-07-19-zstandard-jsonl-session-logs.md) are the default physical encoding, with raw lines configurable.
|
||||
|
||||
Key choices recorded here because they are durable, contested, and surprising:
|
||||
|
||||
- **The canonical durable log persists every `SessionEvent` losslessly, including `assistant/chunk`.** JSONL storage may encode a consecutive delta run as one packed row, but `load` reconstructs the exact event boundaries, sequence numbers, and timestamps. `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* logical log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
|
||||
- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, `load` preserves its contiguous, parseable events and appends risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }`. The synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable.
|
||||
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows. Its database carries a dedicated application id and monotonic schema version. A pristine file creates all tables and stamps both header values in one transaction; an unversioned file with any user-defined schema object or application identity, a foreign current-version identity, and every non-current version reject before journal-mode mutation.
|
||||
- **The canonical durable log persists every `SessionEvent` losslessly, including `assistant/chunk`.** JSONL storage may encode a consecutive delta run as one packed row, but logical readers reconstruct the exact event boundaries, sequence numbers, and timestamps. `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and validation of `events[i].seq === i` require a *contiguous* logical log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
|
||||
- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, cold inspection preserves its contiguous, parseable events and adds risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }` to the in-memory logical view. `prepare` or `load` commits those closers before returning a recoverable view; the synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded during committed repair; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable.
|
||||
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), and reads use SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, logical interrupted-turn closure, single committed repair, contiguous-seq), expressed once over file bytes and once over rows. Its database carries a dedicated application id and monotonic schema version. A pristine file creates all tables and stamps both header values in one transaction; an unversioned file with any user-defined schema object or application identity, a foreign current-version identity, and every non-current version reject before journal-mode mutation.
|
||||
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. `createdAt` is non-negative safe-integer Unix epoch milliseconds: live creation and persistence registration reject fractional values, JSONL validates the decoded header, and SQLite stores it in a strict `INTEGER` column. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).)
|
||||
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and registers the fresh agent under the exact resumed id. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
|
||||
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` obtains the exact unpublished Session through `ctx.sessionPersistence.prepare()`, publishes it under the persisted id, and continues its projections. The [Session preparation decision](2026-08-05-session-preparation.md) owns reuse between history inspection and resume. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
Each key choice above records its rejected alternative where the choice is stated: a **chunk-filtered canonical log** (Codex's `policy.rs` shape) — breaks the contiguous-seq contract; **truncating a crashed turn** — silently destroys a long autonomous run's real work; an **in-log `session/meta` event as line 0** — metadata is not replayable state; **finite fractional `createdAt` values** — have no producer and diverge from integer Unix-millisecond storage and query columns; **adopting a non-pristine unversioned SQLite file** — can overwrite unrelated objects or identity; **hard-injecting `sessionPersistence` into the loop** — would pend non-persistent demos forever.
|
||||
|
||||
Format versioning: the header carries a `version`; `load` rejects any non-current version. The pre-release session format stays pinned at `SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise, while the coordinator may own an explicit narrow import upgrade when persisted user data requires it ([pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later.
|
||||
Format versioning: the header carries a `version`; cold reads reject any non-current version. The pre-release session format stays pinned at `SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise, while the coordinator may own an explicit narrow import upgrade when persisted user data requires it ([pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated during cold preparation) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later.
|
||||
|
||||
## Consequences
|
||||
|
||||
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only, contiguous-seq, lazy-materialization, integer-metadata, and serializability semantics. Persisting the full logical log also settles event fidelity: every `assistant/chunk` survives exactly even when JSONL packs several into one storage row. SQLite initialization either commits its complete owned schema and header identity or leaves no partial schema to strand on the next open.
|
||||
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only, contiguous-seq, lazy-materialization, logical-recovery, integer-metadata, and serializability semantics. Persisting the full logical log also settles event fidelity: every `assistant/chunk` survives exactly even when JSONL packs several into one storage row. SQLite initialization either commits its complete owned schema and header identity or leaves no partial schema to strand on the next open.
|
||||
|
||||
@@ -14,23 +14,23 @@ Status: implemented
|
||||
|
||||
持久化是一个抽象的**能力 seam**（[能力 seam](2026-06-13-capability-seams.md)，`dsh-bash` 模板），而非循环或核心逻辑：
|
||||
|
||||
1. **接口**（`dsh-session-persistence`，`ctx.sessionPersistence`）：一个抽象的 `SessionPersistence` 服务，提供 `create`/`append`/`load`/`list`。其持久化单元就是现有的 `SessionEvent`（`{ type, seq, time, data }`），原样复用，无转换类型。
|
||||
1. **接口**（`dsh-session-persistence`，`ctx.sessionPersistence`）：一个抽象的 `SessionPersistence` 服务，提供 `locate`/`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`/`list`/`listSnapshots`。其持久化单元就是现有的 `SessionEvent`（`{ type, seq, time, data }`），原样复用，无转换类型。
|
||||
2. **实现**（`dsh-session-persistence-jsonl`）：每个会话一个仅追加的逻辑 JSONL 日志：先是一行 `SessionHeader`，随后是无损表示连续 `SessionEvent` 流的存储记录。符合条件的 `assistant/chunk` 增量连续段默认使用打包行；[带校验和的 Zstandard 帧](2026-07-19-zstandard-jsonl-session-logs.md)是默认物理编码，也可通过配置使用原始行。
|
||||
|
||||
以下关键选择记录于此，因为它们长期有效、存在争议且出人意料：
|
||||
|
||||
- **规范的持久日志无损保留每个 `SessionEvent`，包括 `assistant/chunk`。** JSONL 存储可以将一段连续的增量事件编码为一条打包行，但 `load` 会重建精确的事件边界、序号与时间戳。`deriveMessages()` 跳过分片，而过滤分片的方案（Codex 的 `policy.rs`）很有吸引力，但 `seq = log.length` 以及加载验证 `events[i].seq === i` 要求*连续*的逻辑日志；过滤掉分片会留下空洞，同时破坏契约和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现（带有自己的重新编号），但它不是规范日志。
|
||||
- **仅追加；崩溃的轮次被关闭，而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.md)会在模型分发前排空请求、在工具分发前排空已记录的顶层调用，并在步骤结束后排空完整的响应/结果批次；循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作，`load` 保留其连续、可解析的事件，并为未应答的 assistant 调用追加按风险分类的错误结果、补一个缺失的 `step/end`，以及带 `{ kind: 'interrupted' }` 的 `turn/end`。合成的结果保证恢复后的提供方 transcript（文本记录）仍然有效。只有不完整的最后一条记录会被丢弃；在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙，属于数据损坏，会使该会话不可加载。
|
||||
- **文件后端为规范实现，数据库后端为经过验证的直接替换。** `SessionEvent` 1:1 映射到一行 `(session_id, seq, type, time, data)`：`append` 是 INSERT（在一个断言连续 seq 契约的事务中），`load` 是 SELECT … ORDER BY seq。`dsh-session-persistence-sqlite` 正是如此：一个 `SessionPersistence` 子类，接口无变化（opencode 在 SQLite/WAL 上运行的正是这个形状），且通过与 JSONL 后端相同的 `runPersistenceContract` 测试套件。该契约以相同的语义约束两个后端（惰性物化、加载时关闭中断轮次、连续 seq），一次表达在文件字节上，一次表达在数据库行上。其数据库拥有专用的 application id 与单调递增的 schema 版本。系统会在一个事务中为全新文件创建所有表并写入这两个 header 值；未版本化文件若带有任何用户定义的 schema 对象或应用标识、当前版本文件若带有外部应用标识，以及任何非当前版本文件，都会在修改日志模式之前被拒绝。
|
||||
- **规范的持久日志无损保留每个 `SessionEvent`，包括 `assistant/chunk`。** JSONL 存储可以将一段连续的增量事件编码为一条打包行，但逻辑读取方会重建精确的事件边界、序号与时间戳。`deriveMessages()` 跳过分片，而过滤分片的方案（Codex 的 `policy.rs`）很有吸引力，但 `seq = log.length` 以及 `events[i].seq === i` 验证要求*连续*的逻辑日志；过滤掉分片会留下空洞，同时破坏契约和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现（带有自己的重新编号），但它不是规范日志。
|
||||
- **仅追加；崩溃的轮次被关闭，而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.md)会在模型分发前排空请求、在工具分发前排空已记录的顶层调用，并在步骤结束后排空完整的响应/结果批次；循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作，冷检查会保留其连续、可解析的事件，并在内存逻辑视图中为未应答的 assistant 调用添加按风险分类的错误结果、补一个缺失的 `step/end`，以及带 `{ kind: 'interrupted' }` 的 `turn/end`。`prepare` 或 `load` 在返回可恢复视图前提交这些 closer；合成结果保证恢复后的提供方 transcript（文本记录）仍然有效。只有不完整的最后一条记录会在提交修复时被丢弃；在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙，属于数据损坏，会使该会话不可加载。
|
||||
- **文件后端为规范实现，数据库后端为经过验证的直接替换。** `SessionEvent` 1:1 映射到一行 `(session_id, seq, type, time, data)`：`append` 是 INSERT（在一个断言连续 seq 契约的事务中），读取使用 SELECT … ORDER BY seq。`dsh-session-persistence-sqlite` 正是如此：一个 `SessionPersistence` 子类，接口无变化（opencode 在 SQLite/WAL 上运行的正是这个形状），且通过与 JSONL 后端相同的 `runPersistenceContract` 测试套件。该契约以相同的语义约束两个后端（惰性物化、逻辑关闭中断轮次、修复只提交一次、连续 seq），一次表达在文件字节上，一次表达在数据库行上。其数据库拥有专用的 application id 与单调递增的 schema 版本。系统会在一个事务中为全新文件创建所有表并写入这两个 header 值；未版本化文件若带有任何用户定义的 schema 对象或应用标识、当前版本文件若带有外部应用标识，以及任何非当前版本文件，都会在修改日志模式之前被拒绝。
|
||||
- **元数据在日志之外。** 格式版本、cwd 和谱系是存储关注点，不是可回放的对话状态，因此它们存放在 `dsh-session` 拥有的 `SessionHeader` 中，并通过新的只读属性 `session.header` 附加到 `Session` 上——永远不进入 `SessionEventMap`，永远不到达 `deriveMessages()`。`createdAt` 是以 Unix epoch 毫秒表示的非负安全整数：运行时创建和持久化注册会拒绝小数值，JSONL 会验证解码后的 header，SQLite 则将其存入严格的 `INTEGER` 列。替代方案（一个可合并扩展的 `session/meta` 事件作为日志第 0 行）被否决：日志内事件会自然随 seed/fork 的会话携带，但元数据不是可回放状态，因此显式的日志外 header seam 是更清晰的取舍。（header 最初被拆分为不可变的 `SessionHeader` 加可变的 `SessionSummary`，二者的联合类型为 `SessionMeta`；可变 summary 后来因属于死状态而被移除——见 [移除可变会话摘要](../simplification/2026-06-19-drop-mutable-session-summary.md)。）
|
||||
- **`ctx.agents.create()` 和 `ctx.agents.resume()` 是异步工厂；恢复还跨越持久化边界。** `ctx.agents.resume({ resumeSessionId })` 等待 `ctx.sessionPersistence.load`，用加载的事件重建活跃会话（使 `lastTurnNumber`/`deriveMessages` 得以延续），并以原样恢复的 id 注册新 agent。agent loop（智能体循环）不会硬注入 `sessionPersistence`（那样会让非持久化的演示永远挂起）；当它不存在时，`resume` 会以明确的错误拒绝。
|
||||
- **`ctx.agents.create()` 和 `ctx.agents.resume()` 是异步工厂；恢复还跨越持久化边界。** `ctx.agents.resume({ resumeSessionId })` 通过 `ctx.sessionPersistence.prepare()` 取得精确的未发布 Session，以持久化 id 发布它，并继续其投影。[Session 准备阶段决策](2026-08-05-session-preparation.md)定义历史检查与恢复之间的复用。agent loop（智能体循环）不会硬注入 `sessionPersistence`（那样会让非持久化的演示永远挂起）；当它不存在时，`resume` 会以明确的错误拒绝。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
上述每个关键选择都在陈述处记录了被否决的替代方案：**过滤分片的规范日志**（Codex 的 `policy.rs` 形式）破坏连续 seq 契约；**截断崩溃的轮次**会静默销毁长时间自主运行中的真实工作；**日志内 `session/meta` 事件作为第 0 行**——元数据不是可回放状态；**有限的非整数 `createdAt` 值**没有生产方，且与整数 Unix 毫秒存储及查询列不一致；**接受非全新的未版本化 SQLite 文件**可能覆盖无关对象或应用标识；**将 `sessionPersistence` 硬注入循环**会让非持久化的演示永远挂起。
|
||||
|
||||
格式版本控制：header 携带一个 `version`；`load` 拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`，不承诺广泛兼容；当持久化用户数据确有需要时，协调器可以负责显式且范围受限的导入升级（[消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)）。坦率地说：仅追加 + 刷写对部分尾部写入是健壮的（加载时容忍），但对行写入中途的无 fsync 断电不健壮；数据库/WAL 后端是后续更强的选项。
|
||||
格式版本控制：header 携带一个 `version`；冷读取拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`，不承诺广泛兼容；当持久化用户数据确有需要时，协调器可以负责显式且范围受限的导入升级（[消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)）。坦率地说：仅追加 + 刷写对部分尾部写入是健壮的（冷准备时容忍），但对行写入中途的无 fsync 断电不健壮；数据库/WAL 后端是后续更强的选项。
|
||||
|
||||
## 后果
|
||||
|
||||
新增两个包，以及 `dsh-session` 中的元数据 seam（`session.header`，`create(id?, options?)` 签名）。收益：持久恢复/fork、读取/回放路径、崩溃容忍，以及基于现有事件溯源日志的宿主侧会话访问，后端可在同一接口下替换。可复用的 `runPersistenceContract` 测试套件以相同的仅追加、连续 seq、惰性物化、整数元数据与可序列化语义约束每个后端。持久化完整的逻辑日志还确定了事件保真度：即使 JSONL 将多个 `assistant/chunk` 打包到一条存储行中，每个事件也会精确保留。SQLite 初始化要么提交完整的自有 schema 与 header 标识，要么不留下任何会使下次打开受阻的部分 schema。
|
||||
新增两个包，以及 `dsh-session` 中的元数据 seam（`session.header`，`create(id?, options?)` 签名）。收益：持久恢复/fork、读取/回放路径、崩溃容忍，以及基于现有事件溯源日志的宿主侧会话访问，后端可在同一接口下替换。可复用的 `runPersistenceContract` 测试套件以相同的仅追加、连续 seq、惰性物化、逻辑恢复、整数元数据与可序列化语义约束每个后端。持久化完整的逻辑日志还确定了事件保真度：即使 JSONL 将多个 `assistant/chunk` 打包到一条存储行中，每个事件也会精确保留。SQLite 初始化要么提交完整的自有 schema 与 header 标识，要么不留下任何会使下次打开受阻的部分 schema。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md
|
||||
2026-06-18-shared-persistence-write-coordinator.md: 4632351a6f39c44c9ba8af58d508d4665b9e9279
|
||||
2026-06-18-shared-persistence-write-coordinator.zh.md: f5a70d7d6e7ab76663620ca8d416c671f81e2f8f
|
||||
2026-06-18-shared-persistence-write-coordinator.md: 66b73b60ceec9497f1f1226747b8cebd831eb426
|
||||
2026-06-18-shared-persistence-write-coordinator.zh.md: 424ce6ec7384e8af7b979a29f58c31379a1d1850
|
||||
|
||||
@@ -10,9 +10,9 @@ English | [中文](2026-06-18-shared-persistence-write-coordinator.zh.md)
|
||||
|
||||
## Decision
|
||||
|
||||
Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its stateful public methods (`create`/`append`/`load`/`inspect`) to it. Backend-owned metadata and revision listing bypass the coordinator.
|
||||
Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its stateful public methods (`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`) to it. Backend-owned metadata and revision listing bypass the coordinator.
|
||||
|
||||
Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The Agent Note's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks and cannot reach the coordinator's private orchestration state. A third-party backend MAY still implement the abstract service directly without the coordinator, including the non-mutating `inspect` contract used by read models.
|
||||
Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The Agent Note's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks and cannot reach the coordinator's private orchestration state. A third-party backend MAY still implement the abstract service directly without the coordinator, including immutable logical inspection and the default preparation fallback through `load`.
|
||||
|
||||
The coordinator holds one controller for each exact live `Session`; the controller combines initialization, pending events, and the shared flush promise. Each `session/event` starts an eager drain, and `session/flush` observes quiescence rather than initiating the ordinary write path. The [flush-controller simplification](../simplification/2026-07-23-collapse-persistence-flush-state.md) owns this lifecycle.
|
||||
|
||||
@@ -23,9 +23,9 @@ The coordinator retires a session from `session/disposed`: it waits for the cont
|
||||
Five required members plus an optional lifecycle hook form the only boundary between the coordinator and storage:
|
||||
|
||||
- `name` — backend label for the dispose-failure `AggregateError`.
|
||||
- `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL project directory; SQLite's id is globally unique). Resume/load, non-mutating inspection, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication.
|
||||
- `loadStored(id)` — read one stored prefix by id across every storage scope (every JSONL project directory; SQLite's id is globally unique). Preparation, logical load/inspection, physical suffix reads, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication.
|
||||
- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized (the materialize-write and the first event batch must commit together — a crash between them must not leave a materialized-but-empty session; this is why there is no separate `materialize` hook).
|
||||
- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`).
|
||||
- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates-then-appends in two fsync'd steps, SQLite does DELETE+INSERT in one transaction. Used by `prepare`/`load` (truncate + synthetic closers) and live-adoption (truncate only, `closers = []`).
|
||||
- `list()` — list all stored metadata.
|
||||
- `close?()` — optional lifecycle teardown (SQLite closes its db handle; JSONL omits it), awaited in the dispose effect AFTER the quiescence drain so a close failure never masks a drain error.
|
||||
|
||||
@@ -35,7 +35,7 @@ The single design choice that keeps the seam clean: the crash-repair "where is t
|
||||
|
||||
## Testing
|
||||
|
||||
The shared `runPersistenceContract` (public-API contract) runs for every backend and proves that `inspect` leaves interrupted logs and revisions unchanged before `load` performs recovery. `runCoordinatorContract` (`tests/coordinator-contract.ts`) covers adoption, HMR, collision, session and backend disposal drains, and crash-tail repair through an in-memory reference, JSONL, and SQLite. Coordinator-specific tests cover eager follow-up batches, live-controller cleanup, same-id chain-tail races, failed-drain retry, and close ordering. The per-backend specs retain storage mechanics only. A through-coordinator torn-tail repair test per real backend keeps the opaque-marker branch covered because the contract crash case produces synthetic closers without a torn marker.
|
||||
The shared `runPersistenceContract` (public-API contract) runs for every backend and proves that `inspect` balances an interrupted logical view without changing storage or revisions before `prepare` or `load` commits recovery. `runCoordinatorContract` (`tests/coordinator-contract.ts`) covers adoption, HMR, collision, session and backend disposal drains, and crash-tail repair through an in-memory reference, JSONL, and SQLite. `persistence.spec.ts` and `preparations.spec.ts` cover preparation reuse and reservation, bounded prepared-state eviction, eager follow-up batches, live-controller cleanup, same-id chain-tail races, failed-drain retry, and close ordering. The per-backend specs retain storage mechanics only. A through-coordinator torn-tail repair test per real backend keeps the opaque-marker branch covered because the contract crash case produces synthetic closers without a torn marker.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -44,4 +44,4 @@ The shared `runPersistenceContract` (public-API contract) runs for every backend
|
||||
|
||||
## Consequences
|
||||
|
||||
The coordinator adds one indirection, an opaque torn marker, and detached session-retirement tasks, but centralizes correctness-heavy orchestration previously duplicated by every backend. Session disposal remains an observe-only event, so the session owner does not await persistence retirement; the coordinator contains failures, preserves pending events in the live controller, and makes backend teardown the quiescence boundary. Its hook surface stays narrow: identity, adoption, collision checks, and non-mutating inspection reuse `loadStored`; materialization stays atomic inside `appendBatch`; and listing bypasses the coordinator. Read models use `inspect` rather than `load`, so observing a persisted open turn cannot race a new live owner by committing interruption closers. New backends implement storage primitives rather than copy the eager write lifecycle.
|
||||
The coordinator adds one indirection, an opaque torn marker, detached session-retirement tasks, and bounded prepared Session state, but centralizes correctness-heavy orchestration previously duplicated by every backend. Session disposal remains an observe-only event, so the session owner does not await persistence retirement; the coordinator contains failures, preserves pending events in the live controller, and makes backend teardown the quiescence boundary. Its hook surface stays narrow: identity, adoption, collision checks, preparation, and immutable inspection reuse `loadStored`; materialization stays atomic inside `appendBatch`; and listing bypasses the coordinator. Read models use `inspect` rather than `load`, so observing a persisted open turn does not commit interruption closers; the [Session preparation decision](2026-08-05-session-preparation.md) owns reuse, reservation, and publication. New backends implement storage primitives rather than copy the eager write lifecycle.
|
||||
|
||||
@@ -10,9 +10,9 @@ Status: implemented
|
||||
|
||||
## 决策
|
||||
|
||||
将一个后端无关的 `PersistenceCoordinator` 提取到 `dsh-session-persistence` 中。协调器统一拥有编排逻辑；每个第一方后端组合一个协调器实例（`new PersistenceCoordinator(ctx, this)`），实现一个小型 `PersistenceBackend` 钩子接口，并将其有状态的公开方法（`create`/`append`/`load`/`inspect`）委托给协调器。由后端拥有的元数据与修订版本列举会绕过协调器。
|
||||
将一个后端无关的 `PersistenceCoordinator` 提取到 `dsh-session-persistence` 中。协调器统一拥有编排逻辑；每个第一方后端组合一个协调器实例（`new PersistenceCoordinator(ctx, this)`），实现一个小型 `PersistenceBackend` 钩子接口，并将其有状态的公开方法（`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`）委托给协调器。由后端拥有的元数据与修订版本列举会绕过协调器。
|
||||
|
||||
组合，而非继承。协调器是后端持有的具体类，不是后端继承的基类。本 Agent Note 的风险——「协调器不得让非常规后端与继承层级作斗争」——由此规避：后端只暴露钩子，无法触及协调器的私有编排状态。第三方后端仍然可以完全不使用协调器、直接实现抽象服务，包括供读模型使用、不修改状态的 `inspect` 契约。
|
||||
组合，而非继承。协调器是后端持有的具体类，不是后端继承的基类。本 Agent Note 的风险——「协调器不得让非常规后端与继承层级作斗争」——由此规避：后端只暴露钩子，无法触及协调器的私有编排状态。第三方后端仍然可以完全不使用协调器、直接实现抽象服务，包括不可变逻辑检查，以及通过 `load` 实现的默认准备回退。
|
||||
|
||||
协调器为每个存活的 `Session` 实例持有一个控制器；该控制器统合初始化、待处理事件与共享 flush promise。每个 `session/event` 都会立即启动排空，而 `session/flush` 只观察完全停稳，不会发起常规写入路径。[flush 控制器简化](../simplification/2026-07-23-collapse-persistence-flush-state.md)定义该生命周期。
|
||||
|
||||
@@ -23,9 +23,9 @@ Status: implemented
|
||||
五个必需成员加一个可选的生命周期钩子，构成协调器与存储之间唯一的边界：
|
||||
|
||||
- `name`——后端标签，用于 dispose 失败时的 `AggregateError`。
|
||||
- `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀（JSONL 的所有项目目录；SQLite 的 id 全局唯一）。恢复/加载、不修改状态的检查、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id，并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。
|
||||
- `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀（JSONL 的所有项目目录；SQLite 的 id 全局唯一）。准备、逻辑加载/检查、物理后缀读取、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id，并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。
|
||||
- `appendBatch(meta, events, isMaterialized)`——持久追加一个连续批次，在尚未物化时原子地惰性物化会话（物化写入与首批事件必须一起提交——二者之间发生崩溃时，不得留下一个已物化但为空的会话；这就是为什么没有单独的 `materialize` 钩子）。
|
||||
- `commitRepair(meta, tornMarker, closers)`——使崩溃修复持久化：截断损坏的尾部（当且仅当 `tornMarker !== undefined`）并追加 `closers`。**不要求原子性**——JSONL 合理地分两步 fsync（先截断再追加），SQLite 在一个事务中完成 DELETE+INSERT。用于 `load`（截断 + 合成 closers）和 live-adoption（仅截断，`closers = []`）。
|
||||
- `commitRepair(meta, tornMarker, closers)`——使崩溃修复持久化：截断损坏的尾部（当且仅当 `tornMarker !== undefined`）并追加 `closers`。**不要求原子性**——JSONL 合理地分两步 fsync（先截断再追加），SQLite 在一个事务中完成 DELETE+INSERT。用于 `prepare`/`load`（截断 + 合成 closers）和 live-adoption（仅截断，`closers = []`）。
|
||||
- `list()`——列出所有已存储的元数据。
|
||||
- `close?()`——可选的生命周期清理（SQLite 关闭 db 句柄；JSONL 省略），在 dispose effect 中于排空至完全停稳之后被 await，因此 close 失败不会掩盖排空错误。
|
||||
|
||||
@@ -35,7 +35,7 @@ Status: implemented
|
||||
|
||||
## 测试
|
||||
|
||||
共享的 `runPersistenceContract`（公开 API 契约）为每个后端运行，并证明在 `load` 执行恢复之前，`inspect` 会保持被中断的日志与修订版本不变。`runCoordinatorContract`（`tests/coordinator-contract.ts`）通过内存参考实现、JSONL 与 SQLite 覆盖接管、HMR、碰撞、会话与后端 dispose 排空，以及崩溃尾部修复。协调器专属测试覆盖立即执行的后续批次、存活控制器清理、同 id 链尾竞态、排空失败重试与关闭顺序。各后端自身的测试规格只保留存储机制。每个真实后端都有一个经由协调器的崩溃尾部修复测试，以覆盖不透明 marker 分支，因为契约中的崩溃用例会产生合成 closers，却不会产生 torn marker。
|
||||
共享的 `runPersistenceContract`（公开 API 契约）为每个后端运行，并证明 `inspect` 会配平被中断的逻辑视图但不改变存储或修订版本，随后由 `prepare` 或 `load` 提交恢复。`runCoordinatorContract`（`tests/coordinator-contract.ts`）通过内存参考实现、JSONL 与 SQLite 覆盖接管、HMR、碰撞、会话与后端 dispose 排空和崩溃尾部修复。`persistence.spec.ts` 与 `preparations.spec.ts` 覆盖准备复用与预留、有界准备状态淘汰、立即执行的后续批次、存活控制器清理、同 id 链尾竞态、排空失败重试与关闭顺序。各后端自身的测试规格只保留存储机制。每个真实后端都有一个经由协调器的崩溃尾部修复测试，以覆盖不透明 marker 分支，因为契约中的崩溃用例会产生合成 closers，却不会产生 torn marker。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
@@ -44,4 +44,4 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
协调器增加了一层间接、一个不透明的 torn marker 和脱离会话生命周期的退役任务，但将此前每个后端重复的、对正确性要求很高的编排逻辑集中到一处。会话 dispose 仍是仅观察事件，因此会话所有者不会等待持久化退役；协调器会收容失败、在存活控制器中保留待处理事件，并以后端 teardown 为完全停稳边界。其钩子面保持窄小：标识校验、接管、碰撞检查与不修改状态的检查共用 `loadStored`；物化保持在 `appendBatch` 内原子完成；列举绕过协调器。读模型使用 `inspect` 而非 `load`，因此观察已持久化但仍开放的轮次时，不会因提交中断 closers 而与新的存活所有者产生竞态。新后端只需实现存储原语，而无需复制立即写入生命周期。
|
||||
协调器增加了一层间接、一个不透明的 torn marker、脱离会话生命周期的退役任务，以及有界的已准备 Session 状态，但将此前每个后端重复的、对正确性要求很高的编排逻辑集中到一处。会话 dispose 仍是仅观察事件，因此会话所有者不会等待持久化退役；协调器会收容失败、在存活控制器中保留待处理事件，并以后端 teardown 为完全停稳边界。其钩子面保持窄小：标识校验、接管、碰撞检查、准备与不可变检查共用 `loadStored`；物化保持在 `appendBatch` 内原子完成；列举绕过协调器。读模型使用 `inspect` 而非 `load`，因此观察已持久化但仍开放的轮次时不会提交中断 closers；复用、预留与发布由 [Session 准备阶段决策](2026-08-05-session-preparation.md)定义。新后端只需实现存储原语，而无需复制立即写入生命周期。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md
|
||||
2026-07-19-zstandard-jsonl-session-logs.md: 287ec94a91101850e9343d36ffd27870daf1333b
|
||||
2026-07-19-zstandard-jsonl-session-logs.zh.md: 4e578432640651de1eb1977229b7cdd462766c24
|
||||
2026-07-19-zstandard-jsonl-session-logs.md: 93fc20f931c75552352834b9340e7d38680d4254
|
||||
2026-07-19-zstandard-jsonl-session-logs.zh.md: 061d7fcb55c775eed10e99bae47777d32cc8eee1
|
||||
|
||||
@@ -28,7 +28,7 @@ First materialization compresses the two initial frames before opening the tempo
|
||||
|
||||
### Read, listing, and crash recovery
|
||||
|
||||
A frame-boundary scanner reads the standard magic, variable header fields, block headers and payload sizes, and optional checksum trailer. It does not interpret compressed blocks. Complete frames are decompressed independently and sequentially with Node's default `ZSTD_e_end`, which requires frame completion and validates their checksums, and their plaintext is passed to the existing JSONL scanner. A checksum/decompression failure in any complete frame, a malformed complete-frame JSONL tail, or invalid frame structure is corruption and rejects.
|
||||
A frame-boundary scanner reads the standard magic, variable header fields, block headers and payload sizes, and optional checksum trailer. It does not interpret compressed blocks. Complete frames are independently checksum-validated and passed through the [large-session restore pipeline](2026-08-05-large-session-jsonl-restore-pipeline.md), which owns decoder reuse, cooperative yielding, and incremental JSONL scanning. A checksum/decompression failure in any complete frame, a malformed complete-frame JSONL tail, or invalid frame structure is corruption and rejects.
|
||||
|
||||
Listing reads in bounded chunks only until the first complete frame is available, validates and decompresses that header frame, and never reads an event frame. The dedicated header frame therefore preserves metadata-only listing even for very large session logs.
|
||||
|
||||
@@ -53,5 +53,5 @@ The shared persistence and coordinator contracts run against both encodings. Bac
|
||||
- Ordinary session roots store `.jsonl.zstd` and retain append-only, fsync, rollback, and interrupted-turn recovery semantics.
|
||||
- Raw JSONL remains a deliberate configuration, but changing encoding requires a fresh/separate root or selecting the mode that matches existing artifacts.
|
||||
- One frame per durable batch adds bounded framing/checksum overhead and allows header-only listing plus repair from an exact append boundary.
|
||||
- External tools must understand concatenated Zstandard frames or consume raw-mode artifacts; generic one-shot Node decompression reads only the first independent frame, so backend reads walk frames explicitly.
|
||||
- External tools must understand concatenated Zstandard frames or consume raw-mode artifacts; generic one-shot Node decompression reads only the first independent frame, so backend reads walk frames through the [restore pipeline](2026-08-05-large-session-jsonl-restore-pipeline.md).
|
||||
- The implementation depends on Node's experimental built-in Zstandard API without an npm dependency; the supported-version compatibility gate makes drift visible.
|
||||
|
||||
@@ -28,7 +28,7 @@ JSONL 持久化后端会逐字保留每个 `SessionEvent`，其中包括数量
|
||||
|
||||
### 读取、列举与崩溃恢复
|
||||
|
||||
帧边界扫描器会读取标准魔数、可变头字段、块头与负载长度，以及可选校验和尾部，但不会解释压缩块。后端使用 Node 默认的 `ZSTD_e_end` 独立且按顺序解压完整帧；该模式要求帧完整并验证各帧校验和，再把明文交给既有 JSONL 扫描器。任何完整帧的校验和或解压失败、完整帧中畸形的 JSONL 尾部，或者无效帧结构都属于损坏并拒绝加载。
|
||||
帧边界扫描器会读取标准魔数、可变头字段、块头与负载长度，以及可选校验和尾部，但不会解释压缩块。完整帧会独立验证校验和，再进入[大型会话恢复流水线](2026-08-05-large-session-jsonl-restore-pipeline.md)；该流水线负责复用解码器、协作式让出事件循环和增量扫描 JSONL。任何完整帧的校验和或解压失败、完整帧中畸形的 JSONL 尾部，或者无效帧结构都属于损坏并拒绝加载。
|
||||
|
||||
列举只按有界分片读取到第一个完整帧可用为止，验证并解压该头部帧，绝不读取事件帧。因此，即使会话日志很大，专用头部帧仍能维持仅元数据列举。
|
||||
|
||||
@@ -53,5 +53,5 @@ CLI、ACP 与 stdio 应用包公开对称的 `persistenceCompression` 透传配
|
||||
- 普通会话根目录存储 `.jsonl.zstd`，并保留仅追加、fsync、回滚与中断轮次恢复语义。
|
||||
- 原始 JSONL 仍是显式配置，但切换编码需要使用全新或单独根目录，或者选择与既有产物匹配的模式。
|
||||
- 每个持久批次一个帧会增加有界的帧与校验和开销，同时支持仅头部列举和从精确追加边界开始修复。
|
||||
- 外部工具必须理解串联的 Zstandard 帧，或者消费原始模式产物；Node 通用的一次性解压只读取第一个独立帧，因此后端读取会显式遍历各帧。
|
||||
- 外部工具必须理解串联的 Zstandard 帧，或者消费原始模式产物；Node 通用的一次性解压只读取第一个独立帧，因此后端读取会通过[恢复流水线](2026-08-05-large-session-jsonl-restore-pipeline.md)遍历各帧。
|
||||
- 实现依赖 Node 的实验性内置 Zstandard API，但不增加 NPM 依赖；受支持版本兼容性门禁会暴露 API 漂移。
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-05-large-session-jsonl-restore-pipeline.md
|
||||
2026-08-05-large-session-jsonl-restore-pipeline.md: eab53c683880ef7095233ed8122e532eb5add547
|
||||
2026-08-05-large-session-jsonl-restore-pipeline.zh.md: 039e0c193179677b57e742d55f4c7df6bdff852f
|
||||
@@ -0,0 +1,52 @@
|
||||
# Agent Note: Large-session JSONL restore pipeline
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-05-large-session-jsonl-restore-pipeline.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Restoring a stored session activates it and materializes its complete authoritative event log before the agent can run. Large JSONL artifacts made that one-time operation pay several avoidable costs: each independent Zstandard frame created and closed a decoder context, decoded plaintext was accumulated and rescanned as whole-log buffers and strings, and freshly parsed events went through generic snapshot and deep-freeze paths designed for borrowed or cyclic values.
|
||||
|
||||
A representative profile contained 61.8 MiB of Zstandard data, 97.1 MiB of plaintext, and 1,307,073 events. The restore path must reduce its CPU and memory cost without weakening checksum validation, committed-region corruption detection, torn-tail recovery, sequence and surface validation, or the session log's immutability.
|
||||
|
||||
## Decision
|
||||
|
||||
Restoration is one ownership-transfer pipeline from the persistence artifact into `Session.fromRestore`. The compressed artifact remains the source buffer, while each decoding and scanning stage consumes the previous stage's output incrementally without retaining a whole-log plaintext or parsed copy; the resulting event array is the only complete decoded representation.
|
||||
|
||||
### Frame decoding
|
||||
|
||||
The structural Zstandard scanner identifies complete frame ranges before decoding. The dedicated first frame is decoded and parsed separately as the session header; subsequent plaintext frames are yielded in order into the JSONL scanner.
|
||||
|
||||
`ZstdFrameDecoder` gives the reader one lifecycle for interchangeable synchronous implementations. The preferred implementation probes the supported Node 22, 24, and 26 stream shape, reuses one private native decoder context and scratch buffer across all complete frames, and closes it once. If that private shape is unavailable, the factory selects a public `zstdDecompressSync` implementation with the same iterator and checksum-error contract. A yielded scratch view is consumed before the iterator advances.
|
||||
|
||||
After approximately 500 ms of accumulated frame work, the asynchronous reader yields at the next frame boundary and observes cancellation before continuing. A single frame remains an indivisible synchronous operation. Complete frames require end-of-frame and checksum validation; only a structurally incomplete final frame uses the existing prefix decoder for recovery.
|
||||
|
||||
### Incremental JSONL scanning
|
||||
|
||||
`SessionLogScanner` searches raw buffers with `Buffer.indexOf(0x0A)` and converts only complete records to UTF-8 for `JSON.parse`. It carries an incomplete record across decoder writes and copies only that fragment because the private decoder may reuse its output buffer. It does not build a whole plaintext buffer or string, a line array, or a second parsed-record array.
|
||||
|
||||
The scanner stops retaining events at the first unparsable row or sequence gap but continues inspecting later complete records. A later `turn/end` proves that the issue lies in the committed region and rejects the log. The Zstandard reader also rejects any unresolved parse, sequence, or partial-record issue after all complete frames; only a structurally torn final frame may contribute a recoverable suffix. Complete records emitted from that torn frame pass through the same scanner and retain the existing repair offset and recovered-event semantics.
|
||||
|
||||
### Restore admission
|
||||
|
||||
Persistence transfers freshly materialized JSON values to `Session.fromRestore`. These values are detached, acyclic trees, and packed chunk rows expand into newly allocated events, so the restore-only path validates the fixed event envelope with one `for...in` and `switch`, dispatches current-shape checks by event discriminant, and iteratively freezes the owned graph with an explicit `pending` array and no cycle-tracking set. Surface validation records one transition plan and commits that plan when the exact candidate enters the log instead of planning the same event twice.
|
||||
|
||||
Borrowed seeds used by ordinary creation and fork paths still take a JSON snapshot and use the generic cycle-safe deep freeze. The specialization therefore changes only durable restoration; it does not weaken acceptance for caller-owned values.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **One asynchronous native operation per frame** — rejected because dispatch and callback overhead dominates logs containing many small durable batches. Cooperative synchronous decoding pays that overhead only at periodic yield boundaries.
|
||||
- **Process the complete log synchronously without yielding** — rejected because it prevents cancellation and event-loop progress for the full restore duration. Frame-boundary yields retain a bounded observation point without splitting codec operations.
|
||||
- **Concatenate all plaintext before scanning** — rejected because it retains the compressed input, complete plaintext, whole-log UTF-8 string, line metadata, and parsed rows at the same time, and it rescans a torn-frame prefix.
|
||||
- **Implement a streaming JSON parser** — rejected because JSONL already provides record boundaries; native newline search plus `JSON.parse` removes the large intermediates without owning another parser or changing JSON semantics.
|
||||
- **Use a shared `WeakSet` while freezing restored events** — rejected because JSON materialization cannot produce cycles, and the set adds a lookup per object while retaining the complete graph during traversal.
|
||||
- **Skip validation or freezing for restored values** — rejected because durable storage is a runtime boundary and `Session.events` promises immutable accepted history. The optimized path specializes those operations around stronger ownership facts instead of removing them.
|
||||
|
||||
## Consequences
|
||||
|
||||
On the representative profile, incremental scanning reduced JSONL scan time from about 598 ms to 397 ms and peak RSS from about 1,494 MiB to 1,060 MiB. Restore admission reduced `Session.fromRestore` from 604–608 ms to about 263 ms, including an `assertSessionEventEnvelope` reduction from about 77 ms to 13 ms. These measurements characterize the optimization input rather than establish runtime limits.
|
||||
|
||||
The fast decoder depends on runtime-probed Node internals, but incompatibility selects the public implementation rather than changing correctness. Cancellation is observed around cooperative frame-boundary yields; the deadline is not a hard wall-clock bound inside one frame. The complete event array remains resident because it is the active session's authoritative log; the pipeline removes duplicate representations rather than paginating that state.
|
||||
|
||||
Tests force both decoder implementations, compare their frame order and corruption behavior, exercise cooperative cancellation and torn-tail recovery, and retain the existing session envelope, surface, and immutability contracts.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Agent Note: 大型会话 JSONL 恢复流水线
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-05-large-session-jsonl-restore-pipeline.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
恢复已存储会话会激活该会话，并在 agent（智能体）运行前物化完整且权威的事件日志。处理大型 JSONL 产物时，这个一次性操作会产生几项不必要的开销：每个独立 Zstandard 帧都会创建并关闭一个解码上下文；解码后的明文会汇总成整份日志的缓冲区和字符串，再进行重复扫描；刚解析出的事件还会进入面向借用值或循环引用值设计的通用快照与深度冻结路径。
|
||||
|
||||
一份代表性性能剖析包含 61.8 MiB Zstandard 数据、97.1 MiB 明文和 1,307,073 个事件。恢复路径必须降低 CPU 与内存开销，同时保持校验和验证、已提交区域损坏检测、撕裂尾部恢复、序列与 `surface` 校验，以及会话日志不可变性。
|
||||
|
||||
## 决策
|
||||
|
||||
恢复过程是一条从持久化产物进入 `Session.fromRestore` 的所有权转移流水线。压缩产物仍作为源缓冲区驻留，但解码与扫描阶段会增量消费上一阶段的输出，不会保留整份日志的明文或解析副本；最终事件数组是唯一完整的已解码表示。
|
||||
|
||||
### 帧解码
|
||||
|
||||
Zstandard 结构扫描器会在解码前识别完整帧范围。系统单独解码专用首帧并将其解析为会话头部，后续明文帧则按顺序产出并送入 JSONL 扫描器。
|
||||
|
||||
`ZstdFrameDecoder` 为可互换的同步实现提供统一生命周期。首选实现会探测受支持 Node 22、24 与 26 的流结构，在所有完整帧之间复用一个私有原生解码上下文和临时缓冲区，最后只关闭一次。如果私有结构不可用，工厂会选择使用公共 `zstdDecompressSync` 的实现，并保持相同的迭代器和校验和错误契约。迭代器产出的临时视图会在进入下一次迭代前被消费。
|
||||
|
||||
累计帧处理时间约达 500 ms 后，异步读取器会在下一帧边界让出事件循环，并在继续前观察取消信号。单个帧仍是不可分割的同步操作。完整帧必须通过帧结束与校验和验证；只有结构上不完整的最终帧才使用既有前缀解码器进行恢复。
|
||||
|
||||
### 增量 JSONL 扫描
|
||||
|
||||
`SessionLogScanner` 使用 `Buffer.indexOf(0x0A)` 在原始缓冲区中查找换行，只把完整记录转换为 UTF-8 并交给 `JSON.parse`。扫描器会跨解码写入保留不完整记录；由于私有解码器可能复用输出缓冲区，它只复制这个片段。扫描过程不会构造整份明文缓冲区或字符串，也不会构造行数组或第二份解析记录数组。
|
||||
|
||||
扫描器在遇到第一条无法解析的记录或序列缺口后停止保留事件，但会继续检查后续完整记录。后续出现 `turn/end`，说明问题位于已提交区域，系统会拒绝该日志。处理完所有完整帧后，如果仍存在未决的解析错误、序列错误或部分记录，Zstandard 读取器同样会拒绝日志；只有结构上撕裂的最终帧才能提供可恢复后缀。该撕裂帧产出的完整记录会经过同一扫描器，并保持既有修复偏移量与恢复事件语义。
|
||||
|
||||
### 恢复准入
|
||||
|
||||
持久化层把刚物化的 JSON 值转移给 `Session.fromRestore`。这些值是已分离且无环的树，打包的分片行也会展开成新分配的事件。因此，恢复专用路径使用一次 `for...in` 与 `switch` 校验固定事件信封，按事件判别字段执行当前数据形状检查，并通过显式 `pending` 数组迭代冻结所拥有的对象图，不使用循环跟踪集合。`surface` 校验会记录一次转换计划；当同一个候选事件进入日志时，系统直接提交该计划，不再对同一事件规划两次。
|
||||
|
||||
普通创建与 fork 路径使用的借用 `seed` 仍会创建 JSON 快照，并使用支持循环检测的通用深度冻结。因此，这项特化仅改变持久恢复，不会放宽调用方所有值的准入要求。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **每帧执行一次异步原生操作**：不予采纳，因为对于包含大量小型持久化批次的日志，调度与回调开销占据主要部分。协作式同步解码只在周期性让出边界支付这类开销。
|
||||
- **同步处理完整日志且不让出事件循环**：不予采纳，因为整个恢复期间都无法响应取消或推进事件循环。帧边界让出机制无需拆分编解码操作，就能保留有界的观察点。
|
||||
- **扫描前拼接全部明文**：不予采纳，因为该方案会同时保留压缩输入、完整明文、整份日志的 UTF-8 字符串、行元数据和解析记录，并会重新扫描撕裂帧前缀。
|
||||
- **实现流式 JSON 解析器**：不予采纳，因为 JSONL 已提供记录边界；使用原生换行搜索与 `JSON.parse` 就能移除大型中间结构，无需自行维护另一套解析器或改变 JSON 语义。
|
||||
- **冻结恢复事件时共享一个 `WeakSet`**：不予采纳，因为 JSON 物化不可能产生循环引用，而该集合会对每个对象增加一次查找，并在遍历期间保留完整对象图。
|
||||
- **跳过恢复值的校验或冻结**：不予采纳，因为持久存储属于运行时边界，而 `Session.events` 承诺已接受历史不可变。优化路径利用更强的所有权事实特化这些操作，而不是将其移除。
|
||||
|
||||
## 后果
|
||||
|
||||
在代表性性能剖析中，增量扫描将 JSONL 扫描时间从约 598 ms 降至 397 ms，峰值 RSS 从约 1,494 MiB 降至 1,060 MiB。恢复准入将 `Session.fromRestore` 从 604–608 ms 降至约 263 ms，其中 `assertSessionEventEnvelope` 从约 77 ms 降至 13 ms。这些数据用于描述优化输入，不构成运行时上限。
|
||||
|
||||
快速解码器依赖运行时探测的 Node 内部接口，但接口不兼容时会改用公共实现，不会改变正确性。系统会在协作式帧边界让出点观察取消信号；截止时间并不是单个帧内部严格的挂钟时间上限。完整事件数组仍会驻留内存，因为它是活跃会话的权威日志；该流水线移除的是重复表示，并未对这份状态做分页。
|
||||
|
||||
测试会强制执行两种解码器实现，比对帧顺序和损坏处理行为，覆盖协作式取消与撕裂尾部恢复，并保留既有会话信封、`surface` 与不可变性契约。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-05-session-preparation.md
|
||||
2026-08-05-session-preparation.md: 69d39f552ed3041403a24b5aefb435e4e721b09c
|
||||
2026-08-05-session-preparation.zh.md: a0ca27eb63552566c918c299bd5fba976687812c
|
||||
@@ -0,0 +1,68 @@
|
||||
# Agent Note: Reusable Session preparation before publication
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-05-session-preparation.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Cold history inspection and Agent resume independently materialized the same persisted session log. For a large compressed log, each operation repeated the full read, decompression, parse, validation, freezing, and Session construction. Pagination could therefore pay the cold-read cost again, while making a history query activate an Agent would couple a read lifecycle to a live Agent with no natural retirement point.
|
||||
|
||||
Fresh creation and persisted resume also reached the same publication boundary through different construction flows. This obscured the invariant that setup must finish against one unpublished Session before that exact Session and its Agent become visible together.
|
||||
|
||||
## Decision
|
||||
|
||||
`SessionPreparation` owns one exact unpublished `Session` until publication or rollback. It is a Session lifecycle object, not an Agent lifecycle or activation object. Fresh creation wraps the result of `SessionStore.prepare()`; persisted resume obtains a preparation from `SessionPersistence.prepare()`.
|
||||
|
||||
The Agent loop consumes both forms through one setup-and-publication pipeline: it acquires the preparation, builds the private Agent context around `preparation.session`, awaits optional setup, publishes that exact Session and Agent, and disposes the preparation on every exit. Publication transfers the live lifecycle to the existing Session and Agent stores; `SessionPreparation` itself owns no Agent behavior.
|
||||
|
||||
This refines the publication boundary from the [Agent lifecycle and ownership decision](2026-06-18-agent-lifecycle-and-ownership-seams.md) without replacing its ownership model.
|
||||
|
||||
## Persisted preparation lifecycle
|
||||
|
||||
A coordinator-backed persistence implementation loads one cold source into a prepared Session. The backend transfers fresh, mutually unaliased metadata and events together with the source-qualified revision that identifies those exact values; the Session restore path validates and freezes the graphs in place instead of cloning them. The coordinator computes interrupted-turn closers and constructs the exact unpublished Session once. Its immutable header and balanced logical event log form the `SessionInspection` borrowed by readers, while the revision remains internal to persistence.
|
||||
|
||||
`inspect(id, signal?)` does not mutate storage. Synthetic closers exist only in the prepared in-memory view, and a torn physical tail remains untouched. Same-id callers share an in-flight cold read. Once ready, the preparation may remain in a per-coordinator LRU whose capacity defaults to five and is configurable by first-party backends. Before reusing a retained source, the coordinator reads that id's current revision; a mismatch evicts a ready source and repeats the cold materialization. A source already committing or reserved for resume remains exclusively owned, so concurrent inspection borrows that immutable view until publication or release.
|
||||
|
||||
`prepare(id, signal?)` exclusively reserves the prepared Session. It confirms the retained revision before committing any torn-tail and interrupted-turn repair, establishes the durable cursor, then returns a disposable preparation. A stale source is discarded and reloaded instead of being repaired or published. A successful repair also discards the pre-repair source and materializes the committed log again before reservation, so a newer revision is never associated with an older event graph. Another same-id preparation waits until the reservation is published or released. Publication accepts only the exact reserved Session and attaches the committed cursor without rebuilding its history. Failed setup or cancellation returns an unchanged unpublished Session to the LRU; mutation or attachment consumes the reservation.
|
||||
|
||||
The legacy `load(id)` API uses the same preparation and repair machinery, then discards its reservation and returns the immutable logical view. It remains a compatibility API, not the history-to-resume reuse path. This lifecycle extends the [shared persistence coordinator](2026-06-18-shared-persistence-write-coordinator.md) while preserving the storage and recovery rules owned by the [session persistence decision](2026-06-14-session-persistence.md).
|
||||
|
||||
## History and resume reuse
|
||||
|
||||
History reads use `inspect()`, so repeated pages borrow the same immutable prepared state without activating an Agent. A later resume uses `prepare()` and receives the exact Session retained by inspection; it does not read, decompress, parse, clone, validate, or freeze the complete log again.
|
||||
|
||||
If the durable log changes after inspection, its revision changes. The next history read or resume discards a retained ready Session and materializes the new log, so an old event graph cannot be associated with a newer snapshot revision. A source already claimed by an in-flight resume is not evicted: its exclusive owner keeps it through publication or release, and concurrent history may borrow the same immutable view.
|
||||
|
||||
Cold continuable-subagent access follows the same path. Descriptor authorization first inspects the child, then `ctx.agents.resume()` reserves and publishes the retained Session. This preserves the lifecycle and authorization rules in the [continuable subagent conversation decision](../feature/2026-07-28-continuable-subagent-conversations.md) while removing its duplicate cold read.
|
||||
|
||||
## Boundaries
|
||||
|
||||
- `readFrom()` remains a detached physical-suffix API. It neither creates nor consumes a preparation, synthesizes logical closers, or joins the LRU.
|
||||
- HMR adoption keeps the live Session authoritative and reads the stored prefix directly. It may truncate a torn physical fragment but never closes the live open turn as interrupted.
|
||||
- The cache belongs to one persistence coordinator, not a process-global Session map. Live Sessions are owned by the existing stores and never occupy preparation capacity.
|
||||
- A fresh create never claims a cold persisted preparation with the same id. Persistence collisions continue to reject.
|
||||
- Third-party persistence implementations retain the abstract `prepare()` fallback through `load()`. They receive the same publication interface but gain exact-object reuse only when they override preparation.
|
||||
- Revision validation establishes freshness at the reuse and repair-commit points; it does not add cross-process writer exclusion to a backend. Retries converge after the durable log remains unchanged for one read/check round trip, so continuous external writers can delay preparation.
|
||||
|
||||
## Verification
|
||||
|
||||
The shared persistence contract pins non-mutating balanced cold inspection and later repair. `persistence.spec.ts` and `preparations.spec.ts` pin same-id in-flight sharing, exact Session reuse across inspect and prepare, revision-triggered refresh before history and resume, single repair commit, exclusive reservation, release after failed setup, ready-entry LRU eviction, append rejection during reservation, and publication of only the reserved Session. Backend tests pin that full and lightweight reads use the same revision identity. Agent-loop and continuable-subagent tests pin the common publication pipeline and inspection-to-resume path across cancellation and teardown.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Activate an Agent for history reads.** Rejected because pagination would keep query-only Agents live and transfer cache retirement into the Agent lifecycle.
|
||||
|
||||
**Cache only `{ meta, events }`.** Rejected because resume would still reconstruct, validate, freeze, and copy a Session from the cached values. The exact unpublished Session is the reusable unit.
|
||||
|
||||
**Keep a process-global Session map.** Rejected because it would cross backend and runtime ownership boundaries, retain unbounded identities, and duplicate the live Session store.
|
||||
|
||||
**Add a restore transaction or coordinator to the Agent loop.** Rejected because cold reading, repair, reservation, and cursor attachment are persistence and Session concerns. The Agent loop only needs the uniform `SessionPreparation` ownership boundary.
|
||||
|
||||
**Turn `readFrom()` into logical preparation.** Rejected because watermark consumers need a detached physical suffix and, on seek-capable backends, a bounded read. Recovery balancing and whole-Session reuse have different semantics.
|
||||
|
||||
## Consequences
|
||||
|
||||
One cold materialization can serve history pagination, subagent descriptor inspection, and a later resume. Ownership transfer removes redundant restoration clones, while the bounded per-coordinator LRU limits memory and avoids creating live Agents for queries. Create and resume share one publication protocol without merging Agent and Session responsibilities.
|
||||
|
||||
The first cold inspection now pays the complete validation and Session-construction cost and may retain that unpublished Session until eviction. Persistence must coordinate reservation, append, repair, and publication, and callers must treat inspection values as immutable borrowed state. Backends that rely on the default `prepare()` remain correct but do not receive the reuse optimization.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Agent Note: 发布前可复用的 Session 准备阶段
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-05-session-preparation.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
冷历史检查和 agent（智能体）恢复会分别实体化同一份持久会话日志。对于大型压缩日志，每次操作都会重新完整读取、解压、解析、验证、冻结并构造 Session。因此，历史分页可能反复承担冷读成本；如果改为由历史查询激活 agent，读取生命周期又会与缺少自然退出时机的实时 agent 耦合。
|
||||
|
||||
新建和持久化恢复也通过不同构造流程抵达相同的发布边界。这使一项关键不变量不够清楚：设置必须基于一个未发布的 Session 完成，之后系统才能同时公开这个精确 Session 及其 agent。
|
||||
|
||||
## 决策
|
||||
|
||||
`SessionPreparation` 持有一个精确的未发布 `Session`，直至发布或回滚。它属于 Session 生命周期，不属于 agent 生命周期或激活机制。新建流程包装 `SessionStore.prepare()` 的结果；持久化恢复则从 `SessionPersistence.prepare()` 取得准备对象。
|
||||
|
||||
agent loop（智能体循环）通过同一条设置与发布流水线消费这两种形式：先取得准备对象，围绕 `preparation.session` 构建私有 agent 上下文，等待可选设置完成，再发布该精确 Session 和 agent，并在所有退出路径上 dispose 准备对象。发布后，实时生命周期由现有 Session 与 agent 存储接管；`SessionPreparation` 本身不负责任何 agent 行为。
|
||||
|
||||
该机制细化了 [agent 生命周期与所有权决策](2026-06-18-agent-lifecycle-and-ownership-seams.md)中的发布边界，但不替换其所有权模型。
|
||||
|
||||
## 持久化准备生命周期
|
||||
|
||||
使用协调器的持久化实现会将一个冷源加载为准备完成的 Session。后端转移新鲜、彼此无别名的元数据和事件，以及标识这些精确值的来源限定 revision；Session 恢复路径直接验证并冻结这些对象图，不再复制。协调器计算中断轮次的 closer，并且只构造一次精确的未发布 Session。其不可变 header 与平衡逻辑事件日志构成读取方借用的 `SessionInspection`，revision 则保留在持久化内部。
|
||||
|
||||
`inspect(id, signal?)` 不修改存储。合成 closer 只存在于准备完成的内存视图中，撕裂的物理尾部保持不变。同 id 调用方共享进行中的冷读。准备完成后，该对象可以进入每个协调器自己的 LRU；第一方后端可配置容量，默认保留五个。协调器复用保留源之前会读取该 id 的当前 revision；如果不匹配，就淘汰处于就绪阶段的源并重新完成冷实体化。已经进入提交或为恢复而预留的源仍由其所有者独占，因此并发检查会借用该不可变视图，直至发布或释放。
|
||||
|
||||
`prepare(id, signal?)` 独占预留准备完成的 Session。它先确认保留的 revision，再提交撕裂尾部和中断轮次修复、建立持久游标，最后返回可 dispose 的准备对象。陈旧源会被丢弃并重新读取，不会参与修复或发布。修复成功后也会丢弃修复前的源，并在预留前重新实体化已提交日志，以免把较新的 revision 关联到较旧的事件对象图。同 id 的另一个准备请求会等待当前预留发布或释放。发布只接受精确的预留 Session，并直接附接已提交游标，无需重建历史。设置失败或取消时，未发生变化的未发布 Session 会返回 LRU；发生变更或完成附接后，系统会消费该预留。
|
||||
|
||||
存量 `load(id)` API 使用相同的准备和修复机制，随后丢弃其预留并返回不可变逻辑视图。它保留为兼容 API，不承担历史到恢复的复用路径。该生命周期扩展了[共享持久化协调器](2026-06-18-shared-persistence-write-coordinator.md)，同时继续遵循[会话持久化决策](2026-06-14-session-persistence.md)所规定的存储与恢复规则。
|
||||
|
||||
## 历史与恢复复用
|
||||
|
||||
历史读取使用 `inspect()`，因此重复分页可以借用同一份不可变准备状态，而不会激活 agent。后续恢复调用 `prepare()`，直接取得检查阶段保留的精确 Session；系统不会再次完整读取、解压、解析、复制、验证或冻结日志。
|
||||
|
||||
如果持久日志在检查后发生变化，其 revision 也会变化。下一次历史读取或恢复会丢弃保留且处于就绪阶段的 Session，并实体化新日志，因此旧事件对象图不会被关联到较新的快照 revision。已经由进行中恢复操作取得的源不会被淘汰：其独占所有者会持有它直至发布或释放，并发历史读取可以借用同一个不可变视图。
|
||||
|
||||
冷 continuable subagent 访问沿用同一路径。系统先检查子会话并完成 descriptor 授权，再由 `ctx.agents.resume()` 预留并发布保留的 Session。这样既遵循 [continuable subagent 会话决策](../feature/2026-07-28-continuable-subagent-conversations.md)中的生命周期与授权规则，也消除了重复冷读。
|
||||
|
||||
## 边界
|
||||
|
||||
- `readFrom()` 仍是脱离的物理后缀 API。它不会创建或消费准备对象，不会合成逻辑 closer，也不会进入 LRU。
|
||||
- HMR（热模块替换）接管继续以实时 Session 为权威，并直接读取已存储前缀。它可以截断撕裂的物理碎片，但绝不把实时开放轮次关闭为中断状态。
|
||||
- 缓存属于单个持久化协调器，而不是进程全局 Session map。实时 Session 由现有存储持有，绝不占用准备容量。
|
||||
- 新建流程绝不认领相同 id 的冷持久化准备对象。持久化冲突仍会被拒绝。
|
||||
- 第三方持久化实现继续获得通过 `load()` 实现的抽象 `prepare()` 回退。它们使用相同发布接口，但只有覆盖准备流程后才能复用精确对象。
|
||||
- Revision 校验在复用点和修复提交点建立新鲜性，但不会为后端增加跨进程 writer 排他。持久日志在一次读取与复核往返内保持不变后，重试才能收敛，因此持续的外部写入可能延迟准备。
|
||||
|
||||
## 验证
|
||||
|
||||
共享持久化契约覆盖无变更且已配平的冷检查与后续修复。`persistence.spec.ts` 与 `preparations.spec.ts` 覆盖同 id 进行中读取共享、检查与准备之间的精确 Session 复用、在历史读取与恢复前由 revision 触发刷新、修复只提交一次、独占预留、设置失败后释放、就绪项 LRU 淘汰、预留期间拒绝 append，以及只允许发布预留 Session。后端测试覆盖完整读取与轻量读取使用同一 revision 身份。agent loop 与 continuable subagent 测试覆盖统一发布流水线，以及取消和拆卸期间从检查到恢复的路径。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**由历史读取激活 agent。** 不采用，因为分页会使仅用于查询的 agent 长期保持实时状态，并把缓存退出问题转移到 agent 生命周期。
|
||||
|
||||
**只缓存 `{ meta, events }`。** 不采用，因为恢复仍需从缓存值重新构造、验证、冻结并复制 Session。真正可复用的单元是精确的未发布 Session。
|
||||
|
||||
**维护进程全局 Session map。** 不采用，因为它会跨越后端和运行时所有权边界，无界保留身份，并与实时 Session 存储重复。
|
||||
|
||||
**在 agent loop 中增加恢复事务或协调器。** 不采用，因为冷读、修复、预留和游标附接都属于持久化与 Session 职责。agent loop 只需要统一的 `SessionPreparation` 所有权边界。
|
||||
|
||||
**把 `readFrom()` 改成逻辑准备流程。** 不采用，因为水位消费方需要脱离的物理后缀；对于可寻址后端，还需要限制实际读取范围。恢复平衡与完整 Session 复用具有不同语义。
|
||||
|
||||
## 后果
|
||||
|
||||
一次冷实体化可以同时服务历史分页、subagent descriptor 检查和后续恢复。所有权转移去除了恢复阶段的冗余复制；每个协调器的有界 LRU 限制内存占用，也避免查询创建实时 agent。新建和恢复共享同一发布协议，同时保持 agent 与 Session 职责分离。
|
||||
|
||||
首次冷检查需要承担完整验证与 Session 构造成本，并可能保留该未发布 Session 直至淘汰。持久化层必须协调预留、append、修复和发布；调用方必须把检查结果视为借用的不可变状态。依赖默认 `prepare()` 的后端仍然正确，但无法获得复用优化。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-06-coverage-uncovered-locations.md
|
||||
2026-08-06-coverage-uncovered-locations.md: 27c466e7ae505a2d318f8b21d442c31619610f59
|
||||
2026-08-06-coverage-uncovered-locations.zh.md: 93eeb863e51679c60d9e0ccfc742156c098aeca4
|
||||
@@ -0,0 +1,42 @@
|
||||
# Agent Note: Exact uncovered locations on coverage failure
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-06-coverage-uncovered-locations.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
When the per-file 100% coverage gate fails, vitest emits only file-level error lines (`ERROR: Coverage for lines (…) does not meet global threshold (100%) for <file>`) — you learn which file fell short, not which lines. The built-in `text` report does have an Uncovered Line #s column, but it is one giant table over hundreds of files repo-wide: the column truncates at the table width, carries line numbers but no column numbers, does not distinguish statements from branches from functions, and passing files occupy rows all the same. The net effect is that a red coverage run on CI is not directly actionable; the only way to locate the specific gap is to rerun the html report locally.
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/coverage-uncovered-locations.cjs` is a custom istanbul reporter (a `ReportBase` subclass): for every file below 100%, it emits one self-contained single-line record per uncovered statement, untaken branch path, and uncalled function — `<path>:<line>:<col> uncovered <kind> …` — directly clickable in terminals and CI logs, and easy to grep. When every file passes, it prints nothing. istanbul report generation runs before threshold validation, so the records land exactly above the existing ERROR lines.
|
||||
|
||||
The wiring is a single point: the coverage block in the root `vitest.config.ts` is the repo's only coverage configuration, shared by the CI lane (`run-gates ci-coverage`), local `test:coverage`, and focused runs (`--coverage.include`). The reporter joins both the CI and local reporter arrays by absolute path (`fileURLToPath`) — istanbul-reports' `create()` falls back to a bare `require(name)` for non-built-in names, and a relative path would resolve against istanbul's own package directory.
|
||||
|
||||
Output conventions:
|
||||
|
||||
- istanbul's 0-based column numbers are converted to 1-based (the convention editors and terminal links expect).
|
||||
- v8 reports `end.column = Infinity` for whole-line statements: a span crossing lines degrades to a `(to <line>)` suffix carrying only the line number, and a single-line span omits the suffix.
|
||||
- An implicit branch arm (such as a missing else) may carry no location; the reporter falls back to the branch's own span so the record stays clickable; branch records are annotated with the branch type and `path k/n`.
|
||||
- Records within a file are sorted by line, then column; there is no cap on the count.
|
||||
|
||||
Two companion changes: the root `package.json` adds `istanbul-lib-report` as a devDependency (under pnpm's strict layout, `scripts/` cannot reach nested dependencies); the root workspace's entry/project globs in `knip.json` gain `scripts/**/*.cjs`, making the file and its dependencies visible to the hygiene gate.
|
||||
|
||||
CJS is a forced shape, and a justified exception to the ESM-everywhere discipline: istanbul loads custom reporters via a bare `require()` outside the tsx/Vite pipeline, where TypeScript cannot participate; the namespace object `require(esm)` returns also fails its `new Cons(cfg)` construction — CommonJS is the only reliable shape.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Rely on the built-in `text` report's Uncovered Line #s column.** This is precisely the problem as found: one repo-wide table, column-width truncation, line numbers only, no kind distinction, passing files in the same column — not actionable in CI logs.
|
||||
- **Add a `json` reporter plus a separate wrapper script that reads `coverage-final.json` for post-processing after a failure.** Feasible in pure ESM/TS, but the wrapper would have to wrap both entry points — `package.json`'s `test:coverage` and the run-gates gate — changing their command shapes; the custom-reporter route touches one piece of configuration and takes effect at both entry points automatically.
|
||||
- **Write the reporter in TypeScript/ESM.** istanbul's loading mechanism (a bare `require` outside the pipeline) rules this out, as above; swapping out the loading mechanism for the sake of one report file is out of proportion.
|
||||
|
||||
## Verification
|
||||
|
||||
Local matrix: with a deliberately induced failure, all three record kinds appear and their locations match the planted gaps; a mixed run emits records only for the failing files (files at 100% within the same run stay silent); an all-green run produces zero output and exit code 0. CI evidence: after temporarily planting one unreachable statement/branch/function in `clampTimeout`, the coverage lane — under the isolated condition of all tests passing (632 files / 10326 cases) with only the threshold failing — printed the 4 records above the ERROR lines (evidence links in the PR #1716 description); the demonstration commit was removed from the branch before merge.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A red coverage run is self-sufficient: the log gives exact line and column numbers plus the kind of each gap, and rerunning the html report locally to pinpoint it is no longer needed.
|
||||
- The cost is one CJS-file discipline exception and one root devDependency; all-green runs produce zero output and add no log noise.
|
||||
- A file with zero coverage yields output on the order of its statement count (deliberately uncapped): the gate demands zero gaps, so the full listing is the action list, and vitest's own ERROR lines already provide the per-file summary as a backstop.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Agent Note: 覆盖率未达标时输出精确未覆盖位置
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-06-coverage-uncovered-locations.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
per-file 100% 覆盖率门禁失败时，vitest 只输出文件级错误行（`ERROR: Coverage for lines (…) does not meet global threshold (100%) for <file>`）——知道哪个文件没达标，不知道差在哪几行。内置 `text` 报表虽有 Uncovered Line #s 列，但它是全仓几百个文件的大表：该列按表宽截断、只有行号没有列号、不区分语句/分支/函数，且达标文件同样占行。结果是 CI 上的覆盖率红报不可直接行动，定位具体缺口只能本地重跑一遍 html 报表。
|
||||
|
||||
## Decision
|
||||
|
||||
`scripts/coverage-uncovered-locations.cjs` 是一个自定义 istanbul reporter（`ReportBase` 子类）：对每个低于 100% 的文件，按未覆盖语句、未走的分支路径、未调用函数各输出一条自含的单行记录 `<path>:<line>:<col> uncovered <kind> …`——terminal 与 CI 日志中可直接点击跳转，也便于 grep。全部文件达标时零输出。istanbul 报表生成先于 threshold 校验，因此记录恰好落在既有 ERROR 行上方。
|
||||
|
||||
接线是单点的：根 `vitest.config.ts` 的 coverage 块是全仓唯一覆盖率配置，CI lane（`run-gates ci-coverage`）、本地 `test:coverage` 与聚焦跑（`--coverage.include`）共用它。该 reporter 以绝对路径（`fileURLToPath`）加入 CI 与本地两个 reporter 数组——istanbul-reports 的 `create()` 对非内置名回退为裸 `require(name)`，相对路径会按 istanbul 自己的包目录解析。
|
||||
|
||||
输出约定：
|
||||
|
||||
- istanbul 的 0 基列号转为 1 基（编辑器与终端链接的约定）。
|
||||
- v8 对整行语句给出 `end.column = Infinity`：跨行时降级为只带行号的 `(to <line>)` 后缀，单行时省略后缀。
|
||||
- 隐式分支臂（如缺省 else）可能不带位置，回退到分支自身的 span，保证记录仍可点击；分支记录标注类型与 `path k/n`。
|
||||
- 同文件内记录按行、列排序；不设条数上限。
|
||||
|
||||
配套两处：根 `package.json` 增补 devDependency `istanbul-lib-report`（pnpm 严格布局下 `scripts/` 摸不到嵌套依赖）；`knip.json` 根 workspace 的 entry/project 通配增加 `scripts/**/*.cjs`，使该文件及其依赖对 hygiene 门禁可见。
|
||||
|
||||
CJS 是被迫的形态，也是 ESM-everywhere 纪律的一个有据例外：istanbul 在 tsx/Vite 管线之外用裸 `require()` 装载自定义 reporter，TypeScript 无法参与；`require(esm)` 返回的命名空间对象也过不了它的 `new Cons(cfg)` 构造，CommonJS 是唯一可靠形态。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **依赖内置 `text` 报表的 Uncovered Line #s 列。** 正是问题现状：全仓大表、列宽截断、只有行号、不分种类、达标文件同列——CI 日志里不可行动。
|
||||
- **加 `json` reporter，另写 wrapper 脚本失败后读 `coverage-final.json` 后处理。** 纯 ESM/TS 可行，但 wrapper 必须同时包住 `package.json` 的 `test:coverage` 与 run-gates 的 gate 两个入口，命令形状随之改变；自定义 reporter 路线只动一处配置，两个入口自动生效。
|
||||
- **用 TypeScript/ESM 写 reporter。** istanbul 的装载机制（管线外裸 `require`）决定了不可行，见上；为一个报表文件把装载机制换掉不成比例。
|
||||
|
||||
## Verification
|
||||
|
||||
本地矩阵：故意制造未达标时三类记录齐全、位置与埋点一致；混合现场只输出未达标文件（同跑内 100% 的文件静默）；全绿跑零输出、退出码 0。CI 实证：临时在 `clampTimeout` 埋入一处不可达语句/分支/函数后，coverage lane 在全部测试通过（632 文件 / 10326 用例）、仅 threshold 失败的隔离条件下，把 4 条记录打印在 ERROR 行上方（证据链接见 PR #1716 描述）；演示提交在合并前已从分支撤除。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 覆盖率红报自足：日志直接给出精确行列号与缺口种类，不再需要本地重跑 html 报表定位。
|
||||
- 代价是一个 CJS 文件的纪律例外与一个根 devDependency；全绿运行零输出，不增加日志噪音。
|
||||
- 整文件零覆盖时输出条数与该文件语句数同阶（刻意不设上限）：门禁要求零缺口，全量列出即是行动清单，vitest 自身的 ERROR 行已按文件汇总兜底。
|
||||
Reference in New Issue
Block a user