feat(session): default JSONL writes to packed rows

This commit is contained in:
Tianyi Cui
2026-07-26 23:44:52 +08:00
parent cad2af4741
commit 4ff496c65c
47 changed files with 521 additions and 251 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-06-14-session-persistence.md: 52434930bb662b0c97e61f7c2f69b67c309b6317
2026-06-14-session-persistence.zh.md: 143b58d32191108d7ba24b489bd4f898b1547aab
2026-06-14-session-persistence.md: 75e13b860f621ed407849b3b4c62ff7287ab4812
2026-06-14-session-persistence.zh.md: a6bd400a053779c742940236737447d1687622de

View File

@@ -15,11 +15,11 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append
Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic:
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**), encoded as [checksummed Zstandard frames by default](2026-07-19-zstandard-jsonl-session-logs.md) or raw lines by configuration.
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session: a `SessionHeader` line followed by storage records that losslessly represent the contiguous `SessionEvent` stream. Eligible `assistant/chunk` delta runs use packed rows by default; [checksummed Zstandard frames](2026-07-19-zstandard-jsonl-session-logs.md) are the default physical encoding, with raw lines configurable.
Key choices recorded here because they are durable, contested, and surprising:
- **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
- **The canonical durable log persists every `SessionEvent` losslessly, including `assistant/chunk`.** JSONL storage may encode a consecutive delta run as one packed row, but `load` reconstructs the exact event boundaries, sequence numbers, and timestamps. `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* logical log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, `load` preserves its contiguous, parseable events and appends risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }`. The synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable.
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)``append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows. Its database carries a dedicated application id and monotonic schema version. A pristine file creates all tables and stamps both header values in one transaction; an unversioned file with any user-defined schema object or application identity, a foreign current-version identity, and every non-current version reject before journal-mode mutation.
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. `createdAt` is non-negative safe-integer Unix epoch milliseconds: live creation and persistence registration reject fractional values, JSONL validates the decoded header, and SQLite stores it in a strict `INTEGER` column. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).)
@@ -33,4 +33,4 @@ Format versioning: the header carries a `version`; `load` rejects any non-curren
## Consequences
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only, contiguous-seq, lazy-materialization, integer-metadata, and serializability semantics. Persisting the full log also settles event fidelity: `assistant/chunk` remains verbatim. SQLite initialization either commits its complete owned schema and header identity or leaves no partial schema to strand on the next open.
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only, contiguous-seq, lazy-materialization, integer-metadata, and serializability semantics. Persisting the full logical log also settles event fidelity: every `assistant/chunk` survives exactly even when JSONL packs several into one storage row. SQLite initialization either commits its complete owned schema and header identity or leaves no partial schema to strand on the next open.

View File

@@ -15,11 +15,11 @@ Status: implemented
持久化是一个抽象的**能力 seam**[能力 seam](2026-06-13-capability-seams.md)`dsh-bash` 模板),而非循环或核心逻辑:
1. **接口**`dsh-session-persistence``ctx.sessionPersistence`):一个抽象的 `SessionPersistence` 服务,提供 `create`/`append`/`load`/`list`。其持久化单元就是现有的 `SessionEvent``{ type, seq, time, data }`),原样复用,无转换类型。
2. **实现**`dsh-session-persistence-jsonl`):每个会话一个仅追加的逻辑 JSONL 日志一行 `SessionHeader`之后每行一个 `SessionEvent`,逐字节保留,**包括 `assistant/chunk`**),默认编码为[带校验和的 Zstandard 帧](2026-07-19-zstandard-jsonl-session-logs.md),也可通过配置使用原始行。
2. **实现**`dsh-session-persistence-jsonl`):每个会话一个仅追加的逻辑 JSONL 日志:先是一行 `SessionHeader`随后是无损表示连续 `SessionEvent` 流的存储记录。符合条件的 `assistant/chunk` 增量连续段默认使用打包行;[带校验和的 Zstandard 帧](2026-07-19-zstandard-jsonl-session-logs.md)是默认物理编码,也可通过配置使用原始行。
以下关键选择记录于此,因为它们是持久性的、有争议的、且出人意料的:
- **规范的持久日志逐字节保留每个 `SessionEvent`,包括 `assistant/chunk`。** `deriveMessages()` 跳过分片而过滤分片的方案Codex 的 `policy.rs`)很有吸引力,但 `seq = log.length` 以及加载验证 `events[i].seq === i` 要求日志是*连续*的;过滤掉分片会留下空洞,同时破坏契约和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现(带有自己的重新编号),但它不是规范日志。
- **规范的持久日志无损保留每个 `SessionEvent`,包括 `assistant/chunk`。** JSONL 存储可以将一段连续的增量事件编码为一条打包行,但 `load` 会重建精确的事件边界、序号与时间戳。`deriveMessages()` 跳过分片而过滤分片的方案Codex 的 `policy.rs`)很有吸引力,但 `seq = log.length` 以及加载验证 `events[i].seq === i` 要求*连续*的逻辑日志;过滤掉分片会留下空洞,同时破坏契约和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现(带有自己的重新编号),但它不是规范日志。
- **仅追加;崩溃的轮次被关闭,而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.md)会在模型分发前排空请求、在工具分发前排空已记录的顶层调用,并在步骤结束后排空完整的响应/结果批次;循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作,`load` 保留其连续、可解析的事件,并为未应答的 assistant 调用追加按风险分类的错误结果、补一个缺失的 `step/end`,以及带 `{ kind: 'interrupted' }``turn/end`。合成的结果保证恢复后的提供方 transcript文本记录仍然有效。只有不完整的最后一条记录会被丢弃在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙,属于数据损坏,会使该会话不可加载。
- **文件后端为规范实现,数据库后端为经过验证的直接替换。** `SessionEvent` 1:1 映射到一行 `(session_id, seq, type, time, data)``append` 是 INSERT在一个断言连续 seq 契约的事务中),`load` 是 SELECT … ORDER BY seq。`dsh-session-persistence-sqlite` 正是如此:一个 `SessionPersistence` 子类接口无变化opencode 在 SQLite/WAL 上运行的正是这个形状),且通过与 JSONL 后端相同的 `runPersistenceContract` 测试套件。该契约以相同的语义约束两个后端(惰性物化、加载时关闭中断轮次、连续 seq一次表达在文件字节上一次表达在数据库行上。其数据库拥有专用的 application id 与单调递增的 schema 版本。系统会在一个事务中为全新文件创建所有表并写入这两个 header 值;未版本化文件若带有任何用户定义的 schema 对象或应用标识、当前版本文件若带有外部应用标识,以及任何非当前版本文件,都会在修改日志模式之前被拒绝。
- **元数据在日志之外。** 格式版本、cwd 和谱系是存储关注点,不是可回放的对话状态,因此它们存放在 `dsh-session` 拥有的 `SessionHeader` 中,并通过新的只读属性 `session.header` 附加到 `Session` 上——永远不进入 `SessionEventMap`,永远不到达 `deriveMessages()``createdAt` 是以 Unix epoch 毫秒表示的非负安全整数运行时创建和持久化注册会拒绝小数值JSONL 会验证解码后的 headerSQLite 则将其存入严格的 `INTEGER` 列。替代方案(一个可合并扩展的 `session/meta` 事件作为日志第 0 行)被否决:日志内事件会随 seed/fork 的会话免费携带,但元数据不是可回放状态,因此显式的日志外 header seam 是更干净的代价。header 最初被拆分为不可变的 `SessionHeader` 加可变的 `SessionSummary`,二者的联合类型为 `SessionMeta`;可变 summary 后来因属于死状态而被移除——见 [移除可变会话摘要](../simplification/2026-06-19-drop-mutable-session-summary.md)。)
@@ -33,4 +33,4 @@ Status: implemented
## 后果
新增两个包package以及 `dsh-session` 中的元数据 seam`session.header``create(id?, options?)` 签名)。收益:持久恢复/fork、读取/回放路径、崩溃容忍,以及基于现有事件溯源日志的宿主侧会话访问,后端在一个接口之后可替换。可复用的 `runPersistenceContract` 测试套件以相同的仅追加、连续 seq、惰性物化、整数元数据与可序列化语义约束每个后端。持久化完整日志还确定了事件保真度`assistant/chunk` 保持逐字节不变。SQLite 初始化要么提交完整的自有 schema 与 header 标识,要么不留下任何会使下次打开受阻的部分 schema。
新增两个包package以及 `dsh-session` 中的元数据 seam`session.header``create(id?, options?)` 签名)。收益:持久恢复/fork、读取/回放路径、崩溃容忍,以及基于现有事件溯源日志的宿主侧会话访问,后端在一个接口之后可替换。可复用的 `runPersistenceContract` 测试套件以相同的仅追加、连续 seq、惰性物化、整数元数据与可序列化语义约束每个后端。持久化完整的逻辑日志还确定了事件保真度:即使 JSONL 将多个 `assistant/chunk` 打包到一条存储行中,每个事件也会精确保留。SQLite 初始化要么提交完整的自有 schema 与 header 标识,要么不留下任何会使下次打开受阻的部分 schema。

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-07-26-packed-chunk-rows-by-default.md: e1090264238ff15670a58ee33b062ad340241b8e
2026-07-26-packed-chunk-rows-by-default.zh.md: b193e37987946764d6c19583f2e3f195ae31bf61

View File

@@ -0,0 +1,59 @@
# Agent Note: Make packed chunk rows the default JSONL layout
Status: implemented
English | [中文](2026-07-26-packed-chunk-rows-by-default.zh.md)
## Problem
Provider streams produce many token-sized `assistant/chunk` delta events whose repeated JSON envelopes can outweigh their payloads. The session log must retain each chunk as a distinct logical event: live `session/event` delivery, sequence numbers, `sourceEventSeqs`, replay, cancellation evidence, and UI streaming all depend on those boundaries.
The JSONL storage seam can reduce that envelope cost without changing the logical log. A run of at least three consecutive same-block delta events fits in one `text-chunks`, `reasoning-chunks`, or `tool-call-chunks` storage row, and decoding reconstructs every original event, timestamp, and sequence number. A credible default must cover runtime writers, app-level config, snapshot producers, and committed fixtures together; otherwise tests avoid the layout that deployments write.
## Decision
`dsh-session-persistence-jsonl` resolves an omitted `packChunks` to `true`. The ACP demo wrapper exposes the same default, and every composition that omits the field inherits packed writes. `packChunks: false` remains an explicit write-side diagnostic mode that stores one event per line.
Reading is unconditional and layout-blind. Packed, unpacked, and mixed files load into the same contiguous `SessionEvent[]`, so the default does not require a session-format version change or an on-disk runtime migration. The option controls newly appended batches only; it never selects a reader mode.
### Logical events and physical rows
Packing stays at the `dsh-session` storage seam through `packChunkRuns()` and `decodeStorageRecord()`. The encoder recognizes exact delta-event shapes, preserves unrecognized events verbatim, and packs only runs of at least three. A packed row is storage vocabulary, not a `SessionEventMap` member: it never enters `Session.events` or fires `session/event`.
The JSONL backend packs each durable append batch. Raw `compression: 'none'` and default Zstandard framing carry the same logical storage records; selecting raw mode for reviewable fixtures does not disable packing. Repository replay readers and normalizers decode the shared row format instead of maintaining snapshot-specific codecs.
### Canonical snapshot fixtures
Every committed session-format JSONL fixture uses the canonical packed representation. `scripts/session-fixture-layout.snapshot.ts` discovers tracked `*.jsonl` files and unignored untracked additions repository-wide, selects those whose first record is a `session` header, decodes all body records, and rejects content that differs from `packChunkRuns()` output. The inventory therefore includes ACP, headless, TUI, `apps/web`, parent sessions, child sessions, and future fixture names without a maintained path list.
ACP and headless snapshot runs harvest the default JSONL backend output. TUI and web record-mode writers apply `packChunkRuns()` to their in-memory events before writing fixtures. The authored `packed-chunks` ACP scenario runs under the ordinary config and retains all three packed row kinds; its contract decodes both its independent source fixture and target fixture before asserting event-for-event equality.
Focused package tests keep unpacked and mixed-layout inputs for reader compatibility. They do not opt the default snapshot corpus out of the canonical layout.
### In-flight branch convergence
The temporary [`scripts/migrate-packed-session-fixtures.ts`](../../../../scripts/migrate-packed-session-fixtures.ts) command lets in-flight branches converge after merging current `master`: `pnpm run migrate:packed-session-fixtures` discovers the same repository-wide fixture set as the permanent gate, preserves each header line, decodes existing mixed records, writes the canonical packed body, proves decoded equality, and proves idempotence. It never calls a model or regenerates transcript and presentation outputs.
The command remains linked from the testing policy and ACP snapshot README while older branches may carry fixture edits. The [removal proposal](../../proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) deletes the CLI, package command, this transitional section, and the documentation links once a live open-PR inventory shows that every affected branch is merged, closed, or canonical. The shared canonicalizer and snapshot gate remain permanent.
### Verification contract
JSONL persistence tests prove that omission writes a packed row, explicit `false` writes one event per line, and both forms load identical events. Canonicalizer unit tests cover header preservation, unpacked conversion, non-session JSONL, already-packed idempotence, and malformed input. The keyless snapshot gate covers every committed fixture and assembled replay path; documentation gates keep config defaults and bilingual contracts aligned.
## Alternatives considered
**Flip only the backend schema default.** This leaves wrapper defaults, direct TUI/web serializers, existing fixtures, and future fixture policy inconsistent. A default is meaningful only when shipping compositions and the tests representing them share it.
**Keep snapshots unpacked for readability.** Packed rows retain every fragment and timestamp explicitly, while the shared decoder and normalizer provide logical inspection. Keeping the largest committed consumer on a different layout would make snapshot coverage avoid the shipping write path.
**Remove `packChunks` and always pack.** One writer is simpler, but one-event-per-line output remains useful for diagnostics and for focused mixed-layout compatibility tests. The explicit opt-out preserves those current consumers without weakening the default.
**Batch chunks as logical session events.** This reduces event count, but it delays or reshapes live delivery, renumbers provenance, and requires every UI and replay consumer to understand another streaming unit. Physical packing obtains the storage benefit behind the existing persistence interface.
**Keep the branch migrator permanently.** The read-only canonicalizer and snapshot gate own continuing enforcement. A mutation command has value only while in-flight branches still carry the former fixture layout, so its lifetime is explicitly bounded by the removal proposal.
## Consequences
Ordinary JSONL writes and committed fixtures use fewer physical rows while preserving the exact logical event stream. Runtime readers accept every existing layout, and operators retain a deliberate unpacked diagnostic mode. Raw files are less convenient for per-token line processing, and external tools that incorrectly treat every post-header row as a `SessionEvent` encounter storage tags more often; supported readers call `decodeStorageRecord()`.
The repository carries a large mechanical fixture diff, reviewed through decoded equality and the canonical-layout gate rather than token-by-token line inspection. It also temporarily carries one branch migration command and its links; the separate removal proposal prevents that transition aid from becoming permanent process surface.

View File

@@ -0,0 +1,59 @@
# Agent Note: 将打包分片行设为默认 JSONL 布局
Status: implemented
[English](2026-07-26-packed-chunk-rows-by-default.md) | 中文
## 问题
提供方流会产生大量 token 大小的 `assistant/chunk` 增量事件,其重复 JSON 封装可能比载荷本身更大。会话日志必须将每个分片保留为独立的逻辑事件:实时 `session/event` 传递、序号、`sourceEventSeqs`、回放、取消证据和 UI 流式输出都依赖这些边界。
JSONL 存储 seam 可以在不改变逻辑日志的情况下减少这部分封装开销。一段至少包含 3 个连续、同属一个块的增量事件可以编码为一条 `text-chunks``reasoning-chunks``tool-call-chunks` 存储行,解码则会重建每个原始事件、时间戳和序号。一个可信的默认值必须同时覆盖运行时写入器、应用级配置、快照生成器和签入仓库的 fixture测试前置数据否则测试会绕开部署实际写入的布局。
## 决策
`dsh-session-persistence-jsonl` 会将省略的 `packChunks` 解析为 `true`。ACPAgent Client Protocol演示包装层公开相同的默认值所有省略该字段的组合都会继承打包写入。`packChunks: false` 仍是写入侧显式诊断模式,以每事件一行的形式存储。
读取始终不受选项控制且与布局无关。打包、非打包和混合文件都会加载为相同且连续的 `SessionEvent[]`,因此更改默认值不需要变更会话格式版本,也不需要对磁盘数据执行运行时迁移。该选项只控制新追加的批次,绝不会选择读取器模式。
### 逻辑事件与物理行
打包保留在 `dsh-session` 的存储 seam并通过 `packChunkRuns()``decodeStorageRecord()` 实现。编码器识别精确的增量事件形态,原样保留无法识别的事件,并且只打包至少包含 3 个事件的连续段。打包行属于存储词汇,不是 `SessionEventMap` 成员:它绝不会进入 `Session.events`,也不会触发 `session/event`
JSONL 后端会打包每个持久追加批次。原始模式 `compression: 'none'` 与默认 Zstandard 帧承载相同的逻辑存储记录;为使 fixture 便于评审而选择原始模式,不会禁用打包。仓库中的回放读取器和规范化器会解码共享行格式,而不维护快照专用编解码器。
### 规范快照 fixture
每个签入仓库的会话格式 JSONL fixture 都使用规范打包表示。`scripts/session-fixture-layout.snapshot.ts` 会在整个仓库中发现已跟踪的 `*.jsonl` 文件,以及未被忽略的新增未跟踪 JSONL 文件,选择首条记录为 `session` header 的文件,解码所有正文记录,并拒绝与 `packChunkRuns()` 输出不同的内容。因此,该清单无需维护路径列表即可覆盖 ACP、headless、TUI、`apps/web`、父会话、子会话以及未来的 fixture 名称。
ACP 和 headless 快照运行会采集默认 JSONL 后端的输出。TUI 和 web 的记录模式写入器会在写入 fixture 前,对内存事件应用 `packChunkRuns()`。人工编写的 `packed-chunks` ACP 场景在普通配置下运行,并保留全部 3 种打包行类型;其契约先解码独立的源 fixture 和目标 fixture再断言二者逐事件相等。
聚焦的包package测试保留非打包和混合布局输入以验证读取器兼容性。这些测试不会让默认快照语料库豁免规范布局要求。
### 在途分支收敛
临时命令 [`scripts/migrate-packed-session-fixtures.ts`](../../../../scripts/migrate-packed-session-fixtures.ts) 让在途分支合并当前 `master` 后可以完成收敛:`pnpm run migrate:packed-session-fixtures` 会发现与永久门禁相同的仓库级 fixture 集合,保留各文件的 header 行,解码现有混合记录,写入规范打包正文,并证明解码结果相等且操作具有幂等性。该命令绝不会调用模型,也不会重新生成 transcript文本记录与呈现输出。
只要较旧分支仍可能携带 fixture 改动,测试政策和 ACP 快照 README 就会继续链接该命令。最新的开放 PRPull Request清单确认每个受影响分支均已合并、关闭或符合规范后[移除提案](../../proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md)会删除该 CLI、包命令、本过渡章节和文档链接。共享规范布局转换器与快照门禁保持永久存在。
### 验证契约
JSONL 持久化测试证明:省略选项时会写入打包行,显式传入 `false` 时会按每事件一行的形式写入,两种形式都会加载为完全相同的事件。规范布局转换器单元测试覆盖 header 保留、非打包转换、非会话 JSONL、已打包输入的幂等性和畸形输入。无密钥快照门禁覆盖每个签入仓库的 fixture 和组装后的回放路径;文档门禁则确保配置默认值与双语契约保持一致。
## 曾考虑的替代方案
**仅翻转后端 schema 默认值。** 这会让包装层默认值、TUI/web 直接序列化器、现有 fixture 与未来 fixture 政策仍然彼此不一致。只有已交付组合及代表这些组合的测试采用相同默认值时,该默认值才有意义。
**快照继续使用非打包格式以便阅读。** 打包行仍会显式保留每个片段和时间戳,共享解码器与规范化器则提供逻辑检查。如果让规模最大的签入仓库消费方采用不同布局,快照覆盖就会绕开已交付的写入路径。
**删除 `packChunks` 并始终打包。** 只保留一个写入器更简单,但每事件一行的输出仍适用于诊断和聚焦的混合布局兼容性测试。显式停用选项在不削弱默认值的同时,保留了这些现有消费方。
**把分片批量合并为逻辑会话事件。** 这会减少事件数量,但也会延迟或重塑实时传递,改变溯源信息所引用的序号,并要求每个 UI 和回放消费方理解另一种流式单位。物理打包通过现有持久化接口获得存储收益。
**永久保留分支迁移器。** 只读的规范布局转换器与快照门禁负责持续强制执行。只有在途分支仍携带旧 fixture 布局时,会修改仓库内容的命令才有价值,因此移除提案明确限定了其生命周期。
## 后果
常规 JSONL 写入与签入仓库的 fixture 使用更少的物理行,同时精确保留逻辑事件流。运行时读取器接受所有现有布局,操作方也保留有意提供的非打包诊断模式。按 token 逐行处理原始文件较为不便;错误地将 header 后每一行都视为 `SessionEvent` 的外部工具会更频繁地遇到存储 tag受支持的读取器则会调用 `decodeStorageRecord()`
仓库会产生大规模机械 fixture diff评审应依据解码结果相等这一事实和规范布局门禁而不是逐行、逐 token 检查。仓库还会暂时保留一个分支迁移命令及其链接;单独的移除提案会防止这项过渡辅助机制成为永久的流程接口。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write
2026-06-19-acp-snapshot-tests.md: b4cda8f32fe7a84a977bcbdbe5db0671cb9a7083
2026-06-19-acp-snapshot-tests.zh.md: 5337c3852b524af4e8c556e93ec80084b30a6d0b
2026-06-19-acp-snapshot-tests.md: 6e9f07cd65069423a61f94af44713054470395c6
2026-06-19-acp-snapshot-tests.zh.md: 95c2ef6dd55d70202a9985c824c508c2dda42c00

View File

@@ -20,7 +20,7 @@ A snapshot test boots the real ACP example, drives its stdio protocol from a det
Each scenario's `session.jsonl` is harvested from a real run. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary session artifact therefore serves as both replay source and behavioral expected output.
When a scenario pins an alternative physical storage layout, its fixture is mechanically derived from a real unpacked counterpart. The scenario test requires every intended storage-row kind and exact event-for-event equality after decoding before the ordinary replay and log comparison proves that the assembled process consumes and reproduces that layout.
Every committed session-format fixture uses the canonical packed physical layout. The all-row-kinds scenario is mechanically derived from an independent real recording; its test requires every packed storage-row kind and exact event-for-event equality after both fixtures decode, then ordinary replay and log comparison prove that the assembled process consumes and reproduces the layout.
### Replay derives the model script from the log
@@ -44,7 +44,7 @@ Replay is positional and therefore permits only one in-flight model stream per s
### Recording harvests the log; keyless replay needs a providerless config
Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend configured with `persistenceCompression: 'none'`, then copies the produced `.jsonl` into the scenario dir. The explicit raw mode keeps committed replay fixtures line-readable while ordinary deployments use the backend's compressed default. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only.
Recording runs the scenario with the real `llm-deepseek` adapter and the JSONL persistence backend configured with `persistenceCompression: 'none'`, then copies the produced `.jsonl` into the scenario dir. The explicit raw mode keeps committed replay fixtures line-readable while ordinary deployments use the backend's compressed default; eligible chunk runs still use the default packed storage rows. Per-event appends are durable, but the harness shuts the subprocess down gracefully (close stdin → `await ctx.dispose()`) before harvesting so the final events are flushed. `llm-replay` itself does no recording — it is replay-only.
Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with `llm-replay` while retaining the live composition. Recording uses the ordinary config and a harness-supplied persistence root. Replay mode skips `.env` loading, so a stray API key cannot trigger a live call. See the [single-source config Agent Note](2026-07-04-single-source-acp-replay-config.md).
@@ -69,7 +69,7 @@ Tool determinism comes from a generated cwd, scrubbed environment, fresh non-log
### Two subcommands, replay in the default gate
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout expected output. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the harvested session log and stdout expected output. The same keyless gate discovers repository JSONL by its `session` header and rejects any fixture that differs from the shared codec's canonical packed representation. Missing fixtures fail loud. Every scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. `replay.override.json` is required only for scenarios marked `overridden`, because its presence replaces derived replay. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters.
## Alternatives considered

View File

@@ -20,7 +20,7 @@ Status: implemented
每个场景的 `session.jsonl` 都从真实运行中采集。`assistant/chunk` 事件复现模型流;工具、消息和边界事件捕获 harness 行为。因此,一份普通会话产物同时充当重放来源和行为预期输出。
当场景固定另一种物理存储布局时,其 fixture 会从真实的未打包对应项机械派生。场景测试要求包含每一种预期存储行类型,并在解码后逐事件精确相等;随后,普通重放与日志比较会证明组后的进程能够消费并复现该布局。
每个签入仓库的会话格式 fixture 都使用规范的打包物理布局。覆盖所有行类型的场景从一份独立的真实录制机械派生测试要求包含每一种打包存储行类型,并在两份 fixture 解码后逐事件精确相等;随后,普通重放与日志比较会证明组后的进程能够消费并复现该布局。
### 回放从日志推导模型脚本
@@ -44,7 +44,7 @@ Status: implemented
### 录制采集日志;无密钥回放需要无提供方的配置
记录模式使用真实 `llm-deepseek` 适配器和配置为 `persistenceCompression: 'none'` 的 JSONL 持久化后端运行场景,再把生成的 `.jsonl` 复制到场景目录。显式 raw 模式让已提交重放 fixture 保持逐行可读,而普通部署使用后端的压缩默认值。逐事件追加具有持久性,但 harness 会在采集前优雅关闭子进程(关闭 stdin → `await ctx.dispose()`),以确保最终事件已刷出。`llm-replay` 本身不执行记录——它只负责重放。
记录模式使用真实 `llm-deepseek` 适配器和配置为 `persistenceCompression: 'none'` 的 JSONL 持久化后端运行场景,再把生成的 `.jsonl` 复制到场景目录。显式 raw 模式让已提交重放 fixture 保持逐行可读,而普通部署使用后端的压缩默认值;符合条件的分片连续段仍使用默认的打包存储行。逐事件追加具有持久性,但 harness 会在采集前优雅关闭子进程(关闭 stdin → `await ctx.dispose()`),以确保最终事件已刷出。`llm-replay` 本身不执行记录——它只负责重放。
重放使用 `cordis.snapshot.yml` overlay`llm-replay` 替换真实适配器,同时保留实时组合。记录使用普通配置和由 harness 提供的持久化根目录。重放模式跳过 `.env` 加载,因此意外存在的 API 密钥不会触发实时调用。参见[单一来源配置 Agent Note](2026-07-04-single-source-acp-replay-config.md)。
@@ -69,7 +69,7 @@ Status: implemented
### 两个子命令,回放在默认门禁中
`pnpm run test:snapshot` 无需密钥即可重放已提交 fixture`test:snapshot:record` 使用真实 API并重写采集的会话日志与 stdout 预期输出。缺少 fixture 时会响亮失败。每个场景都包含 `input.json``stdout.expected.jsonl``session.jsonl`;不调用模型的情况使用仅有请求头的日志。只有标记为 `overridden` 的场景才需要 `replay.override.json`因为它一旦存在就会取代派生重放。Fixture 守卫会拒绝缺失、不匹配和孤立文件。两个命令都接受场景过滤器。
`pnpm run test:snapshot` 无需密钥即可重放已提交 fixture`test:snapshot:record` 使用真实 API并重写采集的会话日志与 stdout 预期输出。同一无密钥门禁会通过 `session` header 发现仓库中的 JSONL并拒绝与共享编解码器的规范打包表示不同的任何 fixture。缺少 fixture 时会响亮失败。每个场景都包含 `input.json``stdout.expected.jsonl``session.jsonl`;不调用模型的情况使用仅有请求头的日志。只有标记为 `overridden` 的场景才需要 `replay.override.json`因为它一旦存在就会取代派生重放。Fixture 守卫会拒绝缺失、不匹配和孤立文件。两个命令都接受场景过滤器。
## 曾考虑的替代方案