feat(session): bound persistence write batching

This commit is contained in:
Tianyi Cui
2026-08-08 15:43:52 +08:00
parent 7ab20890e6
commit 924c954469
52 changed files with 763 additions and 126 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md
2026-07-23-collapse-persistence-flush-state.md: a9b0f6847712f47d46adb6b01c57563033738964
2026-07-23-collapse-persistence-flush-state.zh.md: e665d164fb66b2bce97a4f6ded88f2ee07324e61
2026-07-23-collapse-persistence-flush-state.md: 2801079e5b322d076eced3a0113e7958d9b4c3b9
2026-07-23-collapse-persistence-flush-state.zh.md: dfc8fc9576416b1f51f4f9c978f49aee4f39f938

View File

@@ -4,15 +4,17 @@ Status: implemented
English | [中文](2026-07-23-collapse-persistence-flush-state.zh.md)
The [bounded write-batching decision](../architecture/2026-08-08-bounded-session-persistence-write-batching.md) supersedes this note's immediate scheduling cadence. The single-controller ownership, failure retention, per-id serialization, retirement, and quiescent-disposal decisions remain current.
## Problem
The persistence coordinator represented one live session's write lifecycle with separate buffer, initialization, and retirement containers plus the per-id operation chain. Those structures mirrored the same fact: whether that exact `Session` still had initialization or events that must settle before its state could be released. The checkpoint-only drain also kept every event volatile until another plugin requested `session/flush`, even though the backend could begin durability work without blocking the synchronous producer.
## Decision
Each live `Session` has one controller containing `pending`, `init`, and the optional current `flush` promise. A `session/event` listener copies the frozen event into `pending` and immediately schedules `ensureFlush()`. Calls during an active write reuse the same promise. The drain snapshots one stable pending prefix and removes it only after `appendBatch` commits; events admitted during the write remain after that prefix and schedule one follow-up batch.
Each live `Session` has one lifecycle entry containing initialization and one package-private write controller. The controller owns `pending`, the fixed batching timer, the optional active write, automatic-retry pause, and the shared flush barrier. A `session/event` listener copies the frozen event into `pending`; the first event starts a fixed deadline, and later events join without resetting it. A write takes one stable pending prefix; events admitted while it runs remain pending for a separately bounded follow-up batch.
`session/flush` is an observation barrier. It waits for initialization and repeatedly awaits or starts the controller's flush until neither a current promise nor pending events remain. An eager failure is logged without rejecting the synchronous event producer, retains the complete batch, and is retried by the next explicit flush, retirement attempt, or backend teardown. Explicit flush and teardown still surface the failure if that retry rejects.
`session/flush` is an immediate quiescence barrier. It waits for initialization, cancels the batching timer, joins any active attempt, and drains pending events, including events admitted while the barrier runs. A background failure is logged without rejecting the synchronous event producer, restores the complete ordered batch, and pauses automatic retry. A new event starts a fresh fixed window; explicit flush, retirement, or backend teardown retries immediately and surfaces a repeated failure.
Initialization now enters the existing per-id operation chain once and calls the unserialized core operations while it owns that turn. The chain remains separate from the live controller because detached public `create`/`append`/`load` calls can race without a `Session` object and still require identity-level serialization.
@@ -22,17 +24,17 @@ The live-controller map is also the retirement registry. Successful retirement d
## Alternatives considered
**Keep checkpoint-only write-behind.** This can form larger batches, but makes durability depend on a separately mounted checkpoint policy and maximizes the crash-loss window between checkpoints. Eager scheduling still coalesces synchronous bursts and events arriving during an active write.
**Keep checkpoint-only write-behind.** This can form larger batches, but makes durability depend on a separately mounted checkpoint policy and maximizes the crash-loss window between checkpoints. Bounded background scheduling still coalesces bursts and persists progress between mandatory barriers.
**Use one coordinator-wide flush promise.** The attachment pattern works for one file, but a global promise would serialize unrelated sessions. One controller per live session preserves independent backend progress while the per-id chain protects same-identity operations.
**Latch the first eager error permanently.** This makes every later flush deterministic, but prevents the existing teardown retry from recovering a transient storage failure. Retaining the batch without latching the error preserves both observability and retry.
**Latch the first background error permanently.** This makes every later flush deterministic, but prevents the existing teardown retry from recovering a transient storage failure. Retaining the batch without latching the error preserves both observability and retry.
**Reject every live load.** This is safe but removes established balanced live snapshots used by persistence consumers and tests. Snapshot-before-flush gives the call a stable linearization point: successful flush proves exactly that snapshot is durable, while the live path never invokes crash repair.
## Verification
- A focused coordinator test gates the first append, admits another event during that write, and observes an automatic second durable batch without calling `session/flush`.
- Focused controller tests use a fake clock to prove the non-resetting fixed window, gate the first append, admit another event during that write, and observe an automatic second durable batch without calling `session/flush`.
- The shared coordinator contract still covers live adoption, collisions, crash repair, and session/backend disposal over the in-memory, JSONL, and SQLite backends.
- Failure and teardown tests keep rejected batches pending, retry them before close, and prove an in-flight controller delays backend close.
- The shared backend contract persists an open live turn, proves `load` rejects without writing synthetic closers, completes and retires the owner, then reloads the exact completed turn.
@@ -42,6 +44,6 @@ The live-controller map is also the retirement registry. Successful retirement d
## Consequences
The coordinator has three long-lived containers: persisted identity state, live-session controllers, and per-id operation chains. Eager writes reduce the ordinary crash-loss window and remove separate buffer, initialization, and retirement registries. They can produce more backend batches than checkpoint-only draining; same-tick bursts and events admitted during one write still coalesce.
The live-session entry keeps initialization beside one controller for pending events, its timer, active write, retry pause, and flush barrier. Separate coordinator containers retain persisted identity state, prepared cold Sessions, retiring identity waiters, and per-id operation chains because those lifecycles also exist without one writable live Session. Bounded writes reduce the ordinary crash-loss window and produce fewer backend batches than immediate scheduling, while mandatory barriers remain unchanged.
`session/flush` no longer chooses when ordinary persistence begins. It remains the ordering and error-observation boundary used by the loop and checkpoint policy, so a successful checkpoint still means every event admitted before its completion is durable.

View File

@@ -4,15 +4,17 @@ Status: implemented
[English](2026-07-23-collapse-persistence-flush-state.md) | 中文
[有界写入批处理决策](../architecture/2026-08-08-bounded-session-persistence-write-batching.md)取代了本 Agent Note 中的即时调度节奏。单控制器归属、失败保留、按 id 串行化、退役和完全停稳的资源释放决策仍然有效。
## 问题
持久化协调器使用彼此独立的缓冲区、初始化容器和退役容器,以及按 id 划分的操作链,表示一个活跃会话的写入生命周期。这些结构反映的是同一个事实:该 `Session` 是否仍有初始化操作或事件必须完成,之后才能释放其状态。仅由检查点触发的排空还会让每个事件都停留在易失状态,直至另一个插件请求 `session/flush`,尽管后端可以在不阻塞同步生产方的情况下开始持久化工作。
## 决策
每个活跃的 `Session` 都有一个控制器,其中包含 `pending`、`init` 和可选的当前 `flush` promise。`session/event` 监听器将冻结的事件复制到 `pending`,并立即调度 `ensureFlush()`。活跃写入期间的调用复用同一个 promise。排空操作会对待处理事件中一个稳定的前缀生成快照,并且只在 `appendBatch` 提交后移除该前缀;写入期间接纳的事件保留在该前缀之后,并调度一个后续批次。
每个活跃的 `Session` 都有一个生命周期条目,其中包含初始化和一个包私有写入控制器。该控制器负责 `pending`、固定批处理计时器、可选的活跃写入、自动重试暂停和共享 flush 屏障。`session/event` 监听器将冻结的事件复制到 `pending`;第一个事件设定固定截止时间,后续事件加入但不会重置。一次写入会取出一个稳定的待处理前缀;写入期间接纳的事件仍留在待处理队列中,并形成另一个独立有界的后续批次。
`session/flush` 是观测屏障。它等待初始化完成,并反复等待或启动控制器的刷新,直至当前 promise 和待处理事件均不存在。即时写入失败会被记录,但不会拒绝同步事件生产方;完整批次会保留下来,由下一次显式刷新、退役尝试或后端资源销毁重试。若该次重试仍失败,显式刷新和资源销毁仍会向调用方暴露失败。
`session/flush` 是即时完全停稳屏障。它等待初始化完成、取消批处理计时器、等待任何活跃尝试完成,并排空待处理事件,包括屏障运行期间接纳的事件。后台写入失败会被记录,但不会拒绝同步事件生产方;系统会恢复完整且顺序不变的批次,并暂停自动重试。新事件会开启新的固定窗口;显式 flush、退役或后端资源销毁会立即重试,并在失败再次发生时向调用方暴露失败。
初始化只进入现有的按 id 操作链一次,并在占有该轮执行权时调用未串行化的核心操作。该操作链与活跃控制器保持分离,因为公共 `create`、`append`、`load` 调用即使没有 `Session` 对象仍可能发生竞态,依然需要按标识串行执行。
@@ -22,17 +24,17 @@ Status: implemented
## 备选方案
**保留仅由检查点触发的延后写入。** 这种方式可以形成更大的批次,但会让持久性依赖另行挂载的检查点策略,并使检查点之间因崩溃而丢失数据的窗口达到最大。即时调度仍会合并同步突发事件,以及活跃写入期间到达的事件。
**保留仅由检查点触发的延后写入。** 这种方式可以形成更大的批次,但会让持久性依赖另行挂载的检查点策略,并使检查点之间因崩溃而丢失数据的窗口达到最大。有界后台调度仍会合并突发事件,并在强制屏障之间持久化进度。
**在整个协调器范围内使用一个刷新 promise。** 这种挂接方式适用于单个文件,但全局 promise 会串行化互不相关的会话。每个活跃会话各有一个控制器,既能让不同会话的后端操作独立推进,又由按 id 操作链保护同一标识的操作。
**永久锁存首次即时写入错误。** 这会让后续每次刷新都得到确定的结果,却会阻止现有的资源销毁重试从暂时性存储故障中恢复。保留批次但不锁存错误,可以同时保留可观测性和重试能力。
**永久锁存首次后台写入错误。** 这会让后续每次刷新都得到确定的结果,却会阻止现有的资源销毁重试从暂时性存储故障中恢复。保留批次但不锁存错误,可以同时保留可观测性和重试能力。
**拒绝对所有活跃会话的加载。** 这样做很安全,但会让持久化消费方和测试无法再使用既有的闭合活跃会话快照。先生成快照再刷新,为调用提供了稳定的线性化点:刷新成功即可证明正是该快照已持久化,而活跃路径绝不调用崩溃修复。
## 验证
- 一个针对协调器的测试会阻塞第一次追加,在该次写入期间接纳另一个事件,并在不调用 `session/flush` 的情况下观测到自动执行的第二个持久批次。
- 针对控制器的测试使用假时钟证明固定窗口不会重置,随后阻塞第一次追加,在该次写入期间接纳另一个事件,并在不调用 `session/flush` 的情况下观测到自动执行的第二个持久批次。
- 共享协调器契约仍覆盖内存、JSONL 和 SQLite 后端上的活跃会话接管、冲突、崩溃修复,以及会话和后端的资源释放。
- 失败和资源销毁测试会让写入失败的批次保持待处理,在关闭前重试这些批次,并证明尚在执行的控制器会延迟后端关闭。
- 共享后端契约会持久化一个仍打开的活跃轮次,证明 `load` 会拒绝且不会写入合成闭合事件,随后完成该轮次并让其所有者退役,最后重新加载完全相同的已完成轮次。
@@ -42,6 +44,6 @@ Status: implemented
## 后果
协调器有三个长生命周期容器:持久化的标识状态、活跃会话控制器和按 id 操作链。即时写入缩短了通常情况下因崩溃而丢失数据的窗口,并移除了彼此独立的缓冲区、初始化注册表和退役注册表。与仅由检查点触发的排空相比,这种方式可能产生更多后端批次;同一轮事件循环内的突发事件和一次写入期间接纳的事件仍会合并。
活跃会话条目把初始化与一个控制器放在一起;该控制器负责待处理事件、计时器、活跃写入、重试暂停和 flush 屏障。协调器仍以独立容器保存已持久化的标识状态、准备好的冷态 Session、标识退役等待方和按 id 操作链,因为即使不存在可写的活跃 Session,这些生命周期仍然存在。有界写入缩短了通常情况下因崩溃而丢失数据的窗口,相比即时调度产生更少的后端批次,同时不改变强制屏障。
`session/flush` 不再决定普通持久化何时开始。它仍是循环和检查点策略使用的顺序与错误观测边界,因此检查点成功仍表示在其完成前接纳的每个事件都已持久化。

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-28-remove-synthetic-log-only-turns.md
2026-07-28-remove-synthetic-log-only-turns.md: 720aa6bbe246ea960a61f6d6f67d36d1f85af08a
2026-07-28-remove-synthetic-log-only-turns.zh.md: 587165a58ae322192b9ed1a9b3312f61e0b21ce6
2026-07-28-remove-synthetic-log-only-turns.md: 61c1aa4b1599fe020e0cd96c32daea01d419e95a
2026-07-28-remove-synthetic-log-only-turns.zh.md: 1661045ca9fb340ebffc9b749263262cb178eae4

View File

@@ -18,7 +18,7 @@ The generic seam also duplicated domain policy. Its marker map said which plugin
Core session invariants continue to enforce core-owned execution relations: turn and step numbering, enclosure of steering, assistant, tool, todo, and request-header events, and same-step tool call/result pairing. Core permits merge-extensible events between turns because only their declaring plugin knows whether they are execution-scoped or standalone. Plugin invariant companions remain responsible for their own event relations.
The title service appends `session/title` directly after its existing service, revision, cancellation, and live-session checks. The bundled model helper appends its literal `session/title-llm-request` record before dispatch. Persistence observes both through the eager `session/event` path and drains them at ordinary checkpoints and lifecycle teardown; neither append forces a flush merely because it is between turns. A fallback, auxiliary request record, or accepted provider title may therefore appear after `turn/end` and before the next `turn/start`. Manual compaction uses the same between-turn capability for a `compact/* { turn: null }` bracket, but explicitly flushes the closed attempt because `/compact` promises durability before releasing queued prompt admission.
The title service appends `session/title` directly after its existing service, revision, cancellation, and live-session checks. The bundled model helper appends its literal `session/title-llm-request` record before dispatch. Persistence admits both through the bounded `session/event` path and drains them at ordinary checkpoints and lifecycle teardown; neither append forces a flush merely because it is between turns. A fallback, auxiliary request record, or accepted provider title may therefore appear after `turn/end` and before the next `turn/start`. Manual compaction uses the same between-turn capability for a `compact/* { turn: null }` bracket, but explicitly flushes the closed attempt because `/compact` promises durability before releasing queued prompt admission.
A session fork may end at any stable event position outside an open turn, not only at `turn/end`. This preserves standalone title and other plugin-owned log-only records in a default fork while still rejecting a prefix cut through active execution.
@@ -40,4 +40,4 @@ Core invariant tests accept an unknown plugin event between turns while continui
## Consequences
Turn counts and outcomes again describe model-loop executions only. Standalone events and manual compaction brackets consume session seqs without consuming a turn number, start eager persistence like every other append, and require owners to request an explicit durability barrier only when their operation promises one. Generic plugin mistakes no longer fail under a core default enclosure rule, so each plugin that needs an execution relation must state and test that relation itself. The title capability keeps revision ordering and lifecycle persistence with less core state, and manual compaction gains durable control with no synthetic-turn or turn-number collision.
Turn counts and outcomes again describe model-loop executions only. Standalone events and manual compaction brackets consume session seqs without consuming a turn number, enter bounded persistence like every other append, and require owners to request an explicit durability barrier only when their operation promises one. Generic plugin mistakes no longer fail under a core default enclosure rule, so each plugin that needs an execution relation must state and test that relation itself. The title capability keeps revision ordering and lifecycle persistence with less core state, and manual compaction gains durable control with no synthetic-turn or turn-number collision.

View File

@@ -18,7 +18,7 @@ Status: implemented
核心会话不变量继续强制核心所属的执行关系:轮次与步骤编号、steering、助手、工具、待办和请求头事件的封闭,以及同一步骤内的工具调用/结果配对。核心允许可合并扩展事件位于轮次之间,因为只有声明它们的插件知道这些事件受执行作用域约束,还是可以独立存在。插件的不变量配套组件仍负责其自身的事件关系。
标题服务会在完成既有的服务状态、修订、取消和实时会话检查后,直接追加 `session/title`。随附模型辅助函数会在发起调用前追加其字面量 `session/title-llm-request` 记录。持久化通过尽快处理的 `session/event` 路径观察两者,并在常规检查点与生命周期 teardown 时排空;二者都不会仅因为位于轮次之间就强制 flush。因此,回退标题、辅助请求记录或已接受的提供方标题可以出现在 `turn/end` 之后、下一个 `turn/start` 之前。手动压缩(compaction)利用同一项轮次间能力记录 `compact/* { turn: null }` 标记对,但会显式 flush 已闭合的尝试,因为 `/compact` 承诺在释放排队提示词接纳预留前完成持久化。
标题服务会在完成既有的服务状态、修订、取消和实时会话检查后,直接追加 `session/title`。随附模型辅助函数会在发起调用前追加其字面量 `session/title-llm-request` 记录。持久化通过有界 `session/event` 路径接纳两者,并在常规检查点与生命周期 teardown 时排空;二者都不会仅因为位于轮次之间就强制 flush。因此,回退标题、辅助请求记录或已接受的提供方标题可以出现在 `turn/end` 之后、下一个 `turn/start` 之前。手动压缩(compaction)利用同一项轮次间能力记录 `compact/* { turn: null }` 标记对,但会显式 flush 已闭合的尝试,因为 `/compact` 承诺在释放排队提示词接纳预留前完成持久化。
会话 fork 可以结束于开放轮次之外的任意稳定事件位置,而不限于 `turn/end`。这样,默认 fork 会保留独立标题和其他插件所属的纯日志记录,同时仍拒绝在活跃执行过程中截断前缀。
@@ -40,4 +40,4 @@ Status: implemented
## 后果
轮次计数和结果重新只描述模型循环执行。独立事件和手动压缩标记对会占用会话 seq,但不占用轮次编号;它们像其他追加一样启动尽快持久化,并且仅当操作承诺持久性时,才要求事件所有方请求显式持久性屏障。通用插件错误不再因核心默认的封闭规则而失败,因此每个需要执行关系的插件都必须自行声明并测试该关系。标题功能保留修订排序和生命周期持久化,同时减少了核心状态;手动压缩则获得持久控制,不产生合成轮次或轮次编号冲突。
轮次计数和结果重新只描述模型循环执行。独立事件和手动压缩标记对会占用会话 seq,但不占用轮次编号;它们像其他追加一样进入有界持久化,并且仅当操作承诺持久性时,才要求事件所有方请求显式持久性屏障。通用插件错误不再因核心默认的封闭规则而失败,因此每个需要执行关系的插件都必须自行声明并测试该关系。标题功能保留修订排序和生命周期持久化,同时减少了核心状态;手动压缩则获得持久控制,不产生合成轮次或轮次编号冲突。