Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #2048 from deepseek-harness/worktree/batch-session-persistence

Browse Source 
Batch durable session writes with a bounded delay
This commit is contained in:
Tianyi Cui
2026-08-08 16:51:15 +08:00
committed by GitHub
parent 41b2fd7a70 1576d049ed
commit a44e35793c
53 changed files with 838 additions and 125 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/architecture.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/architecture.md
architecture.md: 35a73d4a307f5f48cc41cc496742a2ac210e8877
architecture.zh.md: 185958221a477bb690e3ab5c91c33ba892ab2d73
architecture.md: 3080d21a24310cc8851d64719d862ec719150ab6
architecture.zh.md: 6e2bf0c155c68e84580aecc6b7783eeffa25c640

4
docs/architecture.md
View File

@@ -141,9 +141,9 @@ The session log is authoritative. `deriveMessages()` projects model history; raw
**Model-visible ⟺ logged**: messages entering at `step/start` plus the folded `request/header` reconstruct every request. The header marks adapter defaults so later proposals discard them and re-resolve the route without losing explicit settings. `request/context` separately records registration-bound provider, model, and capacity metadata when the route changes; it does not participate in request reconstruction or header equality. `dsh-agent-loop/invariant` asserts reconstructability through `ctx.invariants` ([reconstructability](../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md)).
Durability is a plugin concern. Backends eagerly drain synchronous `session/event` notifications. `session/flush` precedes requests and top-level tool dispatch, and follows `turn/end` before another turn or idle. `SessionPersistence` stores events and header metadata; JSONL defaults to checksummed Zstandard and SQLite shares the contract ([decision](../.agents/notes/implemented/bug-fix/2026-07-21-semantic-session-checkpoints.md)).
Durability is a plugin concern. Backends copy synchronous `session/event` notifications into fixed-window durable batches; `session/flush` bypasses the wait before requests and top-level tool dispatch, and after `turn/end` before another turn or idle. `SessionPersistence` stores events and header metadata; JSONL defaults to checksummed Zstandard and SQLite shares the contract ([checkpoint decision](../.agents/notes/implemented/bug-fix/2026-07-21-semantic-session-checkpoints.md), [batching decision](../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.md)).
Between turns, owners append log-only events through `Session`, flushing only for durability. `session/title` needs eager persistence and lifecycle drains; manual compaction flushes its bracket before the operation completes. Title work never delays responses; latest wins with provenance. Title records are inherited fork boundaries ([decision](../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.md)).
Between turns, owners append log-only events through `Session`, flushing only for durability. `session/title` relies on bounded background persistence and lifecycle drains; manual compaction flushes its bracket before the operation completes. Title work never delays responses; latest wins with provenance. Title records are inherited fork boundaries ([decision](../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.md)).
### Model Content

4
docs/architecture.zh.md
View File

@@ -141,9 +141,9 @@ idle inject:
**模型可见 ⟺ 已记录**：在 `step/start` 进入的消息加上折叠后的 `request/header` 可以重建每个请求。该 header 会标记适配器默认值，使后续提议丢弃这些值并重新解析路由，同时不丢失显式设置。`request/context` 会在路由变化时另行记录与注册项绑定的提供方、模型及容量元数据；它不参与请求重建或 header 相等性判断。`dsh-agent-loop/invariant` 通过 `ctx.invariants` 断言可重建性（[可重建性](../.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md)）。
持久性由插件负责。后端会尽快排空同步的 `session/event` 通知。`session/flush` 位于请求与顶层工具分发之前，并在 `turn/end` 之后、另一个轮次或空闲状态之前执行。`SessionPersistence` 存储事件和 header 元数据；JSONL 默认采用带校验和的 Zstandard，SQLite 遵循同一契约（[决策](../.agents/notes/implemented/bug-fix/2026-07-21-semantic-session-checkpoints.md)）。
持久性由插件负责。后端会将同步的 `session/event` 通知复制到固定窗口的持久化批次中；`session/flush` 会绕过等待，在请求与顶层工具分发之前执行，并在 `turn/end` 之后、另一个轮次或空闲状态之前执行。`SessionPersistence` 存储事件和 header 元数据；JSONL 默认采用带校验和的 Zstandard，SQLite 遵循同一契约（[检查点决策](../.agents/notes/implemented/bug-fix/2026-07-21-semantic-session-checkpoints.md)、[批处理决策](../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.md)）。
在轮次之间，事件所有方通过 `Session` 追加纯日志事件，仅为持久性而刷写。`session/title` 需要尽快持久化与生命周期排空；手动压缩会在操作完成前 flush 其标记对。标题工作绝不延迟响应；最新标题按后写覆盖并携带来源信息。标题记录是可继承的 fork 边界（[决策](../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.md)）。
在轮次之间，事件所有方通过 `Session` 追加纯日志事件，仅为持久性而刷写。`session/title` 依赖有界后台持久化与生命周期排空；手动压缩会在操作完成前 flush 其标记对。标题工作绝不延迟响应；最新标题按后写覆盖并携带来源信息。标题记录是可继承的 fork 边界（[决策](../.agents/notes/implemented/feature/2026-07-21-log-backed-session-titles.md)）。
### 模型内容

8
docs/config-catalog.md
View File

@@ -1249,13 +1249,15 @@ export interface Config {
compression?: JsonlCompression
/** Maximum cold Session preparations retained for history-to-resume reuse. */
preparedSessionCacheSize?: number
/** Fixed live-event coalescing window; not a backend completion deadline. */
writeBatchMaxDelayMs?: number
}
/** Physical encoding selected for JSONL session artifacts. */
export type JsonlCompression = 'zstd' | 'none'
```
Source: [`packages/session-persistence/session-persistence-jsonl/src/index.ts:58`](../packages/session-persistence/session-persistence-jsonl/src/index.ts)
Source: [`packages/session-persistence/session-persistence-jsonl/src/index.ts:59`](../packages/session-persistence/session-persistence-jsonl/src/index.ts)
## `@deepseek-ai/dsh-session-persistence-sqlite`
@@ -1283,6 +1285,8 @@ export interface Config {
journalMode?: JournalMode
/** Maximum cold Session preparations retained for history-to-resume reuse. */
preparedSessionCacheSize?: number
/** Fixed live-event coalescing window; not a backend completion deadline. */
writeBatchMaxDelayMs?: number
}
/**
@@ -1296,7 +1300,7 @@ export interface Config {
export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist'
```
Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:66`](../packages/session-persistence/session-persistence-sqlite/src/index.ts)
Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:67`](../packages/session-persistence/session-persistence-sqlite/src/index.ts)
## `@deepseek-ai/dsh-session-projection-cache`

2
docs/cordis-catalog/services.md
View File

@@ -1309,7 +1309,7 @@ abstract listSnapshots(signal?: AbortSignal): Promise<SessionPersistenceSnapshot
Types: [SessionEvent](../core-data-structures/core.md) · [SessionHeader](../core-data-structures/persistence.md) · [SessionId](../core-data-structures/core.md) · [SessionInspection](../core-data-structures/persistence.md) · [SessionLocation](../core-data-structures/persistence.md) · [SessionPersistenceSnapshot](../core-data-structures/persistence.md) · [SessionPreparation](../core-data-structures/persistence.md)
Source: [`packages/session-persistence/session-persistence/src/index.ts:70`](../../packages/session-persistence/session-persistence/src/index.ts)
Source: [`packages/session-persistence/session-persistence/src/index.ts:72`](../../packages/session-persistence/session-persistence/src/index.ts)
## `ctx.sessionProjectionCache` — `SessionProjectionCache`

4
docs/core-data-structures/persistence.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/persistence.md
persistence.md: 0968496201defa869d94925e8e5ae3c5da1bbd37
persistence.zh.md: efb01427b4355e531fb9b86922223cf27d3b3db0
persistence.md: 3d2bc7ff3b45cd3f38fca0b880a0df9adb928bad
persistence.zh.md: 035f46d76e0189bd686e991c3d56d486b6f309d5

2
docs/core-data-structures/persistence.md
View File

@@ -8,7 +8,7 @@ The seam is a textbook [capability seam](../../.agents/notes/implemented/archite
## The flush checkpoint
`session/event` is a *synchronous* notification; persistence plugins copy the event into a per-session controller and start an eager write without blocking the producer. Concurrent events share the current batch, and events admitted during that write trigger a follow-up batch. `session/flush` waits until no current or pending batch remains, so the loop still uses it as the ordering and error-observation checkpoint before claiming the next ordinary turn. A rejected eager write retains its events; an explicit flush retries them and reports failure through `agent/error` and the logger, never as a session event past the closed turn. Disposal performs the same final drain.
`session/event` is a *synchronous* notification; persistence plugins copy the event into a per-session controller without blocking the producer. The first pending event starts a fixed batching window, and later events join without resetting its deadline. Expiry starts one durable batch; events admitted during that write receive their own deadline and form a follow-up batch. `session/flush` cancels the wait and drains through quiescence, so the loop still uses it as the ordering and error-observation checkpoint before claiming the next ordinary turn. A rejected background write retains its events and pauses automatic retry; a new event starts a fresh window, while explicit flush retries immediately and reports failure through `agent/error` and the logger, never as a session event past the closed turn. Disposal performs the same final drain. The configured maximum bounds only intentional batching wait, not event-loop scheduling or backend durability latency ([decision](../../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.md)).
## Crash recovery preserves an interrupted turn

2
docs/core-data-structures/persistence.zh.md
View File

@@ -8,7 +8,7 @@
## flush 检查点
`session/event` 是一个*同步*通知；持久化插件会将事件复制到逐会话控制器，并立即启动写入而不阻塞生产方。并发事件会加入当前批次；在该批次写入期间接纳的事件会触发后续批次。`session/flush` 会等待当前与待处理批次全部清空，因此循环仍将其用作在领取下一个普通轮次之前的顺序与错误观察检查点。立即写入被拒绝时会保留对应事件；显式 flush 会重试这些事件，并通过 `agent/error` 和 logger 报告失败，绝不会把失败记录成已关闭轮次之后的会话事件。dispose（资源释放）会执行同样的最终排空。
`session/event` 是一个*同步*通知；持久化插件会将事件复制到逐会话控制器，而不阻塞生产方。第一个待处理事件会开启固定批处理窗口，后续事件会加入但不会重置截止时间。窗口到期后会启动一个持久化批次；该次写入期间接纳的事件会获得自己的截止时间，并形成后续批次。`session/flush` 会取消等待并排空至完全停稳，因此循环仍将其用作在领取下一个普通轮次之前的顺序与错误观察检查点。后台写入被拒绝时会保留对应事件并暂停自动重试；新事件会开启新的固定窗口，而显式 flush 会立即重试，并通过 `agent/error` 和 logger 报告失败，绝不会把失败记录成已关闭轮次之后的会话事件。dispose（资源释放）会执行同样的最终排空。配置的最大值只限制有意的批处理等待，不限制事件循环调度或后端完成持久化的延迟（[决策](../../.agents/notes/implemented/architecture/2026-08-08-bounded-session-persistence-write-batching.md)）。
## 崩溃恢复保留被中断的轮次

4
docs/core-data-structures/session.i18n.yaml
View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/session.md
session.md: fb04ff3eb3c15d1f3c923ee73d4ba19537d36f11
session.zh.md: 138159d97a5c368f28821715512632e0ca95b29e
session.md: 5d54b03df8ec3345e8bf04702f242e3aacf8ec39
session.zh.md: 5a867cfe302bf02994bfe5a5a704bed55222e8ea

2
docs/core-data-structures/session.md
View File

@@ -561,7 +561,7 @@ interface TurnEndReasonMap {
## Execution enclosure and standalone events
A turn encloses one model-loop execution, not the whole session log. AgentLoop records injected `user/message` events only from entering pre-step batches inside a turn; plugin-owned log-only events may still appear between `turn/end` and the next `turn/start`, consuming event seqs without incrementing turn numbers. Persistence eagerly records every contiguous accepted event, while crash repair closes only a genuinely open trailing turn. A producer that needs a durability barrier explicitly awaits `ctx.sessions.flush(session)`.
A turn encloses one model-loop execution, not the whole session log. AgentLoop records injected `user/message` events only from entering pre-step batches inside a turn; plugin-owned log-only events may still appear between `turn/end` and the next `turn/start`, consuming event seqs without incrementing turn numbers. Persistence admits every contiguous accepted event into a bounded durable batch, while crash repair closes only a genuinely open trailing turn. A producer that needs an immediate durability barrier explicitly awaits `ctx.sessions.flush(session)`.
The optional `dsh-session/invariant` companion enforces the relations owned by core: turn and step numbering, execution-event enclosure, and same-step tool call/result pairing. Merge-extensible event relations belong to the plugin that declares them, so core does not reject an unknown event merely because no turn is open. See [the standalone-event decision](../../.agents/notes/implemented/simplification/2026-07-28-remove-synthetic-log-only-turns.md).

2
docs/core-data-structures/session.zh.md
View File

@@ -565,7 +565,7 @@ interface TurnEndReasonMap {
## 执行封闭与独立事件
一个轮次包围一次模型循环执行，而不是整个会话日志。AgentLoop 只会从轮次内返回 enter 的 pre-step 批次记录注入的 `user/message` 事件；插件所属的纯日志事件仍可出现在 `turn/end` 与下一个 `turn/start` 之间，占用事件 seq 但不递增轮次编号。持久化会尽快记录每个连续且已接受的事件，而崩溃修复只关闭确实仍处于开放状态的尾部轮次。需要持久性屏障的生产方会显式等待 `ctx.sessions.flush(session)`。
一个轮次包围一次模型循环执行，而不是整个会话日志。AgentLoop 只会从轮次内返回 enter 的 pre-step 批次记录注入的 `user/message` 事件；插件所属的纯日志事件仍可出现在 `turn/end` 与下一个 `turn/start` 之间，占用事件 seq 但不递增轮次编号。持久化会将每个连续且已接受的事件纳入有界持久化批次，而崩溃修复只关闭确实仍处于开放状态的尾部轮次。需要即时持久性屏障的生产方会显式等待 `ctx.sessions.flush(session)`。
可选的 `dsh-session/invariant` 配套插件会强制核心拥有的关系：轮次与步骤编号、执行事件封闭，以及同一步骤内的工具调用／结果配对。可合并扩展事件的关系由声明它的插件拥有，因此核心不会仅因没有开放轮次就拒绝未知事件。见[独立事件决策](../../.agents/notes/implemented/simplification/2026-07-28-remove-synthetic-log-only-turns.md)。

3
docs/module-graph.md
View File

@@ -449,6 +449,7 @@ flowchart TD
pkg_session_persistence --> pkg_brand
pkg_session_persistence --> pkg_invariants
pkg_session_persistence --> pkg_session
pkg_session_persistence --> pkg_timeout
pkg_acp_snapshot --> pkg_invariants
pkg_acp_snapshot --> pkg_session
pkg_app_boot --> pkg_environment
@@ -1220,7 +1221,7 @@ flowchart TD
| [`web-search-exa`](../packages/web/web-search-exa) | `web` | [`environment`](../packages/util/environment), [`invariants`](../packages/support/invariants), [`web`](../packages/web/web) |
| [`web-search-perplexity`](../packages/web/web-search-perplexity) | `web` | [`environment`](../packages/util/environment), [`invariants`](../packages/support/invariants), [`web`](../packages/web/web) |
| [`spill`](../packages/spill/spill) | `spill` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
| [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) |
| [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) |
| [`acp-snapshot`](../packages/support/acp-snapshot) | `support` | [`invariants`](../packages/support/invariants), [`session`](../packages/core/session) |
| [`app-boot`](../packages/ui/app-boot) | `ui` | [`environment`](../packages/util/environment), [`invariants`](../packages/support/invariants), [`paths`](../packages/util/paths), [`system-prompt`](../packages/core/system-prompt) |
| [`client-ui-question`](../packages/client/ui-question) | `client` | [`client-locale`](../packages/client/locale), [`invariants`](../packages/support/invariants) |