Add generated persistence log event catalog with freshness + completeness gates

docs/persistence-catalog/log-events.md enumerates every SessionEventMap
member — the owning dsh-session vocabulary plus the dsh-compact and
dsh-hook-protocol declaration merges — with payload, surface/log-only badge,
JSDoc prose, and declaration site. scripts/gen-persistence-catalog.ts is a
pure AST pass in the gen-cordis-catalog mold: verify-persistence-catalog
(--check) joins doc-sync, so a stale committed catalog fails pre-push and CI.

The walk enforces JSDoc completeness (every member needs description prose;
@mode is rejected as a category error — log events do not dispatch on the
cordis bus), derives the surface badge from the SurfaceEventType union with a
stale-member cross-check, and hard-errors on duplicate declarations. Payloads
render through the TypeScript printer so newline-separated multi-line type
literals still emit valid one-line fragments.

Documented the five previously JSDoc-less core events (turn/step boundaries,
tool/call), removed the two stray @mode tags on the hook/* merges, and
replaced the hand-restated event enumerations (session.md hook/* table,
compact README table, hook-protocol README bullets, session README name-list
— whose merge note had already drifted) with links to the catalog. RFC:
docs/rfc/implemented/process/2026-07-04-persistence-log-catalog.md.
This commit is contained in:
Tianyi Cui
2026-07-04 22:58:28 +08:00
parent c2c13529cd
commit 232f314c3a
16 changed files with 883 additions and 40 deletions

View File

@@ -17,7 +17,7 @@ Every fact has exactly one home — the tier whose job it is — and every other
| [cookbook/](cookbook/adding-a-package.md) | Step-by-step how-tos with numbered verify steps | Design rationale (→ the RFC each guide links) |
| Package README | The per-package contract: config, semantics, limitations, extension points | JSDoc restatement, generated-catalog restatement (event/tool tables), other packages' concerns |
| [development.md](development.md) | Human-facing setup and daily workflow; a bilingual pair under the [i18n contract](i18n/README.md) | Gate-by-gate enumerations that drift from `package.json` scripts |
| Generated catalogs: [cordis-catalog](cordis-catalog/events-and-services.md), [tool-catalog](tool-catalog/tools.md), [module-graph.md](module-graph.md) | Exhaustive enumerations regenerated from source, freshness-gated | Hand edits of any kind |
| Generated catalogs: [cordis-catalog](cordis-catalog/events-and-services.md), [tool-catalog](tool-catalog/tools.md), [persistence-catalog](persistence-catalog/log-events.md), [module-graph.md](module-graph.md) | Exhaustive enumerations regenerated from source, freshness-gated | Hand edits of any kind |
| Skills (`.agents/skills/`) | Workflows: how to carry out a recurring task against the contracts | The contracts themselves (→ docs) |
Placement test: a story about a bug → postmortem. Why we chose X → RFC. How to do task Y → cookbook. What type Z looks like → core-data-structures. What package P promises → its README. A rule every agent must always obey → root AGENTS.md, one line, linking the home that holds the why.

View File

@@ -1,6 +1,6 @@
# Session Persistence
The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log.
The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log. The event vocabulary the log carries is enumerated, member by member, in the generated [persistence log event catalog](../persistence-catalog/log-events.md).
The seam is a textbook [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md): one abstract service ([dsh-session-persistence](../../packages/session-persistence/session-persistence), `ctx.sessionPersistence`) defining create/append/load/list over the existing `SessionEvent`**no parallel persisted type** — and two interchangeable backends that pass the same `runPersistenceContract` suite. See the [session-persistence RFC](../rfc/implemented/architecture/2026-06-14-session-persistence.md).

View File

@@ -6,7 +6,7 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t
## `SessionEventMap` — the event vocabulary
The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`, and `@deepseek-ai/dsh-hook-protocol` adds log-only `hook/invoked` / `hook/result` provenance for a hook bridge. Like `compact/*`, these are NOT `SurfaceEventType`s (no `surfaceOp`).
The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`, and `@deepseek-ai/dsh-hook-protocol` adds log-only `hook/invoked` / `hook/result` provenance for a hook bridge. Like `compact/*`, these are NOT `SurfaceEventType`s (no `surfaceOp`). The generated [persistence log event catalog](../persistence-catalog/log-events.md) enumerates every member — core and merged — with its payload, surface badge, and declaration site.
```ts type-equiv
interface SessionEventMap {
@@ -224,14 +224,9 @@ Every session event lives **inside** a turn (between a `turn/start` and its `tur
## Plugin-contributed log-only events
A plugin may declaration-merge extra `SessionEventMap` types. These are **log-only**: NOT `SurfaceEventType`s (they carry no `surfaceOp` and contribute nothing to derived history), but, like every event, they must sit inside an open turn. The compaction seam's `compact/*` are documented on [compaction.md](compaction.md); the hook bridges' `hook/*` provenance (from `@deepseek-ai/dsh-hook-protocol`) are:
A plugin may declaration-merge extra `SessionEventMap` types. These are **log-only**: NOT `SurfaceEventType`s (they carry no `surfaceOp` and contribute nothing to derived history), but, like every event, they must sit inside an open turn. The full per-event enumeration — core and plugin-contributed alike, with payloads and provenance — is the generated [persistence log event catalog](../persistence-catalog/log-events.md); the compaction seam's `compact/*` semantics are discussed on [compaction.md](compaction.md).
| Event | Payload | Role |
|---|---|---|
| `hook/invoked` | `{ turn, point, dialect, matcher?, handlerId }` | A hook command was invoked at a hook `point` (`PreToolUse`, `Stop`, …). `dialect` is the bridge (`claude`/`codex`/`native`); `matcher` the matcher-group pattern that selected it (absent for match-all); `handlerId` correlates with the result. |
| `hook/result` | `{ turn, point, handlerId, decision, exitCode?, stderrSummary?, durationMs }` | The decided outcome, paired by `handlerId`. `decision` is the resolved neutral outcome (`deny`/`allow`/`block`/`stop`/`pass`/…); `exitCode` absent when the hook could not run; `stderrSummary` the truncated block-reason source. |
The mid-turn hook points (`PreToolUse`/`PostToolUse`/`UserPromptSubmit`/`Stop`) fire inside the loop's open turn, so their `hook/*` records are turn-enclosed by construction. `SessionStart` gets no `hook/*` record — its injected `context/message` is the durable evidence — because it has no open turn to enclose one (see the hooks RFC).
The hook bridges' `hook/invoked` / `hook/result` provenance pairs (from `@deepseek-ai/dsh-hook-protocol`) correlate by `handlerId`. The mid-turn hook points (`PreToolUse`/`PostToolUse`/`UserPromptSubmit`/`Stop`) fire inside the loop's open turn, so their `hook/*` records are turn-enclosed by construction. `SessionStart` gets no `hook/*` record — its injected `context/message` is the durable evidence — because it has no open turn to enclose one (see [the hook-bridges RFC](../rfc/implemented/feature/2026-06-30-hook-bridges.md)).
## Durability contract

View File

@@ -0,0 +1,240 @@
<!-- Generated by scripts/gen-persistence-catalog.ts — do not edit by hand.
Run `pnpm run gen-persistence-catalog` to regenerate. -->
# Persistence Log Event Catalog
Every event type that can appear in a session's durable event log: each member of the merge-extensible `SessionEventMap` — the owning vocabulary in `@deepseek-ai/dsh-session` plus every plugin declaration merge in this repo — with the payload it carries, its surface badge, and the declaration it comes from. It complements [session.md](../core-data-structures/session.md) (the `SessionEvent` envelope, surface list, and `deriveMessages()` projection), [persistence.md](../core-data-structures/persistence.md) (how the log is made durable), and the [cordis catalog](../cordis-catalog/events-and-services.md) (the live bus wiring — a log event is NOT a cordis event; it reaches listeners via the single `session/event` emit).
This file is GENERATED from source (`scripts/gen-persistence-catalog.ts`) and verified fresh by `pnpm run verify-persistence-catalog` (part of `doc-sync`) — do not edit it by hand. Payload blocks use a `ts persistence-catalog` fence (skipped by doc-typecheck, since a bare payload fragment is not standalone-compilable). Type names in a payload link to the page that documents them. See [the persistence-log-catalog RFC](../rfc/implemented/process/2026-07-04-persistence-log-catalog.md).
The on-disk envelope around every payload is `SessionEvent``type`, monotonic `seq`, epoch-ms `time`, the `data` documented here, plus `surfaceOp`/`sourceEventSeqs` on **surface** events only ([envelope](../core-data-structures/session.md#sessioneventt--one-log-entry)). **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`), and the whole format is pinned at `SESSION_FORMAT_VERSION = 0` — pre-release, no compatibility implied ([the version stance](../core-data-structures/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further event types, which are outside this catalog by construction.
## Events
### `assistant/*`
#### `assistant/chunk` — log-only
Raw stream chunk — token-level replay fidelity.
```ts persistence-catalog
'assistant/chunk': { turn: number; step: number; chunk: StreamChunk }
```
Types: [StreamChunk](../core-data-structures/llm-streaming.md)
Source: [`packages/core/session/src/types.ts:238`](../../packages/core/session/src/types.ts)
#### `assistant/message` — surface
Assembled assistant message for one step (derived history uses this). Carries the step's `usage` when the adapter reported token accounting, so the model output and its accounting travel together (there is no separate usage record). `usage` is absent when the adapter reported none.
```ts persistence-catalog
'assistant/message': { turn: number; step: number; content: ContentBlock[]; usage?: TokenUsage }
```
Types: [ContentBlock](../core-data-structures/core.md) · [TokenUsage](../core-data-structures/llm-streaming.md)
Source: [`packages/core/session/src/types.ts:245`](../../packages/core/session/src/types.ts)
### `compact/*`
#### `compact/end` — log-only
Marks the end of a compaction — log-only, releases the lock. `error` set if summarization failed.
```ts persistence-catalog
'compact/end': { turn: number; error?: string }
```
Source: [`packages/compact/compact/src/types.ts:37`](../../packages/compact/compact/src/types.ts)
#### `compact/start` — log-only
Marks the start of a compaction — log-only, holds the lock until `compact/end`.
```ts persistence-catalog
'compact/start': { turn: number }
```
Source: [`packages/compact/compact/src/types.ts:23`](../../packages/compact/compact/src/types.ts)
#### `compact/summary` — log-only
Provenance record of a completed summarization — log-only, no surfaceOp. The summary content is in `data.summary`; the actual surface replacement is performed by a subsequent `user/message` event that shadows the compacted range.
```ts persistence-catalog
'compact/summary': { summary: ContentBlock[]; shadowedRange: { start: number; end: number }; shadowedSeqs: number[]; shadowedTokenCount: number }
```
Types: [ContentBlock](../core-data-structures/core.md)
Source: [`packages/compact/compact/src/types.ts:30`](../../packages/compact/compact/src/types.ts)
### `context/*`
#### `context/message` — surface
In-session context injection (file-change notices, subdir AGENTS.md, skill content, cron notifications, …). Rendered into the derived history as tagged synthetic context — NOT a user prompt.
```ts persistence-catalog
'context/message': { content: ContentBlock[]; source: MessageSource }
```
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:236`](../../packages/core/session/src/types.ts)
### `hook/*`
#### `hook/invoked` — log-only
A hook command was invoked at a hook point — log-only provenance (like `compact/*`; NOT a SurfaceEventType, carries no `surfaceOp`). `dialect` is the bridge that ran it (`claude`/`codex`/`native`), `point` the hook point (`PreToolUse`, `Stop`, …), `matcher` the matcher-group pattern that selected it (absent for match-all), `handlerId` a stable id for the command (so an invoked/result pair correlates). `turn` is the open turn the invocation lives inside.
```ts persistence-catalog
'hook/invoked': { turn: number; point: string; dialect: HookDialect; matcher?: string; handlerId: string }
```
Source: [`packages/hooks/hook-protocol/src/types.ts:27`](../../packages/hooks/hook-protocol/src/types.ts)
#### `hook/result` — log-only
A hook command's outcome — log-only, paired with a prior `hook/invoked` (same `handlerId`). `decision` is the resolved dialect-neutral outcome the bridge mapped it to (`allow`/`deny`/`ask`/`block`/`continue`/`stop`/`pass`), `exitCode` the process exit (absent if it never ran), `stderrSummary` a truncated stderr (the block reason source on exit 2), `durationMs` the wall time. `turn` matches the `hook/invoked`.
```ts persistence-catalog
'hook/result': { turn: number; point: string; handlerId: string; decision: string; exitCode?: number; stderrSummary?: string; durationMs: number }
```
Source: [`packages/hooks/hook-protocol/src/types.ts:42`](../../packages/hooks/hook-protocol/src/types.ts)
### `prompt/*`
#### `prompt/blocked` — log-only
A queued prompt an `agent/prompt-submit` listener VETOED — the durable record of a blocked prompt and why. Appended in place of the `user/message` the prompt would have become, so the block survives replay even in a MIXED batch where another queued prompt is allowed (there the turn does not end `rejected`, so the boundary reason alone would not preserve it). `content` is the original prompt the listener rejected; `reason` is the veto text (PromptDecision `block.reason`). NOT a SurfaceEventType: a blocked prompt produces no LLM message and never reaches `deriveMessages()`.
```ts persistence-catalog
'prompt/blocked': { content: ContentBlock[]; source: MessageSource; reason: string }
```
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:230`](../../packages/core/session/src/types.ts)
### `steering/*`
#### `steering/message` — surface
Steering content injected between steps of a running turn.
```ts persistence-catalog
'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource }
```
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:263`](../../packages/core/session/src/types.ts)
### `step/*`
#### `step/end` — log-only
Closes step `step` of turn `turn`.
```ts persistence-catalog
'step/end': { turn: number; step: number }
```
Source: [`packages/core/session/src/types.ts:217`](../../packages/core/session/src/types.ts)
#### `step/start` — log-only
Opens step `step` of turn `turn` — one model call plus the tool executions it requested.
```ts persistence-catalog
'step/start': { turn: number; step: number }
```
Source: [`packages/core/session/src/types.ts:215`](../../packages/core/session/src/types.ts)
### `todo/*`
#### `todo/write` — log-only
The agent's whole todo list, carried as a full snapshot and replaced wholesale on each write — the current list is the most recent `todo/write` (last-write-wins on replay, no fold). Appended by an owning agent via `session.append('todo/write', { todos })`.
NOT a SurfaceEventType: it produces no LLM message and never reaches `deriveMessages()`, so it carries no `surfaceOp` and stays off the surface — it is durable, replayable UI state, distinct from the conversation history. It is a `SessionEventMap` member riding the existing `session/event` emit, not a first-class Cordis `interface Events` notification, so it has no cordis-catalog row.
```ts persistence-catalog
'todo/write': { todos: TodoItem[] }
```
Types: [TodoItem](../core-data-structures/session.md)
Source: [`packages/core/session/src/types.ts:277`](../../packages/core/session/src/types.ts)
### `tool/*`
#### `tool/call` — log-only
The model requested one tool invocation: `name` with the raw `arguments` JSON string exactly as the model produced it (unparsed). `callId` pairs the call with its `tool/result`.
```ts persistence-catalog
'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string }
```
Types: [CallId](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:251`](../../packages/core/session/src/types.ts)
#### `tool/result` — surface
A completed tool call's model-facing result, plus an optional tool-private `meta` presentation payload. `meta` is opaque to the core (`unknown` — the producing tool owns its shape and reads it back in `presentResult`) but MUST be JSON-serializable: `Session.append` runtime-validates all event data with `isJsonValue`, so a non-serializable `meta` is rejected at the source, and the durable log reproduces the identical card on replay. Absent unless the tool attaches one (e.g. `dsh-tool-fs` carries its result-time contextual diff here).
```ts persistence-catalog
'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string }; meta?: unknown }
```
Types: [CallId](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:261`](../../packages/core/session/src/types.ts)
### `turn/*`
#### `turn/end` — log-only
Closes turn `turn` with the TurnEndReason that ended it. The loop fires the awaited `session/flush` checkpoint at every turn end, so the turn boundary is also the durable-commit boundary.
```ts persistence-catalog
'turn/end': { turn: number; reason: TurnEndReason }
```
Types: [TurnEndReason](../core-data-structures/session.md)
Source: [`packages/core/session/src/types.ts:213`](../../packages/core/session/src/types.ts)
#### `turn/start` — log-only
Opens turn `turn`. `trigger` records what started it — a drained message batch, a continuation, or an idle-time injection. The turn is the durability/replay boundary: every event sits between a `turn/start` and its matching `turn/end` (the turn-enclosure invariant).
```ts persistence-catalog
'turn/start': { turn: number; trigger: TurnTrigger }
```
Types: [TurnTrigger](../core-data-structures/session.md)
Source: [`packages/core/session/src/types.ts:207`](../../packages/core/session/src/types.ts)
### `user/*`
#### `user/message` — surface
A user-visible prompt (queued message drained at turn start).
```ts persistence-catalog
'user/message': { content: ContentBlock[]; source: MessageSource }
```
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
Source: [`packages/core/session/src/types.ts:219`](../../packages/core/session/src/types.ts)

View File

@@ -172,6 +172,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
| [JSDoc completeness gate for the cordis surface](implemented/process/2026-07-04-cordis-jsdoc-completeness-gate.md) | 2026-07-04 |
| [Documentation tiers, budgets, and the ceiling gate](implemented/process/2026-07-04-doc-tiers-and-budgets.md) | 2026-07-04 |
| [Generate the RFC index tables](implemented/process/2026-07-04-generate-rfc-index-tables.md) | 2026-07-04 |
| [Generated persistence log event catalog](implemented/process/2026-07-04-persistence-log-catalog.md) | 2026-07-04 |
### Testing

View File

@@ -0,0 +1,29 @@
# RFC: Generated persistence log event catalog
Status: implemented (accepted 2026-07-04)
## Context
The session event log is the harness's on-disk contract: every `SessionEventMap` member is a record a persistence backend writes verbatim and a replay reconstructs from, and adding one that breaks the durability rules is a breaking change to the on-disk format. Yet the vocabulary had no single reference. The declarations are split across three files — the owning interface in `@deepseek-ai/dsh-session` plus declaration merges in `@deepseek-ai/dsh-compact` and `@deepseek-ai/dsh-hook-protocol` — and the doc surfaces covered it with hand-copies: a `hook/*` payload table in [session.md](../../../core-data-structures/session.md), a `compact/*` payload table in the compact README, payload bullets in the hook-protocol README, and a name-list in the session README. The name-list's merge note had already drifted (it named the compaction merge and omitted the hook merge entirely), and nothing could catch the next merge going undocumented: a hand-copy only checks the names someone already wrote down. This is the same gap the [cordis catalog](2026-06-20-generated-cordis-catalog.md) closed for bus events and the [tool catalog](2026-07-02-tool-schema-catalog.md) closed for model-facing tools — and log events are covered by neither: a `SessionEventMap` member is not a cordis `Events` declaration (it reaches listeners via the single `session/event` emit), so it has no cordis-catalog row by design.
## Decision
Generate `docs/persistence-catalog/log-events.md` from source, with a freshness gate, as the fourth reference surface: the *records* a persisted session log can contain, complementing the cordis catalog (wiring), core-data-structures (vocabulary), and the tool catalog (tools).
`scripts/gen-persistence-catalog.ts` is a pure TypeScript-AST pass, like `gen-cordis-catalog.ts` and unlike the boot-based tool catalog — the right technique because log events ARE statically knowable: every member is a string-literal-named property with a static type annotation, so the AST is the whole truth. The walk collects every `interface SessionEventMap` declaration under `packages/*/*/src` — the owning top-level interface and every `declare module '@deepseek-ai/dsh-session'` merge — so a brand-new event, core or merged, appears in the next regenerate and an un-regenerated file fails `--check` (`verify-persistence-catalog`, a `doc-sync` member, so pre-push and CI both run it). Each entry renders the member's JSDoc prose, its payload (printed through the TypeScript printer, so a newline-separated multi-line type literal still yields a valid one-line fragment), a surface badge, cross-links into core-data-structures, and the declaration's source pointer, grouped by scope.
Specific choices:
- **JSDoc completeness, enforced.** Every member must carry description prose — the JSDoc becomes the catalog entry, the same forcing function the cordis catalog applies to bus events. An `@mode` tag on a member is a hard error: dispatch modes belong to cordis bus events, and a log event has none — the tag would misread as "this fires on the bus with mode X". Violations aggregate into one error listing every offender.
- **The surface badge is derived, not hand-listed.** `SurfaceEventType` — the subset that produces LLM messages and may carry `surfaceOp` — is parsed from its union declaration in the owning package; a union member naming no declared event is a hard error (a stale union member would otherwise silently badge nothing). Everything else renders **log-only**.
- **A dedicated fence.** Payload blocks use a ` ```ts persistence-catalog ` info string that `doc-typecheck` recognizes and skips, excluded from the opt-out ratio — the same treatment as `ts cordis-catalog` (a bare payload fragment is not standalone-compilable).
- **Repo scope.** The catalog enumerates the packages in this repo, matching the siblings' packages-only scope; a downstream plugin can merge further event types, which are outside the catalog by construction. Nothing else in the repo may name an interface `SessionEventMap` — the walk treats every such declaration as the merged vocabulary, and a duplicate member across declarations is a hard error.
This supersedes the hand-copies: the session.md `hook/*` table, the compact README's event table, the hook-protocol README's payload bullets, and the session README's name-list now link the catalog instead of restating payloads (the surrounding semantics prose stays where it was). The two stray `@mode emit` tags on the hook-protocol merge members are removed — the new gate rejects them as the category error they were.
## Consequences
- The catalog cannot drift: a vocabulary change the committed file doesn't reflect fails `verify-persistence-catalog` in the pre-push hook and CI, and a new merged event with no JSDoc fails the generator outright — a plugin can no longer add an undocumented on-disk record type.
- Event prose has a single home, the JSDoc at the declaration; thin JSDoc yields a thin catalog entry, pressuring authors to document at the source.
- The `SurfaceEventType` union is now structurally load-bearing for docs: renaming an event without updating the union (or vice versa) fails the generator, not just the compiler.
- The badge derivation assumes the union stays a closed set of string literals with exactly one owner; a refactor away from that shape must update the generator in the same change.