feat(session-persistence-sqlite): second backend validating the abstraction

Add a SQLite SessionPersistence backend (node:sqlite), a SECOND
implementation built to prove the abstract seam + the shared
runPersistenceContract suite are genuinely backend-agnostic. Each
SessionEvent maps 1:1 onto an events row (session_id, seq, type, time,
data); append is an INSERT inside a transaction asserting the
contiguous-seq contract; the mutable SessionSummary lives in the
sessions metadata row.

It satisfies the SAME contract semantics as the JSONL backend, expressed
over rows instead of file bytes:
- Lazy materialization: create() records intent in memory; no row until
  the first append (a never-appended session is absent from has()/list()
  via a materialized flag set inside the first append transaction).
- Crash-tail-on-load: load() returns events only through the last
  complete turn/end and deletes the uncommitted tail; a seq gap in the
  committed region makes the session unloadable.
- Transactional append: a mid-batch failure (a UNIQUE seq collision from
  a concurrent writer) rolls back entirely, keeping the cursor truthful.

Like the JSONL backend it is also the write-path plugin (session/event →
buffer → session/flush drain, onCreated seed/adopt/collision handling,
HMR seeding, dispose-to-quiescence). The package runs the shared
runPersistenceContract suite plus SQLite-specific tests (transaction
rollback, crash-tail cut, schema version, HMR adoption).

Docs flip every "SQLite is future/deferred" reference (ADR 0016,
architecture.md, the persistence module doc + README) to "implemented;
the contract holds both backends to identical semantics".
This commit is contained in:
Tianyi Cui
2026-06-15 21:45:21 +08:00
parent 9a4006cb2b
commit 9126697d87
15 changed files with 1317 additions and 7 deletions

View File

@@ -19,7 +19,7 @@ Key choices recorded here because they are durable, contested, and surprising:
- **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
- **Append-only with a single exception.** Committed events — those at or below a flushed `turn/end` — are never rewritten. The loop only flushes at `turn/end`, so a crash can leave a half-written final turn below the last checkpoint; `load` returns events only up to the **last complete `turn/end`**, and the first post-load `append` runs a one-time **truncation-repair** (`ftruncate` + `fsync`) that physically discards only that never-committed crash tail before writing.
- **File backend canonical, DB backend a drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)``append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. A future `dsh-session-persistence-sqlite` is a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL).
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)``append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, crash-tail-on-load, contiguous-seq), expressed once over file bytes and once over rows.
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionMeta` (`SessionHeader & SessionSummary`) owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost.
- **Resume is an async factory, not a change to synchronous create.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and starts a fresh agent on the resumed id (NOT `${agentId}-session`). The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a typed error when it is absent.

View File

@@ -95,7 +95,7 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source
Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`.
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list/update over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, truncation-repair of a never-committed crash tail, and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionMeta`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A SQLite/WAL backend is a future drop-in `SessionPersistence` subclass (the row shape `(session_id, seq, type, time, data)` maps 1:1 onto `SessionEvent`).
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list/update over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, truncation-repair of a never-committed crash tail, and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionMeta`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent`the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
## Prompt assembly (dsh-system-prompt)
@@ -241,7 +241,6 @@ Code skeletons for the three plugin shapes (tool, hook/permission-gate, UI) and
Tracked here deliberately — each is designed-for but not implemented:
- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events.
- **SQLite/WAL persistence backend** — a drop-in `SessionPersistence` subclass (the abstract seam + the JSONL backend landed; see the durability-seam paragraph).
- **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging.
- **Parallel tool execution** (concurrency-safety hints on ToolDefinition).
- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based forking.