feat(session-persistence): preserve interrupted turns on crash; don't truncate (review #33)

A crash can leave a durable log whose final turn never closed. The old
behavior truncated everything after the last turn/end as a "crash tail".
But a single turn can be HUGE in a long-horizon task (many steps, large
tool output), so truncating it silently destroys real, durably-written
work — truncating a turn is wrong.

New crash recovery (ADR 0018): load() PRESERVES the interrupted turn's
events and CLOSES the orphaned turn by durably appending synthetic
boundary events — a step/end if a step was open, then a turn/end carrying
the new merge-extensible TurnEndReason {kind:'interrupted'}. load()
returns the balanced log, so a resumed session is immediately usable. Only
a never-fully-written TORN tail fragment is discarded; corruption in the
committed region is still unloadable.

- dsh-session: TurnEndReason {kind:'interrupted'} + shared
  interruptedTurnClosers() repair helper.
- JSONL backend: scanLog preserves the longest contiguous prefix
  (including a partial final turn); loadCore truncates a torn fragment and
  durably writes the closers, returning the balanced log.
- runPersistenceContract gains a crash-recovery test (both backends + mock).
- Docs: ADR 0018/0017, architecture.md, package READMEs.

Also (review #33): RFC 013 records the "move event vocabulary to Zod"
question (merge-extensible maps → runtime schema registry) + blast radius;
deferred, not done here.

packages/session-persistence/README.md

@@ -10,14 +10,14 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
 |---|---|
 | `create(meta): Promise<void>` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). |
 | `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
-| `load(id): Promise<{ meta; events }>` | Reload meta + log up to the last complete `turn/end`; events contiguous (`events[i].seq === i`); rejects a mid-log gap/parse error or unknown `version`. |
+| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. |
 | `list(): Promise<SessionMeta[]>` | Lightweight listing from metadata, no full-log parse. |
 | `has(id)` / `delete(id)` | Existence / removal. A zero-event lazily-materialized session is absent from `has`/`list`. |
 | `update(id, summary): Promise<void>` | Update mutable `SessionSummary` fields without touching the append-only log. |
 
 ## Invariants every backend must honor
 
-- **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. The only exception is the one-time truncation-repair of a never-committed crash tail on the first `append` after a `load`.
+- **Append-only; a crashed turn is closed, not truncated.** Committed events (at or below a flushed `turn/end`) are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (`step/end?`+`turn/end {interrupted}`) to balance the log. Only a never-fully-written torn tail fragment is discarded.
 - **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq.
 - **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable).
 - **Durability.** `append` returns only once the batch is durable.

packages/session-persistence/src/index.ts

@@ -39,14 +39,16 @@ declare module 'cordis' {
  * Contracts every implementation MUST honor (a DB backend asserts them inside
  * a transaction; a file backend appends at EOF):
  *
- * - **Append-only.** Committed events — those at or below a flushed `turn/end`
- *   — are never rewritten. The ONLY exception is the one-time truncation-repair
- *   of a never-committed crash tail on the first {@link append} after a
- *   {@link load} (see {@link load}).
+ * - **Append-only; a crashed turn is closed, not truncated.** Committed events
+ *   — those at or below a flushed `turn/end` — are never rewritten. A crash can
+ *   leave an unclosed final turn whose events are real (and possibly large);
+ *   {@link load} preserves them and closes the orphaned turn with synthetic
+ *   boundary events (see {@link load}). Only a never-fully-written torn tail
+ *   fragment is discarded.
  * - **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`.
- *   {@link load} rejects a parse error or a `seq` gap in the MIDDLE of the log
+ *   {@link load} rejects a parse error or a `seq` gap in the COMMITTED region
  *   (unloadable); {@link append}'s first event `seq` MUST equal the backend's
- *   stored next-seq after any repair.
+ *   stored next-seq (after `load` has balanced any interrupted turn).
  * - **JSON-serializable data.** `SessionEventMap` is merge-extensible and
  *   `event.data` is typed only as `SessionEventMap[K]`, so {@link append}
  *   REJECTS non-JSON-serializable data with an error naming the offending
@@ -72,9 +74,9 @@ export abstract class SessionPersistence extends Service {
   /**
    * Durably persist a batch of events (called from the write-behind drain at
    * the `session/flush` checkpoint). Honors the append-only and contiguous-seq
-   * contracts: the first event's `seq` MUST equal the stored next-seq after
-   * any truncation-repair of a crash tail. Rejects non-JSON-serializable
-   * `event.data` with an error naming the offending event type.
+   * contracts: the first event's `seq` MUST equal the stored next-seq (after
+   * `load` has durably closed any interrupted turn). Rejects non-JSON-
+   * serializable `event.data` with an error naming the offending event type.
    */
   abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
 
@@ -83,13 +85,19 @@ export abstract class SessionPersistence extends Service {
    * durable checkpoint. Returns `meta` AND `events` so the live session is
    * reconstructed with its `cwd`/lineage, not just its log.
    *
-   * The loop only flushes at `turn/end`, so a crash can leave a half-written
-   * final turn below the last committed checkpoint. `load` returns events only
-   * up to the **last complete `turn/end`**; a subsequent {@link append} runs
-   * the one-time truncation-repair that physically discards the orphaned tail
-   * before writing. Returned events are contiguous (`events[i].seq === i`); a
-   * parse error or a `seq` gap in the MIDDLE of the log makes the session
-   * unloadable (reject). Rejects an unknown format `version`.
+   * The loop only flushes at `turn/end`, so a crash can leave a durable log
+   * whose final turn never closed: real, fully-written events sit after the last
+   * `turn/end`. Those events are PRESERVED — a single turn can be huge in a
+   * long-horizon task, so truncating it would destroy real work — and `load`
+   * CLOSES the orphaned turn by durably appending the minimal synthetic boundary
+   * events (a `step/end` if a step was open, then a `turn/end` carrying the
+   * `{ kind: 'interrupted' }` reason). The returned `events` therefore end on a
+   * balanced `turn/end` and are immediately usable as a session seed. Only a
+   * never-fully-written TORN tail fragment (a half-written final record) is
+   * discarded. Returned events are contiguous (`events[i].seq === i`); a parse
+   * error or a `seq` gap in the COMMITTED region (at or before the last real
+   * `turn/end`) makes the session unloadable (reject). Rejects an unknown format
+   * `version`. See ADR 0018 for the crash-recovery contract.
    */
   abstract load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }>
 

packages/session-persistence/tests/contract.ts

@@ -64,6 +64,44 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
       }
     })
 
+    it('crash recovery: load preserves an interrupted (unclosed) turn and closes it with turn/end {interrupted}', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('interrupted')
+        await persistence.create(m)
+        await persistence.append(m.id, oneTurnLog()) // turn 1, committed (seqs 0..5)
+        // A second turn that crashed mid-flight: turn/start + step/start were
+        // durably written, but no step/end / turn/end ever arrived.
+        await persistence.append(m.id, [
+          { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
+          { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
+        ])
+
+        // load PRESERVES the interrupted turn's events (a turn can be huge — they
+        // must not be truncated) and closes the orphaned turn with synthetic
+        // boundary events: step/end (the step was open) then turn/end {interrupted}.
+        const loaded = await persistence.load(m.id)
+        expect(loaded.events.map(e => e.type)).toEqual([
+          'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
+          'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers
+        ])
+        expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
+        const last = loaded.events.at(-1)!
+        expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
+
+        // The closed log is durable and continuable: a fresh append continues at
+        // the balanced length (seq 10), and a reload round-trips identically.
+        await persistence.append(m.id, [
+          { type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } },
+          { type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
+        ])
+        const reloaded = await persistence.load(m.id)
+        expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
+      } finally {
+        await dispose()
+      }
+    })
+
     it('has()/list() exclude a created-but-never-appended (zero-event) session', async () => {
       const { persistence, dispose } = await make()
       try {

packages/session-persistence/tests/persistence.spec.ts

@@ -1,6 +1,6 @@
 import { describe, expect, it } from 'vitest'
 import { Context } from 'cordis'
-import { SessionId, isJsonValue } from '@deepseek-ai/dsh-session'
+import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session'
 import type { SessionEvent, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
 import { SessionPersistence } from '../src/index.ts'
 import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
@@ -46,6 +46,11 @@ class MemoryPersistence extends SessionPersistence {
   async load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
     const entry = this.store.get(id)
     if (!entry) throw new Error(`session "${id}" not found`)
+    // Honor the crash-recovery contract: if the stored log ends mid-turn, close
+    // the orphaned turn durably with synthetic boundary events and continue from
+    // the balanced length.
+    const closers = interruptedTurnClosers(entry.events)
+    if (closers.length > 0) entry.events.push(...structuredClone(closers))
     return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
   }