feat(session-persistence): abstract seam + JSONL backend + wiring

Add the durable session-persistence capability seam (ADR 0016): an
abstract SessionPersistence service (dsh-session-persistence,
ctx.sessionPersistence) defining create/append/load/list/has/delete/
update over the existing SessionEvent — no parallel persisted type — and
a first implementation (dsh-session-persistence-jsonl): an append-only
JSONL log per session with crash-safe atomic writes, truncation-repair
of a never-committed crash tail, and a read/replay path. SessionMeta
(format version, cwd, lineage) travels out-of-log via session.header.

A shared runPersistenceContract suite holds every backend to the same
append-only / contiguous-seq / lazy-materialization / serializability
semantics.

Config-driven create() now uses a per-run ${id}-session-<uuid> session
id so a fixed name no longer collides with an on-disk log once a durable
backend is loaded; each run is a new session (a demo simplification). The
examples drop their hand-rolled session-jsonl.ts and load the JSONL
backend via cordis.yml; CI smoke-loads it too.

The agent-facing create/resume factory that consumes load() is a
separate seam, deferred to a follow-up; this change stops at the load
primitive and does not reach into the loop.

packages/session-persistence/README.md

@@ -0,0 +1,33 @@
+# @deepseek-ai/dsh-session-persistence
+
+The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, list, and update sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([ADR 0009](../../docs/adr/0009-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
+
+The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionMeta`, owned by `dsh-session` and re-exported here.
+
+## Service API (`ctx.sessionPersistence`)
+
+| Method | Contract |
+|---|---|
+| `create(meta): Promise<void>` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). |
+| `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
+| `load(id): Promise<{ meta; events }>` | Reload meta + log up to the last complete `turn/end`; events contiguous (`events[i].seq === i`); rejects a mid-log gap/parse error or unknown `version`. |
+| `list(): Promise<SessionMeta[]>` | Lightweight listing from metadata, no full-log parse. |
+| `has(id)` / `delete(id)` | Existence / removal. A zero-event lazily-materialized session is absent from `has`/`list`. |
+| `update(id, summary): Promise<void>` | Update mutable `SessionSummary` fields without touching the append-only log. |
+
+## Invariants every backend must honor
+
+- **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. The only exception is the one-time truncation-repair of a never-committed crash tail on the first `append` after a `load`.
+- **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq.
+- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable).
+- **Durability.** `append` returns only once the batch is durable.
+
+## Testing backends
+
+Import `runPersistenceContract` from `tests/contract.ts` and call it with a factory that yields a fresh, empty backend plus a teardown. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics; a backend's own spec adds implementation-specific tests (crash repair, path sanitization) on top.
+
+> **TODO (validate the abstraction with a second backend):** `dsh-session-persistence-jsonl` is currently the only implementation, so the interface and `runPersistenceContract` are only proven against one storage model. A second backend — a SQLite implementation (`dsh-session-persistence-sqlite`), where each `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — would run the SAME `runPersistenceContract` suite and so prove the seam is genuinely backend-agnostic (lazy materialization, crash-tail-on-load, contiguous-seq all expressed against a transactional store rather than an append-only file).
+
+## Metadata types
+
+Re-exported from `dsh-session`: `SessionHeader` (immutable: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`), `SessionSummary` (mutable: `updatedAt`, `title?`, `firstPrompt?`), `SessionMeta` (their intersection).

packages/session-persistence/package.json

@@ -0,0 +1,30 @@
+{
+  "name": "@deepseek-ai/dsh-session-persistence",
+  "description": "Abstract durable session persistence seam (ctx.sessionPersistence) for the DeepSeek Harness",
+  "version": "0.0.1",
+  "private": true,
+  "type": "module",
+  "main": "lib/index.js",
+  "types": "lib/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./lib/index.d.ts",
+      "default": "./lib/index.js"
+    },
+    "./src/*": "./src/*",
+    "./package.json": "./package.json"
+  },
+  "files": [
+    "lib",
+    "src"
+  ],
+  "license": "BSD-3-Clause",
+  "peerDependencies": {
+    "@deepseek-ai/dsh-session": "^0.0.1",
+    "cordis": "^4.0.0-rc.6"
+  },
+  "devDependencies": {
+    "@deepseek-ai/dsh-session": "^0.0.1",
+    "cordis": "^4.0.0-rc.6"
+  }
+}

packages/session-persistence/src/index.ts

@@ -0,0 +1,114 @@
+/**
+ * The durable session-persistence seam (`ctx.sessionPersistence`): an abstract
+ * service defining WHAT a persistence backend does — durably store, reload,
+ * list, and update sessions — without saying HOW. Implementations subclass
+ * {@link SessionPersistence} and register themselves as the
+ * `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl`
+ * (an append-only JSONL log per session) is the first. Future backends swap in
+ * SQLite/WAL, an object store, or a remote service without touching the
+ * consumers (the write-path plugin, the agent-loop resume seam).
+ *
+ * The persisted unit IS the existing {@link SessionEvent} — there is no
+ * parallel "persisted message" type the log must be converted to and from
+ * (faithful to the event-sourced model: the log is the single source of
+ * truth). Metadata that is NOT replayable conversation state (format version,
+ * cwd, lineage) travels separately as {@link SessionMeta}, which is owned by
+ * `dsh-session` and re-exported here.
+ *
+ * @module @deepseek-ai/dsh-session-persistence
+ */
+
+import { Context, Service } from 'cordis'
+import type { SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
+
+// Re-export the metadata vocabulary so consumers import it from the seam.
+export type { SessionHeader, SessionSummary, SessionMeta } from '@deepseek-ai/dsh-session'
+
+declare module 'cordis' {
+  interface Context {
+    sessionPersistence: SessionPersistence
+  }
+}
+
+/**
+ * Abstract durable session-persistence service. Subclass, implement the
+ * abstract methods, and load the subclass as a plugin — it registers as
+ * `ctx.sessionPersistence` (one implementation per context; loading a second
+ * throws, cordis' standard duplicate-service behavior).
+ *
+ * Contracts every implementation MUST honor (a DB backend asserts them inside
+ * a transaction; a file backend appends at EOF):
+ *
+ * - **Append-only.** Committed events — those at or below a flushed `turn/end`
+ *   — are never rewritten. The ONLY exception is the one-time truncation-repair
+ *   of a never-committed crash tail on the first {@link append} after a
+ *   {@link load} (see {@link load}).
+ * - **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`.
+ *   {@link load} rejects a parse error or a `seq` gap in the MIDDLE of the log
+ *   (unloadable); {@link append}'s first event `seq` MUST equal the backend's
+ *   stored next-seq after any repair.
+ * - **JSON-serializable data.** `SessionEventMap` is merge-extensible and
+ *   `event.data` is typed only as `SessionEventMap[K]`, so {@link append}
+ *   REJECTS non-JSON-serializable data with an error naming the offending
+ *   event type. A backend snapshots (serializes/clones) each event when it
+ *   buffers, since `session.events` hands out the live mutable object.
+ * - **Durability.** {@link append} returns only once the batch is durable
+ *   (the file backend fsyncs; a DB commits). {@link create} MAY defer the
+ *   physical write until the first {@link append} (lazy materialization).
+ */
+export abstract class SessionPersistence extends Service {
+  constructor(ctx: Context) {
+    super(ctx, 'sessionPersistence')
+  }
+
+  /**
+   * Register a new session's metadata. A backend MAY defer the physical write
+   * until the first {@link append} (lazy materialization), in which case a
+   * created-but-never-appended session is absent from {@link has}/{@link list}
+   * — abandoned sessions leave nothing behind.
+   */
+  abstract create(meta: SessionMeta): Promise<void>
+
+  /**
+   * Durably persist a batch of events (called from the write-behind drain at
+   * the `session/flush` checkpoint). Honors the append-only and contiguous-seq
+   * contracts: the first event's `seq` MUST equal the stored next-seq after
+   * any truncation-repair of a crash tail. Rejects non-JSON-serializable
+   * `event.data` with an error naming the offending event type.
+   */
+  abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
+
+  /**
+   * Reload a session: its {@link SessionMeta} plus the event log up to the last
+   * durable checkpoint. Returns `meta` AND `events` so the live session is
+   * reconstructed with its `cwd`/lineage, not just its log.
+   *
+   * The loop only flushes at `turn/end`, so a crash can leave a half-written
+   * final turn below the last committed checkpoint. `load` returns events only
+   * up to the **last complete `turn/end`**; a subsequent {@link append} runs
+   * the one-time truncation-repair that physically discards the orphaned tail
+   * before writing. Returned events are contiguous (`events[i].seq === i`); a
+   * parse error or a `seq` gap in the MIDDLE of the log makes the session
+   * unloadable (reject). Rejects an unknown format `version`.
+   */
+  abstract load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }>
+
+  /** Lightweight listing from metadata, without a full-log parse. */
+  abstract list(): Promise<SessionMeta[]>
+
+  /** Whether a session is durably present (materialized). */
+  abstract has(id: SessionId): Promise<boolean>
+
+  /** Remove a session and all its persisted artifacts. */
+  abstract delete(id: SessionId): Promise<void>
+
+  /**
+   * Update mutable metadata ({@link SessionSummary}: `updatedAt`, `title`,
+   * `firstPrompt`) WITHOUT touching the append-only event log. A backend
+   * stores the summary beside the log (a sidecar file, a header row) and
+   * rewrites only it.
+   */
+  abstract update(id: SessionId, summary: Partial<SessionSummary>): Promise<void>
+}
+
+export default SessionPersistence

packages/session-persistence/tests/contract.ts

@@ -0,0 +1,167 @@
+/**
+ * Reusable contract test for any {@link SessionPersistence} backend. A backend
+ * package imports {@link runPersistenceContract} and calls it with a factory
+ * that yields a fresh, empty backend (and a teardown), so every backend is held
+ * to the same append-only / contiguous-seq / lazy-materialization / crash
+ * semantics. The JSONL backend's own spec adds file-specific tests on top.
+ *
+ * @module @deepseek-ai/dsh-session-persistence/tests/contract
+ */
+
+import { describe, expect, it } from 'vitest'
+import { SessionId } from '@deepseek-ai/dsh-session'
+import type { SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session'
+import type { SessionPersistence } from '../src/index.ts'
+
+/** A backend under test plus its teardown. */
+export interface ContractBackend {
+  persistence: SessionPersistence
+  dispose: () => Promise<void>
+}
+
+/** Build a minimal {@link SessionMeta} for a session id. */
+export function meta(id: string, cwd?: string): SessionMeta {
+  return {
+    version: 1,
+    id: SessionId(id),
+    createdAt: 1000,
+    updatedAt: 1000,
+    ...cwd !== undefined ? { cwd } : {},
+  }
+}
+
+/** A well-formed one-turn event log (contiguous seqs from 0). */
+export function oneTurnLog(): SessionEvent[] {
+  return [
+    { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
+    { type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } } },
+    { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } },
+    { type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } },
+    { type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } },
+    { type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } },
+  ]
+}
+
+/**
+ * Run the backend-agnostic contract suite. `make()` MUST return a fresh, empty
+ * backend each call.
+ */
+export function runPersistenceContract(name: string, make: () => Promise<ContractBackend>): void {
+  describe(`SessionPersistence contract: ${name}`, () => {
+    it('round-trips a session: create + append → load returns identical meta and byte-identical events', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s1', '/work')
+        const log = oneTurnLog()
+        await persistence.create(m)
+        await persistence.append(m.id, log)
+
+        const loaded = await persistence.load(m.id)
+        expect(loaded.meta).toMatchObject({ version: 1, id: m.id, cwd: '/work' })
+        expect(loaded.events).toEqual(log)
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('has()/list() exclude a created-but-never-appended (zero-event) session', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        await persistence.create(meta('empty'))
+        expect(await persistence.has(SessionId('empty'))).toBe(false)
+        expect((await persistence.list()).map(m => m.id)).not.toContain(SessionId('empty'))
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('has()/list() include a session once it has events', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s2')
+        await persistence.create(m)
+        await persistence.append(m.id, oneTurnLog())
+        expect(await persistence.has(m.id)).toBe(true)
+        expect((await persistence.list()).map(x => x.id)).toContain(m.id)
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('append rejects a batch whose first seq does not match the stored next-seq', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s3')
+        await persistence.create(m)
+        await persistence.append(m.id, oneTurnLog()) // seqs 0..5, next-seq = 6
+        // A re-append of an already-stored seq must be rejected, not duplicated.
+        const restated = oneTurnLog()
+        await expect(persistence.append(m.id, restated)).rejects.toThrow()
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('append rejects a mid-batch seq gap', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s4')
+        await persistence.create(m)
+        const gapped: SessionEvent[] = [
+          { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
+          { type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // gap: missing seq 1
+        ]
+        await expect(persistence.append(m.id, gapped)).rejects.toThrow()
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('append rejects non-JSON-serializable event data, naming the event type', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s5')
+        await persistence.create(m)
+        // A plugin-added event carrying a BigInt (not JSON-serializable).
+        const bad = [
+          { type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: 1n } },
+        ] as unknown as SessionEvent[]
+        await expect(persistence.append(m.id, bad)).rejects.toThrow(/user\/message/)
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('delete removes a session', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s6')
+        await persistence.create(m)
+        await persistence.append(m.id, oneTurnLog())
+        expect(await persistence.has(m.id)).toBe(true)
+        await persistence.delete(m.id)
+        expect(await persistence.has(m.id)).toBe(false)
+      } finally {
+        await dispose()
+      }
+    })
+
+    it('update mutates summary fields without touching the event log', async () => {
+      const { persistence, dispose } = await make()
+      try {
+        const m = meta('s7')
+        const log = oneTurnLog()
+        await persistence.create(m)
+        await persistence.append(m.id, log)
+        await persistence.update(m.id, { title: 'My session', firstPrompt: 'hi' })
+
+        const loaded = await persistence.load(m.id)
+        expect(loaded.meta.title).toBe('My session')
+        expect(loaded.meta.firstPrompt).toBe('hi')
+        expect(loaded.events).toEqual(log) // log untouched
+      } finally {
+        await dispose()
+      }
+    })
+  })
+}

packages/session-persistence/tests/persistence.spec.ts

@@ -0,0 +1,110 @@
+import { describe, expect, it } from 'vitest'
+import { Context } from 'cordis'
+import { SessionId } from '@deepseek-ai/dsh-session'
+import type { SessionEvent, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session'
+import { SessionPersistence } from '../src/index.ts'
+import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
+
+/**
+ * A minimal in-memory {@link SessionPersistence} used to (a) cover the abstract
+ * base's constructor + service registration and (b) validate the reusable
+ * contract suite itself. The real durable backend is
+ * `@deepseek-ai/dsh-session-persistence-jsonl`.
+ */
+class MemoryPersistence extends SessionPersistence {
+  private store = new Map<string, { meta: SessionMeta; events: SessionEvent[] }>()
+  private pending = new Map<string, SessionMeta>()
+
+  async create(m: SessionMeta): Promise<void> {
+    // Lazy: record the intended meta, but stay absent from has/list until the
+    // first append materializes the session.
+    this.pending.set(m.id, m)
+  }
+
+  async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
+    const existing = this.store.get(id)
+    const nextSeq = existing ? existing.events.length : 0
+    if (events.length > 0 && events[0]!.seq !== nextSeq) {
+      throw new Error(`append seq mismatch for "${id}": expected ${nextSeq}, got ${events[0]!.seq}`)
+    }
+    for (let i = 0; i < events.length; i++) {
+      const e = events[i]!
+      if (e.seq !== nextSeq + i) throw new Error(`non-contiguous seq in batch for "${id}" at index ${i}`)
+      if (containsNonSerializable(e.data)) {
+        throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
+      }
+    }
+    if (!existing) {
+      const m = this.pending.get(id)
+      if (!m) throw new Error(`append before create for "${id}"`)
+      this.store.set(id, { meta: m, events: structuredClone(events) as SessionEvent[] })
+    } else {
+      existing.events.push(...structuredClone(events) as SessionEvent[])
+    }
+  }
+
+  async load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> {
+    const entry = this.store.get(id)
+    if (!entry) throw new Error(`session "${id}" not found`)
+    return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
+  }
+
+  async list(): Promise<SessionMeta[]> {
+    return [...this.store.values()].map(e => structuredClone(e.meta))
+  }
+
+  async has(id: SessionId): Promise<boolean> {
+    return this.store.has(id)
+  }
+
+  async delete(id: SessionId): Promise<void> {
+    this.store.delete(id)
+    this.pending.delete(id)
+  }
+
+  async update(id: SessionId, summary: Partial<SessionSummary>): Promise<void> {
+    const entry = this.store.get(id)
+    if (entry) Object.assign(entry.meta, summary)
+  }
+}
+
+/** Detect BigInt (and other JSON-hostile values) in event data. */
+function containsNonSerializable(value: unknown): boolean {
+  if (typeof value === 'bigint' || typeof value === 'function' || typeof value === 'symbol') return true
+  if (value && typeof value === 'object') {
+    return Object.values(value).some(containsNonSerializable)
+  }
+  return false
+}
+
+// Run the shared contract against the in-memory backend.
+runPersistenceContract('memory', async () => {
+  const ctx = new Context()
+  const fiber = await ctx.plugin(MemoryPersistence)
+  return {
+    persistence: ctx.sessionPersistence,
+    dispose: async () => { await fiber.dispose() },
+  }
+})
+
+describe('SessionPersistence service registration', () => {
+  it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => {
+    const ctx = new Context()
+    const fiber = await ctx.plugin(MemoryPersistence)
+    expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence)
+
+    await fiber.dispose()
+    expect(ctx.sessionPersistence).toBeUndefined()
+  })
+
+  it('round-trips through the registered service instance', async () => {
+    const ctx = new Context()
+    const fiber = await ctx.plugin(MemoryPersistence)
+    const m = meta('reg')
+    await ctx.sessionPersistence.create(m)
+    await ctx.sessionPersistence.append(m.id, oneTurnLog())
+    const loaded = await ctx.sessionPersistence.load(m.id)
+    expect(loaded.events).toHaveLength(6)
+    await fiber.dispose()
+  })
+})

packages/session-persistence/tsconfig.json

@@ -0,0 +1,13 @@
+{
+  "extends": "../../tsconfig.base.json",
+  "compilerOptions": {
+    "rootDir": "src",
+    "outDir": "lib"
+  },
+  "include": ["src"],
+  "references": [
+    { "path": "../../vendor/cosmokit" },
+    { "path": "../../vendor/cordis" },
+    { "path": "../session" }
+  ]
+}