diff --git a/AGENTS.md b/AGENTS.md index 0533c4cf0a..1bb32121e9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,6 +6,16 @@ This is the monorepo for the DeepSeek Harness group. It currently hosts the code **This applies only while the harness is unreleased — remove this section at the first tagged/published release.** There are no external consumers yet, so optimize for the *correct foundation*, not for a small diff. When the right structure means moving a file across package boundaries, renaming a public symbol, or repackaging a plugin, do it — and update every reference in the same change. Do **not** add backward-compat shims, deprecation aliases, re-export stubs, or "keep it where it is to avoid churn" hedges; those are debts you take on to protect callers you do not have. Churn now is cheap; a wrong foundation set in stone is not. (Once released, this inverts — backward compatibility becomes a real constraint and this section comes out.) +This extends to **on-disk formats, schemas, and stored data**: while unreleased there is no persisted user data to preserve, so a format/schema/contract change needs **no migration path**. Bump the version and reject (don't migrate) anything not at the current version — e.g. the SQLite backend's `SCHEMA_VERSION` bump that drops columns simply rejects any non-current `user_version` on open, with no v1→v2 migration. A migration written now is a shim for data that does not exist. + +## Tests document behavior, not golden truth + +A passing test pins the behavior the code **currently** has — not necessarily the behavior it **should** have. Existing tests faithfully document existing behavior, but existing behavior is not automatically golden: it can be the residue of a past compromise, a half-built feature, or a limitation that no longer applies. So when a refactor or review makes you ask "can I change this?", a green test is **not** the answer — the question is whether the behavior the test pins is actually correct. + +Before you preserve a behavior solely to keep a test green, ask: is this behavior load-bearing (a real consumer depends on it, a contract promises it, a user observes it), or is it an artifact? If it's an artifact, **change the behavior AND its test together, in the same change, and say why in the PR** — do not contort new code to keep an obsolete assertion passing, and do not treat "but the test expects X" as a reason X must stay. Conversely, do not delete a test just because it is inconvenient: the discipline cuts both ways — you must show the *behavior* is dead, not merely that the test is in your way. + +The worked example is [Drop the mutable session summary](docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md): an entire `SessionSummary` type, a `SessionPersistence.update()` method, a JSONL sidecar, and SQLite columns existed and were exercised by their own contract test — yet nothing in production read or wrote any of it. The tests documented the behavior perfectly; the behavior was dead. Deleting the behavior and its tests together removed ~400 lines and erased a durability divergence the next refactor would have had to model. (This is the test-tier echo of "verify the world, not a synthetic stand-in" in § Defensive patterns: a test agrees with whatever it was written to assert; only a real consumer proves the behavior matters.) + ## Architecture This codebase is based on the **Cordis** framework, built microkernel-style: **everything is a plugin**. All necessary Cordis dependencies are copied into this monorepo as vendored source (under `vendor/`) instead of being depended on via npm. diff --git a/docs/architecture.md b/docs/architecture.md index ef143153bb..81a478b714 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -45,7 +45,7 @@ Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop` |---|---|---|---| | `ctx.llm` | `LlmService` | dsh-llm | adapter registry; `stream()` / `streamBlocks()` / `generate()` | | `ctx.sessions` | `SessionStore` | dsh-session | creates/holds event-sourced `Session`s | -| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list/update sessions | +| `ctx.sessionPersistence` | `SessionPersistence` (abstract) | dsh-session-persistence | durable persistence seam: create/append/load/list sessions | | `ctx.systemPrompt` | `SystemPrompt` | dsh-system-prompt | ordered sections + tool schemas → `assemble()` | | `ctx.tools` | `ToolRegistry` | dsh-tools | tool definitions; `execute()` through waterfall | | `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam | @@ -85,7 +85,7 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`. -**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list/update over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionMeta`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. +**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic. ## Prompt assembly (dsh-system-prompt) diff --git a/docs/rfc/README.md b/docs/rfc/README.md index 48e99d37b8..bc915f2292 100644 --- a/docs/rfc/README.md +++ b/docs/rfc/README.md @@ -60,6 +60,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r | [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 | | [ACP snapshot tests — record-once / replay-deterministic](implemented/2026-06-19-acp-snapshot-tests.md) | 2026-06-19 | | [Real-API e2e in CI against the external DeepSeek API](implemented/2026-06-19-real-api-e2e-ci.md) | 2026-06-19 | +| [Drop the mutable session summary](implemented/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 | ## Rejected diff --git a/docs/rfc/implemented/2026-06-14-session-persistence.md b/docs/rfc/implemented/2026-06-14-session-persistence.md index e676181465..3e66be3730 100644 --- a/docs/rfc/implemented/2026-06-14-session-persistence.md +++ b/docs/rfc/implemented/2026-06-14-session-persistence.md @@ -16,15 +16,15 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic: -1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`/`has`/`delete`/`update`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type. -2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**) plus an atomic `.summary.json` sidecar for the mutable `SessionSummary`. +1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`/`has`/`delete`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type. +2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**). Key choices recorded here because they are durable, contested, and surprising: - **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log. - **Append-only; a crashed turn is closed, never truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. The loop only flushes at `turn/end`, so a crash can leave a durable log whose final turn never closed: real, fully-written events sit after the last `turn/end`. **A single turn can be huge in a long-horizon task** (many steps, large tool output spanning a long autonomous run), so discarding the interrupted turn would silently destroy a large amount of real work — truncating a turn is wrong. Instead, on reload `load` PRESERVES those events and CLOSES the orphaned turn by durably appending the minimal synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered, then a `step/end` if a step was still open, then a `turn/end` carrying the merge-extensible `{ kind: 'interrupted' }` reason (a marker that records the turn was cut short by a crash, not completed by the model — no loop ever emits it). The synthetic tool results matter for resume correctness: the loop logs the `assistant/message` (carrying the `tool-call` blocks) BEFORE running the tools, so a crash mid-tool leaves calls without results; `deriveMessages()` would then replay a dangling assistant tool-call, which every provider rejects as an invalid transcript on the next request. Answering each orphaned call with an error result keeps the rehydrated history valid. `load` returns the balanced log, so a resumed session is immediately usable. The ONLY thing discarded is a never-fully-written **torn tail fragment** — a final record whose bytes (JSONL) or row were never completely flushed; that fragment is not a valid event and is dropped before the synthetic closers are written. A parse error or `seq` gap in the COMMITTED region (at or before the last real `turn/end`) is genuine corruption and makes the session unloadable. - **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows. -- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionMeta` (`SessionHeader & SessionSummary`) owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. +- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](2026-06-19-drop-mutable-session-summary.md).) - **Resume is an async factory, not a change to synchronous create.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and starts a fresh agent on the resumed id (NOT `${agentId}-session`). The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent. Format versioning: the header carries a `version`; `load` rejects an unknown version (no v1 migration). Stated honestly: append-only + flush is robust to partial trailing writes (tolerated on load) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option later. diff --git a/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md b/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md new file mode 100644 index 0000000000..f2b82fe2f9 --- /dev/null +++ b/docs/rfc/implemented/2026-06-19-drop-mutable-session-summary.md @@ -0,0 +1,31 @@ +# RFC: Drop the mutable session summary + +Status: implemented (proposed and accepted 2026-06-19) + +## Context + +The [session-persistence seam](2026-06-14-session-persistence.md) split a session's out-of-log metadata into two types owned by `dsh-session`: an immutable `SessionHeader` (`version`, `id`, `createdAt`, `cwd?`, `parentSession?`) written once at creation, and a mutable `SessionSummary` (`updatedAt`, `title?`, `firstPrompt?`) "updateable without touching the append-only log". Their union was `SessionMeta = SessionHeader & SessionSummary`, and the abstract `SessionPersistence` service carried a seventh method — `update(id, summary)` — for rewriting the summary. Each backend implemented the mutable store its own way: JSONL wrote a separate atomic `.summary.json` **sidecar** beside the log (temp-write + rename, best-effort), SQLite kept `updated_at`/`title`/`first_prompt` **columns** bumped inside the append transaction. + +The summary was designed for a future session picker (recency ordering via `updatedAt`, a `title`/`firstPrompt` preview). That picker was never built. An audit of the whole repo found the entire `SessionSummary` surface is **dead state**: + +- `SessionPersistence.update()` has **zero production callers** (every `.update(` hit is `createHash().update()` or a test). +- `firstPrompt` is **never read** anywhere in production. +- `title` *is* read in the ACP bridge — but from a tool-call **presenter** (`present.title`), never from stored session metadata. +- `updatedAt` has **no consumer**: the only production caller of `list()` reads `meta.cwd` (a `SessionHeader` field) to validate a workspace on `session/load`; resume reads `createdAt`/`cwd`/`parentSession` — all header fields. +- Decisively: the live `Session.header` was already typed `SessionHeader`, not `SessionMeta` — the summary never existed on the live session object; it lived only in the persistence layer, written and read by nothing but its own contract test. + +## Decision + +Delete the mutable session summary entirely. `SessionSummary` and the `SessionMeta` name are removed; the metadata a backend stores and returns is just `SessionHeader`. `SessionPersistence.update()` is removed from the abstract service and every backend. JSONL loses the whole sidecar machinery (`writeSidecar`/`readSidecar`/`touchSummary`/`removeSidecars`/`sidecarPath` and the load/list overlays); SQLite drops the `updated_at`/`title`/`first_prompt` columns and the per-append `updated_at` bump, and its `SCHEMA_VERSION` goes `1 → 2`. + +Anything the summary was meant to provide is **derivable from the append-only log** when a consumer actually needs it (`firstPrompt` = first `user/message`; recency = the last event's `time` or the file mtime) or already lives in the immutable header (`createdAt`, `cwd`). The one thing *not* derivable — a user-*edited* title — had no implementation and is pure YAGNI; it can return as its own log event or header field if a real feature ever needs it. + +This is recorded as a decision because it is **durable** (it narrows a public service contract and an on-disk format across two backends), **contested** (the summary was a deliberate forward-looking design, not an accident), and **surprising** (a future reader finding `SessionHeader` where the original RFC describes `SessionMeta` would otherwise ask why the summary vanished). It also unblocks the [shared persistence write coordinator](../proposed/2026-06-18-shared-persistence-write-coordinator.md): with no mutable summary, the coordinator's hook interface needs no `updateSummary` hook and the JSONL-sidecar-vs-SQLite-column durability divergence disappears, so the two backends' write paths converge. + +## No migration + +This is unreleased software (see [root AGENTS.md](../../../AGENTS.md) § "Pre-release stance: foundation over blast radius"), so there are no on-disk databases or logs to preserve. SQLite does not migrate a v1 database: the `openDatabase` guard now rejects any non-current on-disk `user_version` (`onDisk !== 0 && onDisk !== SCHEMA_VERSION`) — older *or* newer — so a stale v1 DB is cleanly rejected rather than half-read against the new column set. A fresh database stamps the current version; that is the only path that needs to work. + +## What we gave up + +A future session picker now has to derive its preview/ordering from the log (or reintroduce a typed field) rather than reading a ready-made summary row. That is the correct cost: a cache for a feature that does not exist is dead weight that every backend pays to maintain and every contract test pays to assert. The principle — **a passing test pins current behavior, not necessarily correct behavior; behavior can be an artifact of a past compromise** — is now recorded as a standalone convention in [root AGENTS.md](../../../AGENTS.md), with this change as its worked example. diff --git a/packages/acp/tests/load.spec.ts b/packages/acp/tests/load.spec.ts index f06c707d85..b1e4acfda5 100644 --- a/packages/acp/tests/load.spec.ts +++ b/packages/acp/tests/load.spec.ts @@ -166,7 +166,7 @@ describe('acp bridge — session/load replay', () => { loader = await makeBridgeHarness({ storageDir, script: [] }) const otherCwd = '/some/other/workspace' await loader.ctx.sessionPersistence.create({ - version: 1, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd, updatedAt: 1, + version: 1, id: SessionId('elsewhere'), createdAt: 1, cwd: otherCwd, }) await loader.ctx.sessionPersistence.append(SessionId('elsewhere'), [ { type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, @@ -203,7 +203,7 @@ describe('acp bridge — session/load replay', () => { // to the server's launch dir (the request cwd does not override the header). loader = await makeBridgeHarness({ storageDir, script: [] }) await loader.ctx.sessionPersistence.create({ - version: 1, id: SessionId('legacy'), createdAt: 1, updatedAt: 1, // no cwd + version: 1, id: SessionId('legacy'), createdAt: 1, // no cwd }) await loader.ctx.sessionPersistence.append(SessionId('legacy'), [ { type: 'turn/start', seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }, diff --git a/packages/session-persistence-jsonl/README.md b/packages/session-persistence-jsonl/README.md index ee83eb2927..c012886099 100644 --- a/packages/session-persistence-jsonl/README.md +++ b/packages/session-persistence-jsonl/README.md @@ -1,6 +1,6 @@ # @deepseek-ai/dsh-session-persistence-jsonl -The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session plus a small atomic `.summary.json` sidecar for mutable metadata. +The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session. ## On-disk layout @@ -8,7 +8,6 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence / cwd-/ # per-project bucket (or _no-cwd/ when no cwd) .jsonl # header line + one SessionEvent per line (verbatim) - .summary.json # mutable SessionSummary (atomic temp-write + rename) ``` - The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`). diff --git a/packages/session-persistence-jsonl/src/format.ts b/packages/session-persistence-jsonl/src/format.ts index 1a410d9b75..32258c6a37 100644 --- a/packages/session-persistence-jsonl/src/format.ts +++ b/packages/session-persistence-jsonl/src/format.ts @@ -2,15 +2,15 @@ * On-disk format helpers for the JSONL session-persistence backend: path * sanitization (a {@link SessionId} is an unvalidated branded string, so it * MUST be encoded before use in a path — no traversal, no collision), the - * per-cwd directory layout, header-line (de)serialization, the atomic sidecar - * for mutable summary fields, and the truncation-repair offset computation. + * per-cwd directory layout, header-line (de)serialization, and the + * truncation-repair offset computation. * * @module dsh-session-persistence-jsonl/format */ import { createHash } from 'node:crypto' import { join } from 'node:path' -import type { SessionEvent, SessionHeader, SessionId, SessionMeta } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session' /** * The first line of a session's `.jsonl` file: the immutable @@ -109,11 +109,6 @@ export function logPath(root: string, cwd: string | undefined, id: SessionId): s return join(sessionDir(root, cwd), `${encodeSegment(id)}.jsonl`) } -/** The mutable-summary sidecar path for a session (beside its log). */ -export function sidecarPath(root: string, cwd: string | undefined, id: SessionId): string { - return join(sessionDir(root, cwd), `${encodeSegment(id)}.summary.json`) -} - /** Serialize one event as a JSONL line (no trailing newline). */ export function eventLine(event: SessionEvent): string { return JSON.stringify(event) @@ -138,7 +133,7 @@ export function eventLine(event: SessionEvent): string { * (`Session.append` enforces it): only the final turn can be open, so the * preserved tail is at most one unclosed turn. */ -export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEvent[]; committedBytes: number } { +export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionEvent[]; committedBytes: number } { const text = buffer.toString('utf8') // Split into complete (newline-terminated) lines, tracking the byte offset of // each line's end so the truncation point is exact (multi-byte chars make the @@ -233,26 +228,16 @@ export function scanLog(buffer: Buffer): { meta: SessionMeta; events: SessionEve // synthetic closers + new events. const lastPreserved = parsed[preserved.length - 1] const committedBytes = preserved.length > 0 && lastPreserved ? lastPreserved.endByte : headerEntry.endByte - return { meta: metaFrom(headerLine), events: preserved, committedBytes } -} - -/** Build the load-time {@link SessionMeta} from a header line (summary overlaid later). */ -function metaFrom(headerLine: HeaderLine): SessionMeta { - return { - ...fromHeaderLine(headerLine), - updatedAt: headerLine.createdAt, // overlaid by the sidecar in load() - } + return { meta: fromHeaderLine(headerLine), events: preserved, committedBytes } } /** - * Parse just the header line of a log into load-time {@link SessionMeta}, or + * Parse just the header line of a log into a {@link SessionHeader}, or * `undefined` if it is missing/not a header. Used by `list()` to read session * metadata WITHOUT parsing the whole log: a session picker scales with the - * number of sessions, not the total size of every conversation. The summary - * sidecar is overlaid by the caller; `updatedAt` here mirrors `createdAt` until - * then (same as {@link scanLog}'s load-time meta). + * number of sessions, not the total size of every conversation. */ -export function parseHeaderMeta(firstLine: string): SessionMeta | undefined { +export function parseHeaderMeta(firstLine: string): SessionHeader | undefined { let parsed: unknown try { parsed = JSON.parse(firstLine) @@ -260,5 +245,5 @@ export function parseHeaderMeta(firstLine: string): SessionMeta | undefined { return undefined } if (!isHeaderLine(parsed)) return undefined - return metaFrom(parsed) + return fromHeaderLine(parsed) } diff --git a/packages/session-persistence-jsonl/src/index.ts b/packages/session-persistence-jsonl/src/index.ts index 7a0c97637c..110569d3e3 100644 --- a/packages/session-persistence-jsonl/src/index.ts +++ b/packages/session-persistence-jsonl/src/index.ts @@ -5,8 +5,7 @@ * * 1. **The backend** — a concrete {@link SessionPersistence}: one append-only * `.jsonl` event log per session (a header line then one `SessionEvent` per - * line, verbatim including `assistant/chunk` so `seq` stays contiguous) plus - * a small atomic `.summary.json` sidecar for the mutable `SessionSummary`. + * line, verbatim including `assistant/chunk` so `seq` stays contiguous). * Lazy materialization (no file until the first `append`), atomic first * write, and load-time repair of a never-committed crash tail. * @@ -22,16 +21,16 @@ import { Context } from 'cordis' import z from 'schemastery' -import { open, mkdir, readFile, readdir, rename, link, rm, truncate } from 'node:fs/promises' +import { open, mkdir, readFile, readdir, link, rm, truncate } from 'node:fs/promises' import { dirname, resolve } from 'node:path' import { randomBytes } from 'node:crypto' import { SessionPersistence, assertSerializable, seedCoversPrefix, } from '@deepseek-ai/dsh-session-persistence' import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' import { - encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, sidecarPath, toHeaderLine, + encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine, } from './format.ts' export interface Config { @@ -45,7 +44,7 @@ export interface Config { /** Per-session write state held by the backend's in-memory bookkeeping. */ interface SessionState { - meta: SessionMeta + meta: SessionHeader /** The next seq the backend expects to append (the stored log length). */ cursor: number /** Whether the `.jsonl` file has been physically materialized. */ @@ -130,17 +129,17 @@ export class SessionPersistenceJsonl extends SessionPersistence { // --- SessionPersistence backend surface (all serialized per session id) --- - create(meta: SessionMeta): Promise { + create(meta: SessionHeader): Promise { // Snapshot the metadata at call time: the op runs later (behind the // per-session chain) and the snapshot is also stored as the lazy state, so // keeping the caller's object by reference would let a later mutation of // `id`/`cwd` register under one key but materialize under a different - // path/header. A shallow copy is enough — SessionMeta is a flat record. - const snapshot: SessionMeta = { ...meta } + // path/header. A shallow copy is enough — SessionHeader is a flat record. + const snapshot: SessionHeader = { ...meta } return this.serialize(snapshot.id, () => this.createCore(snapshot)) } - private async createCore(meta: SessionMeta): Promise { + private async createCore(meta: SessionHeader): Promise { // Do NOT clobber an existing session. If we already track it, or a log // exists on disk under this id, refuse — the SessionId IS the identity, and // silently resetting state (cursor 0, materialized false) over committed @@ -218,18 +217,15 @@ export class SessionPersistenceJsonl extends SessionPersistence { await this.appendLines(state, events) } // The durable event log is the transaction: advance the cursor as soon as - // the log write commits. The sidecar (mutable summary) is best-effort here - // — a failed sidecar write must NOT reject an append whose log already - // landed (that would desync the cursor and let a retry duplicate seqs). + // the log write commits. state.cursor += events.length - await this.touchSummary(state).catch(() => { /* sidecar is recoverable metadata; log is durable */ }) } - load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { return this.serialize(id, () => this.loadCore(id)) } - private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { + private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { const cwd = this.states.get(id)?.meta.cwd const file = await this.findLog(id, cwd) if (file === undefined) throw new Error(`session "${id}" not found`) @@ -237,9 +233,6 @@ export class SessionPersistenceJsonl extends SessionPersistence { const { meta, events, committedBytes } = scanLog(buffer) this.assertVersion(meta) - const summary = await this.readSidecar(id, meta.cwd) - const fullMeta: SessionMeta = { ...meta, ...summary } - // Crash-recovery: if the log ended mid-turn (an open turn with real, // preserved events but no closing turn/end), close it durably DURING load so // disk, the returned log, and the cursor all agree — both append routes then @@ -253,7 +246,7 @@ export class SessionPersistenceJsonl extends SessionPersistence { // Set state BEFORE the repair writes so they can resolve the log path. const needsTorn = committedBytes < buffer.byteLength const state: SessionState = { - meta: { ...fullMeta }, + meta: { ...meta }, cursor: events.length, materialized: true, } @@ -267,15 +260,12 @@ export class SessionPersistenceJsonl extends SessionPersistence { if (closers.length > 0) { // Durably append the synthetic closers, then advance the cursor to the // balanced length. After this, disk == balanced and the next append (live - // or direct) continues cleanly. No sidecar touch here: load is not a - // summary-changing op (the closers carry no new title/firstPrompt), and - // the next real append bumps `updatedAt` — keeping the summary write off - // the recovery path avoids a second best-effort failure mode. + // or direct) continues cleanly. await this.appendLines(state, closers) state.cursor = balanced.length } - return { meta: fullMeta, events: balanced } + return { meta, events: balanced } } private async adoptLiveDiskPrefix( @@ -290,9 +280,8 @@ export class SessionPersistenceJsonl extends SessionPersistence { throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) } - const summary = await this.readSidecar(session.header.id, meta.cwd) const state: SessionState = { - meta: { ...meta, ...summary }, + meta: { ...meta }, cursor: events.length, materialized: true, owner: session, @@ -306,8 +295,8 @@ export class SessionPersistenceJsonl extends SessionPersistence { if (suffix.length > 0) await this.appendCore(session.header.id, suffix) } - async list(): Promise { - const metas: SessionMeta[] = [] + async list(): Promise { + const metas: SessionHeader[] = [] for (const dir of await this.listCwdDirs()) { for (const name of await this.listJsonl(dir)) { // Read ONLY the header line, not the whole log: a session picker must @@ -318,8 +307,7 @@ export class SessionPersistenceJsonl extends SessionPersistence { if (first === undefined) continue // empty/half-written file const meta = parseHeaderMeta(first) if (meta === undefined) continue // not a session header - const summary = await this.readSidecarForList(meta.id, meta.cwd) - metas.push({ ...meta, ...summary }) + metas.push(meta) } } return metas @@ -367,41 +355,9 @@ export class SessionPersistenceJsonl extends SessionPersistence { const cwd = this.states.get(id)?.meta.cwd const file = await this.findLog(id, cwd) if (file) await rm(file.path, { force: true }) - // Remove the sidecar too. A lazy session (update() before the first - // append()) has a `.summary.json` sidecar but NO `.jsonl` log, and after a - // restart the in-memory cwd is gone — so keying sidecar removal off the log - // or the in-memory cwd would leak its possibly-sensitive title/firstPrompt. - // Scan every cwd bucket for the sidecar by its (sanitized) filename. - await this.removeSidecars(id) this.states.delete(id) } - /** Remove a session's summary sidecar from EVERY cwd bucket (id is unique). */ - private async removeSidecars(id: SessionId): Promise { - const target = `${encodeSegment(id)}.summary.json` - for (const dir of await this.listCwdDirs()) { - await rm(`${dir}/${target}`, { force: true }) - } - } - - update(id: SessionId, summary: Partial): Promise { - return this.serialize(id, () => this.updateCore(id, summary)) - } - - private async updateCore(id: SessionId, summary: Partial): Promise { - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - // Build the NEXT meta separately and commit it to in-memory state only AFTER - // the sidecar write succeeds. update's only durable effect is the sidecar, - // so a failure DOES reject (unlike append, whose log is the transaction and - // sidecar is best-effort) — but if we mutated state.meta first, a later - // touchSummary() on a successful append would persist the rejected - // title/firstPrompt, making a failed update durable after the fact. - const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() } - if (state.materialized) await this.writeSidecar(nextMeta) - state.meta = nextMeta - } - // --- materialization / append / repair --- /** Atomically write the header line + first batch (temp-write, fsync, rename). */ @@ -521,76 +477,6 @@ export class SessionPersistenceJsonl extends SessionPersistence { } } - // --- sidecar (mutable summary) --- - - private async touchSummary(state: SessionState): Promise { - state.meta = { ...state.meta, updatedAt: Date.now() } - await this.writeSidecar(state.meta) - } - - /** - * Atomic sidecar write (temp-write + rename), summary fields only. - * - * Deliberately NOT directory-fsynced (unlike {@link materialize}): the - * sidecar holds mutable, recoverable summary metadata (updatedAt, title, - * firstPrompt), not source-of-truth log data. The rename is atomic so a - * reader never sees a torn file, but a power loss may lose the most recent - * summary — acceptable because it is re-derivable and the durable log (the - * transaction) is independently synced. Strict crash-durability is reserved - * for the event log. - */ - private async writeSidecar(meta: SessionMeta): Promise { - const dir = sessionDir(this.root, meta.cwd) - await mkdir(dir, { recursive: true, mode: 0o700 }) - const path = sidecarPath(this.root, meta.cwd, meta.id) - const summary: SessionSummary = { - updatedAt: meta.updatedAt, - ...meta.title !== undefined ? { title: meta.title } : {}, - ...meta.firstPrompt !== undefined ? { firstPrompt: meta.firstPrompt } : {}, - } - const tmp = `${path}.${randomBytes(6).toString('hex')}.tmp` - // Exclusive owner-only create ('wx', 0o600), matching the log-materialization - // temp write: the sidecar can carry user data (title/firstPrompt), so a - // predictable/pre-existing temp path must never be silently truncated and - // followed (symlink race / disclosure). The random suffix already makes a - // collision unlikely; 'wx' makes reuse an error rather than a clobber. - const handle = await open(tmp, 'wx', 0o600) - try { - await handle.writeFile(JSON.stringify(summary)) - } finally { - await handle.close() - } - await rename(tmp, path) - } - - /** - * Read the mutable-summary sidecar, or `undefined` if it is absent (a session - * that has never been `update()`d). Non-ENOENT failures surface on strict - * load/adopt paths so corrupt metadata does not masquerade as a clean default. - */ - private async readSidecar(id: SessionId, cwd: string | undefined): Promise { - try { - const raw = await readFile(sidecarPath(this.root, cwd, id), 'utf8') - return JSON.parse(raw) as SessionSummary - } catch (error) { - if (isENOENT(error)) return undefined - throw error - } - } - - /** - * Best-effort summary read for list(): a corrupt sidecar should degrade one - * row to header metadata, not hide every session from a picker. - */ - private async readSidecarForList(id: SessionId, cwd: string | undefined): Promise { - try { - return await this.readSidecar(id, cwd) - } catch (error: unknown) { - this.ctx.logger.warn(`session-persistence-jsonl: ignoring unreadable summary for session "${id}" while listing: ${String(error)}`) - return undefined - } - } - // --- discovery helpers --- /** Find a session's log file across cwd buckets (when cwd is unknown). */ @@ -657,7 +543,7 @@ export class SessionPersistenceJsonl extends SessionPersistence { return state } - private assertVersion(meta: SessionMeta): void { + private assertVersion(meta: SessionHeader): void { if (meta.version !== 1) { throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) } @@ -827,7 +713,7 @@ export class SessionPersistenceJsonl extends SessionPersistence { // case 4: a genuinely new session. Register its meta (lazy), then persist // its seed (events present at creation time) once. - const meta: SessionMeta = { ...session.header, updatedAt: Date.now() } + const meta: SessionHeader = { ...session.header } await this.create(meta) // Bind this state to the live session so a later DIFFERENT session reusing // the id is detected as a collision (case 1) rather than silently no-opped. diff --git a/packages/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session-persistence-jsonl/tests/jsonl.spec.ts index 8257a9853c..07cc470c6e 100644 --- a/packages/session-persistence-jsonl/tests/jsonl.spec.ts +++ b/packages/session-persistence-jsonl/tests/jsonl.spec.ts @@ -4,9 +4,9 @@ import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } fr import { tmpdir } from 'node:os' import { join } from 'node:path' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl' -import { encodeSegment, logPath, scanLog, sessionDir, sidecarPath } from '../src/format.ts' +import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts' import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' let root: string @@ -274,7 +274,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => { it('path-traversal session ids are neutralized (no escape from root)', async () => { const evil = SessionId('../../etc/pwn') - const m = { version: 1, id: evil, createdAt: 1, updatedAt: 1 } + const m = { version: 1, id: evil, createdAt: 1 } await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(evil, oneTurnLog()) // The file lives UNDER root, not at ../../etc. @@ -581,23 +581,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { expect(await ctx.sessionPersistence.has(m.id)).toBe(false) }) - it('append resolves even when the best-effort sidecar write fails (log is the transaction)', async () => { - const m = meta('sidecar-fail') - await ctx.sessionPersistence.create(m) - // Force the sidecar write to reject AFTER the durable log append commits. - // The append must still resolve and advance the cursor — a failed sidecar - // is recoverable metadata and must never desync the log (which would let a - // retry duplicate seqs). This exercises the `.catch()` on touchSummary. - const backend = ctx.sessionPersistence as unknown as { writeSidecar: (state: unknown) => Promise } - const original = backend.writeSidecar.bind(backend) - backend.writeSidecar = () => Promise.reject(new Error('disk full')) - await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).resolves.toBeUndefined() - backend.writeSidecar = original - // The durable log landed in full despite the sidecar failure. - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) - }) - it('append rejects non-JSON-serializable undefined-producing data', async () => { const m = meta('undef') await ctx.sessionPersistence.create(m) @@ -610,87 +593,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined() }) - it('update adopts a session that exists only on disk', async () => { - const m = meta('disk-only') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // A fresh backend has no in-memory state → update must adopt from disk. - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.update(m.id, { title: 'adopted' }) - const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('adopted') - await ctx2.fiber.dispose() - }) - - it('a failed update does not become durable via a later append', async () => { - const m = meta('update-fail') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - // Force the sidecar write to fail for the update. - const backend = ctx.sessionPersistence as unknown as { writeSidecar: (meta: unknown) => Promise } - const original = backend.writeSidecar.bind(backend) - backend.writeSidecar = () => Promise.reject(new Error('disk full')) - await expect(ctx.sessionPersistence.update(m.id, { title: 'rejected-title' })).rejects.toThrow(/disk full/) - backend.writeSidecar = original - // A later successful append's touchSummary must NOT persist the rejected - // title (it was never committed to in-memory state). - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }, - { type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } }, - ] as SessionEvent[]) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBeUndefined() - }) - - it('update before the first append keeps summary in memory and writes no orphan sidecar', async () => { - const m = meta('lazy-update', '/a') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.update(m.id, { title: 'secret', firstPrompt: 'sensitive' }) - const sidecar = sidecarPath(root, '/a', m.id) - await expect(stat(sidecar)).rejects.toThrow() - await expect(stat(logPath(root, '/a', m.id))).rejects.toThrow() - - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('secret') - expect(loaded.meta.firstPrompt).toBe('sensitive') - expect((await stat(sidecar)).isFile()).toBe(true) - }) - - it('a lazy update leaves no sidecar that can leak into a future same-id session after restart', async () => { - await ctx.sessionPersistence.create(meta('restart-lazy', '/a')) - await ctx.sessionPersistence.update(SessionId('restart-lazy'), { title: 'secret' }) - await expect(stat(sidecarPath(root, '/a', SessionId('restart-lazy')))).rejects.toThrow() - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - const m2 = meta('restart-lazy', '/a') - await ctx2.sessionPersistence.create(m2) - await ctx2.sessionPersistence.append(m2.id, oneTurnLog()) - const loaded = await ctx2.sessionPersistence.load(m2.id) - expect(loaded.meta.title).toBeUndefined() - await ctx2.fiber.dispose() - }) - - it('delete removes a materialized cwd-bucket sidecar after a restart', async () => { - await ctx.sessionPersistence.create(meta('restart-del', '/a')) - await ctx.sessionPersistence.append(SessionId('restart-del'), oneTurnLog()) - await ctx.sessionPersistence.update(SessionId('restart-del'), { title: 'secret' }) - const sidecar = sidecarPath(root, '/a', SessionId('restart-del')) - expect((await stat(sidecar)).isFile()).toBe(true) - - const ctx2 = new Context() - await ctx2.plugin(SessionStore) - await ctx2.plugin(SessionPersistenceJsonl, { root }) - await ctx2.sessionPersistence.delete(SessionId('restart-del')) - await expect(stat(sidecar)).rejects.toThrow() - await expect(stat(logPath(root, '/a', SessionId('restart-del')))).rejects.toThrow() - await ctx2.fiber.dispose() - }) - it('an abandoned lazy session (never materialized) releases its id for reuse', async () => { // A live session is created then disposed BEFORE its first append: cursor 0, // never materialized, nothing on disk. A new live session reusing the id @@ -765,27 +667,6 @@ describe('SessionPersistenceJsonl: edge cases', () => { expect(ids).toEqual(['p1', 'p2', 'p3']) }) - it('list tolerates one corrupt sidecar and still returns other sessions', async () => { - const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) - const bad = meta('bad-list-summary', '/proj') - await ctx.sessionPersistence.create(bad) - await ctx.sessionPersistence.append(bad.id, oneTurnLog()) - await ctx.sessionPersistence.update(bad.id, { title: 'hidden by corrupt sidecar' }) - await writeFile(sidecarPath(root, '/proj', bad.id), '{not json') - const good = meta('good-list-summary', '/proj') - await ctx.sessionPersistence.create(good) - await ctx.sessionPersistence.append(good.id, oneTurnLog()) - await ctx.sessionPersistence.update(good.id, { title: 'visible' }) - - const listed = await ctx.sessionPersistence.list() - - const badListed = listed.find(m => m.id === bad.id) - expect(badListed).toMatchObject({ id: bad.id }) - expect(badListed).not.toHaveProperty('title') - expect(listed.find(m => m.id === good.id)).toMatchObject({ id: good.id, title: 'visible' }) - expect(warn).toHaveBeenCalledWith(expect.stringContaining('bad-list-summary')) - }) - it('list on an empty root returns nothing', async () => { expect(await ctx.sessionPersistence.list()).toEqual([]) }) @@ -1062,36 +943,13 @@ describe('SessionPersistenceJsonl: edge cases', () => { }) it('round-trips a header with parentSession (fork lineage)', async () => { - const m: SessionMeta = { version: 1, id: SessionId('forked-child'), createdAt: 1, updatedAt: 1, parentSession: SessionId('the-parent') } + const m: SessionHeader = { version: 1, id: SessionId('forked-child'), createdAt: 1, parentSession: SessionId('the-parent') } await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, oneTurnLog()) const loaded = await ctx.sessionPersistence.load(m.id) expect(loaded.meta.parentSession).toBe('the-parent') }) - it('loads a log that has no sidecar (default summary)', async () => { - // Hand-write a valid log WITHOUT a sidecar, then load it. - const dir = sessionDir(root, undefined) - await (await import('node:fs/promises')).mkdir(dir, { recursive: true }) - const header = JSON.stringify({ type: 'session', version: 1, id: 'no-sidecar', createdAt: 5 }) - const body = oneTurnLog().map(e => JSON.stringify(e)).join('\n') - await writeFile(logPath(root, undefined, SessionId('no-sidecar')), header + '\n' + body + '\n') - const loaded = await ctx.sessionPersistence.load(SessionId('no-sidecar')) - expect(loaded.events).toHaveLength(6) - expect(loaded.meta.title).toBeUndefined() // no sidecar → no title - // With no sidecar, updatedAt falls back to the header createdAt (5), NOT 0 - // — reporting an active session as updated at the Unix epoch would be wrong. - expect(loaded.meta.updatedAt).toBe(5) - }) - - it('load rejects a corrupt sidecar instead of treating it as absent', async () => { - const m = meta('bad-sidecar') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - await writeFile(sidecarPath(root, undefined, m.id), '{not json') - await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow() - }) - it('list returns nothing when the root directory does not exist', async () => { const ctx2 = new Context() await ctx2.plugin(SessionStore) diff --git a/packages/session-persistence-sqlite/README.md b/packages/session-persistence-sqlite/README.md index 2025bd8a80..d821b56446 100644 --- a/packages/session-persistence-sqlite/README.md +++ b/packages/session-persistence-sqlite/README.md @@ -6,9 +6,9 @@ A SQLite durable session-persistence backend — a second `SessionPersistence` i ## Storage model -Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionMeta`) lives in a `sessions` row, including the mutable `SessionSummary` fields (`updatedAt`, `title`, `firstPrompt`) that `update()` rewrites without touching the event log. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`has`/`list` report exactly the sessions that have a row), so no separate column is needed. +Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). Out-of-log metadata (`SessionHeader`) lives in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`has`/`list` report exactly the sessions that have a row), so no separate column is needed. -The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by a newer, incompatible build (higher `user_version`) is rejected rather than opened against an unknown layout. +The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and `journal_mode = WAL`. The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software). ## Contract semantics over rows diff --git a/packages/session-persistence-sqlite/src/index.ts b/packages/session-persistence-sqlite/src/index.ts index d28fc8d6f7..34ed6213da 100644 --- a/packages/session-persistence-sqlite/src/index.ts +++ b/packages/session-persistence-sqlite/src/index.ts @@ -7,8 +7,7 @@ * / interrupted-turn-close-on-load semantics the JSONL backend expresses over * file bytes, expressed here over `node:sqlite` rows. Each `SessionEvent` maps * 1:1 onto a row `(session_id, seq, type, time, data)`; `append` is an INSERT - * inside a transaction that asserts the contiguous-seq contract; the mutable - * `SessionSummary` lives in the `sessions` metadata row. + * inside a transaction that asserts the contiguous-seq contract. * * Like the JSONL backend it is also the write-path plugin: it installs the * `session/event` → buffer → `session/flush` drain, persists a fork's seed once @@ -28,7 +27,7 @@ import { SessionPersistence, assertSerializable, seedCoversPrefix, } from '@deepseek-ai/dsh-session-persistence' import { interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' import { openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow, } from './schema.ts' @@ -47,7 +46,7 @@ export interface Config { /** Backend bookkeeping for a session id (NOT the live Session object). */ interface SessionState { - meta: SessionMeta + meta: SessionHeader /** Next seq to write — equals the number of committed events. */ cursor: number /** Whether the session has at least one persisted event (materialized). */ @@ -108,12 +107,12 @@ export class SessionPersistenceSqlite extends SessionPersistence { // --- SessionPersistence backend surface (all serialized per session id) --- - create(meta: SessionMeta): Promise { - const snapshot: SessionMeta = { ...meta } + create(meta: SessionHeader): Promise { + const snapshot: SessionHeader = { ...meta } return this.serialize(snapshot.id, () => this.createCore(snapshot)) } - private async createCore(meta: SessionMeta): Promise { + private async createCore(meta: SessionHeader): Promise { await this.ready if (this.states.has(meta.id)) { throw new Error(`session "${meta.id}" already exists in this backend`) @@ -173,11 +172,7 @@ export class SessionPersistenceSqlite extends SessionPersistence { for (const event of events) { insertEvent.run(id, event.seq, event.type, event.time, JSON.stringify(event.data)) } - // Bump updatedAt on every append (the mutable summary lives in the row). - const updatedAt = Date.now() - this.db.prepare('UPDATE sessions SET updated_at = ? WHERE id = ?').run(updatedAt, id) this.db.exec('COMMIT') - state.meta = { ...state.meta, updatedAt } } catch (error) { this.db.exec('ROLLBACK') throw error @@ -186,11 +181,11 @@ export class SessionPersistenceSqlite extends SessionPersistence { state.cursor += events.length } - load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { + load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { return this.serialize(id, () => this.loadCore(id)) } - private async loadCore(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { + private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { await this.ready const row = this.rowFor(id) if (row === undefined) throw new Error(`session "${id}" not found`) @@ -290,7 +285,7 @@ export class SessionPersistenceSqlite extends SessionPersistence { if (suffix.length > 0) await this.appendCore(session.header.id, suffix) } - async list(): Promise { + async list(): Promise { await this.ready // Every metadata row is a materialized session: the row is written only by // the first append (a created-but-never-appended session has no row), so @@ -320,23 +315,6 @@ export class SessionPersistenceSqlite extends SessionPersistence { this.states.delete(id) } - update(id: SessionId, summary: Partial): Promise { - return this.serialize(id, () => this.updateCore(id, summary)) - } - - private async updateCore(id: SessionId, summary: Partial): Promise { - await this.ready - let state = this.states.get(id) - if (state === undefined) state = await this.adopt(id) - const nextMeta: SessionMeta = { ...state.meta, ...summary, updatedAt: summary.updatedAt ?? Date.now() } - // update's only durable effect is the summary fields; the event log is - // untouched. If the row is not materialized yet (a lazy session updated - // before its first append) there is nothing to write — keep the pending - // summary in memory so the materializing append carries it. - if (state.materialized) this.writeRow(nextMeta) - state.meta = nextMeta - } - // --- row helpers --- /** Fetch a session's row, or undefined if absent. */ @@ -346,32 +324,26 @@ export class SessionPersistenceSqlite extends SessionPersistence { } /** - * Insert-or-replace a session's metadata row. The only callers are the first - * materializing `append` and a post-materialization `update`, so writing the - * row IS the materialization (its existence is the signal `has`/`list` read); - * a never-appended session has no row at all. + * Insert-or-replace a session's metadata row. The only caller is the first + * materializing `append`, so writing the row IS the materialization (its + * existence is the signal `has`/`list` read); a never-appended session has no + * row at all. */ - private writeRow(meta: SessionMeta): void { + private writeRow(meta: SessionHeader): void { this.db.prepare(` - INSERT INTO sessions (id, version, created_at, cwd, parent_session, updated_at, title, first_prompt) - VALUES (?, ?, ?, ?, ?, ?, ?, ?) + INSERT INTO sessions (id, version, created_at, cwd, parent_session) + VALUES (?, ?, ?, ?, ?) ON CONFLICT(id) DO UPDATE SET version = excluded.version, created_at = excluded.created_at, cwd = excluded.cwd, - parent_session = excluded.parent_session, - updated_at = excluded.updated_at, - title = excluded.title, - first_prompt = excluded.first_prompt + parent_session = excluded.parent_session `).run( meta.id, meta.version, meta.createdAt, meta.cwd ?? null, meta.parentSession ?? null, - meta.updatedAt, - meta.title ?? null, - meta.firstPrompt ?? null, ) } @@ -384,7 +356,7 @@ export class SessionPersistenceSqlite extends SessionPersistence { return state } - private assertVersion(meta: SessionMeta): void { + private assertVersion(meta: SessionHeader): void { if (meta.version !== 1) { throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`) } @@ -513,7 +485,7 @@ export class SessionPersistenceSqlite extends SessionPersistence { } // case 4: a genuinely new session. - const meta: SessionMeta = { ...session.header, updatedAt: Date.now() } + const meta: SessionHeader = { ...session.header } await this.create(meta) const created = this.states.get(id) /* v8 ignore next -- create() always sets the state for the id */ diff --git a/packages/session-persistence-sqlite/src/schema.ts b/packages/session-persistence-sqlite/src/schema.ts index 247079c3a8..b6e05a0a3f 100644 --- a/packages/session-persistence-sqlite/src/schema.ts +++ b/packages/session-persistence-sqlite/src/schema.ts @@ -8,18 +8,18 @@ */ import { DatabaseSync } from 'node:sqlite' -import type { SessionEvent, SessionId, SessionMeta } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' /** * The on-disk schema version. Bumped only on a breaking change to the table * layout; orthogonal to a session's own `version` (which versions the EVENT * vocabulary, stored per session in the `sessions` row). */ -export const SCHEMA_VERSION = 1 +export const SCHEMA_VERSION = 2 /** - * A row of the `sessions` table — the out-of-log metadata (`SessionMeta`). The - * row's EXISTENCE is the materialization signal: it is written only by the + * A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}). + * The row's EXISTENCE is the materialization signal: it is written only by the * first `append` (lazy materialization), so a created-but-never-appended * session has no row and is absent from `has`/`list`, mirroring the JSONL * backend's "no file until first append". @@ -30,9 +30,6 @@ export interface SessionRow { created_at: number cwd: string | null parent_session: string | null - updated_at: number - title: string | null - first_prompt: string | null } /** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */ @@ -51,10 +48,11 @@ export interface EventRow { * * The table-layout version is persisted in SQLite's `PRAGMA user_version` and * checked on open: a fresh database (user_version 0) is stamped with the - * current {@link SCHEMA_VERSION}; an existing database with a NEWER version - * (written by a future, incompatible build) is rejected rather than opened - * against a layout this build does not understand. (An older-but-compatible - * version would be migrated here when migrations exist; v1 has none.) + * current {@link SCHEMA_VERSION}; an existing database whose version is NOT the + * current one (written by a different, incompatible build — older or newer) is + * REJECTED rather than opened against a layout this build does not understand. + * There are no migrations: v1 had a different `sessions` layout and is not + * upgraded in place. */ export function openDatabase(path: string): DatabaseSync { const db = new DatabaseSync(path) @@ -62,9 +60,9 @@ export function openDatabase(path: string): DatabaseSync { db.exec('PRAGMA journal_mode = WAL') // `PRAGMA user_version` always returns exactly one row { user_version }. const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number } - if (onDisk > SCHEMA_VERSION) { + if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) { db.close() - throw new Error(`session database at "${path}" has schema version ${onDisk}, newer than this build supports (${SCHEMA_VERSION})`) + throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`) } if (onDisk === 0) { // Fresh (or pre-versioning) database: stamp the current layout version. @@ -78,10 +76,7 @@ export function openDatabase(path: string): DatabaseSync { version INTEGER NOT NULL, created_at INTEGER NOT NULL, cwd TEXT, - parent_session TEXT, - updated_at INTEGER NOT NULL, - title TEXT, - first_prompt TEXT + parent_session TEXT ) STRICT `) db.exec(` @@ -97,17 +92,14 @@ export function openDatabase(path: string): DatabaseSync { return db } -/** Reconstruct the full {@link SessionMeta} from a `sessions` row. */ -export function rowToMeta(row: SessionRow): SessionMeta { +/** Reconstruct the {@link SessionHeader} from a `sessions` row. */ +export function rowToMeta(row: SessionRow): SessionHeader { return { version: row.version, id: row.id as SessionId, createdAt: row.created_at, - updatedAt: row.updated_at, ...row.cwd !== null ? { cwd: row.cwd } : {}, ...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {}, - ...row.title !== null ? { title: row.title } : {}, - ...row.first_prompt !== null ? { firstPrompt: row.first_prompt } : {}, } } diff --git a/packages/session-persistence-sqlite/tests/sqlite.spec.ts b/packages/session-persistence-sqlite/tests/sqlite.spec.ts index d6216b7b50..6cdc0dcb7a 100644 --- a/packages/session-persistence-sqlite/tests/sqlite.spec.ts +++ b/packages/session-persistence-sqlite/tests/sqlite.spec.ts @@ -4,7 +4,7 @@ import { mkdtemp, rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' -import type { Session, SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' +import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite' import { openDatabase, scanRows, type EventRow } from '../src/schema.ts' import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' @@ -232,14 +232,23 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { await b2.dispose() }) - it('rejects opening a database whose schema version is newer than this build', async () => { + it('rejects opening a database whose schema version is not the current build (newer OR older)', async () => { const path = await freshDbPath() openDatabase(path).close() // stamp user_version = SCHEMA_VERSION // Bump user_version past what this build supports. - const db = openDatabase(path) - db.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`) - db.close() - expect(() => openDatabase(path)).toThrow(/newer than this build/) + const dbNewer = openDatabase(path) + dbNewer.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`) + dbNewer.close() + expect(() => openDatabase(path)).toThrow(/incompatible with this build/) + + // A stale OLDER version (e.g. a pre-summary-drop v1 DB) is also rejected — + // we do not migrate (unreleased software, no backward-compat). + const olderPath = await freshDbPath() + openDatabase(olderPath).close() + const dbOlder = openDatabase(olderPath) + dbOlder.exec('PRAGMA user_version = 1') + dbOlder.close() + expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/) }) it('append snapshots the batch: mutating an event after the call does not corrupt the persisted copy', async () => { @@ -323,7 +332,6 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path }) await ctx1.sessionPersistence.create(m) await ctx1.sessionPersistence.append(m.id, oneTurnLog()) - await ctx1.sessionPersistence.update(m.id, { title: 'T', firstPrompt: 'hi' }) await fiber1.dispose() const ctx2 = new Context() @@ -331,7 +339,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path }) expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id) const loaded = await ctx2.sessionPersistence.load(m.id) - expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj', title: 'T', firstPrompt: 'hi' }) + expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj' }) expect(loaded.events).toEqual(oneTurnLog()) await fiber2.dispose() }) @@ -340,8 +348,8 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { const path = await freshDbPath() // Materialize a row with version 2 directly via the real schema. const db = openDatabase(path) - db.prepare('INSERT INTO sessions (id, version, created_at, updated_at) VALUES (?, ?, ?, ?)') - .run('v2', 2, 1, 1) + db.prepare('INSERT INTO sessions (id, version, created_at) VALUES (?, ?, ?)') + .run('v2', 2, 1) db.close() const ctx = new Context() @@ -372,7 +380,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => { }) it('exposes the schema version constant', () => { - expect(SCHEMA_VERSION).toBe(1) + expect(SCHEMA_VERSION).toBe(2) }) }) @@ -468,22 +476,6 @@ describe('SessionPersistenceSqlite: write path (session/event → flush)', () => await expect(ctx2.parallel('session/flush', s2)).rejects.toThrow(/id collision/) await fiber2.dispose() }) - - it('update before the first append keeps the summary in memory and the session lazy', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' }) - const m = meta('lazy-update') - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.update(m.id, { title: 'pending' }) - // Still lazy: no materialized row yet. - expect(await ctx.sessionPersistence.has(m.id)).toBe(false) - // The first append materializes and carries the pending title. - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const loaded = await ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('pending') - await fiber.dispose() - }) }) describe('SessionPersistenceSqlite: edge cases', () => { @@ -529,21 +521,6 @@ describe('SessionPersistenceSqlite: edge cases', () => { await b2.dispose() }) - it('update adopts a session that exists only in the DB (fresh instance)', async () => { - const path = await freshDbPath() - const m = meta('adopt-update') - const b1 = await backend(path) - await b1.ctx.sessionPersistence.create(m) - await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) - await b1.dispose() - - const b2 = await backend(path) - await b2.ctx.sessionPersistence.update(m.id, { title: 'after restart' }) - const loaded = await b2.ctx.sessionPersistence.load(m.id) - expect(loaded.meta.title).toBe('after restart') - await b2.dispose() - }) - it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => { const path = await freshDbPath() const m = meta('rollback-insert') @@ -574,7 +551,7 @@ describe('SessionPersistenceSqlite: edge cases', () => { it('round-trips a header with parentSession (fork lineage)', async () => { const { ctx, dispose } = await backend() - const m: SessionMeta = { ...meta('child'), parentSession: SessionId('parent') } + const m: SessionHeader = { ...meta('child'), parentSession: SessionId('parent') } await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, oneTurnLog()) const loaded = await ctx.sessionPersistence.load(m.id) diff --git a/packages/session-persistence/README.md b/packages/session-persistence/README.md index 4da1f00df5..521020e80d 100644 --- a/packages/session-persistence/README.md +++ b/packages/session-persistence/README.md @@ -1,8 +1,8 @@ # @deepseek-ai/dsh-session-persistence -The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, list, and update sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../docs/rfc/implemented/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface. +The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../docs/rfc/implemented/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface. -The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionMeta`, owned by `dsh-session` and re-exported here. +The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here. ## Service API (`ctx.sessionPersistence`) @@ -11,9 +11,8 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l | `create(meta): Promise` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). | | `append(id, events): Promise` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. | | `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. | -| `list(): Promise` | Lightweight listing from metadata, no full-log parse. | +| `list(): Promise` | Lightweight listing from metadata, no full-log parse. | | `has(id)` / `delete(id)` | Existence / removal. A zero-event lazily-materialized session is absent from `has`/`list`. | -| `update(id, summary): Promise` | Update mutable `SessionSummary` fields without touching the append-only log. | ## Invariants every backend must honor @@ -30,4 +29,4 @@ Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file l ## Metadata types -Re-exported from `dsh-session`: `SessionHeader` (immutable: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`), `SessionSummary` (mutable: `updatedAt`, `title?`, `firstPrompt?`), `SessionMeta` (their intersection). +Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`). diff --git a/packages/session-persistence/src/index.ts b/packages/session-persistence/src/index.ts index 0ce1e25146..48402b54a7 100644 --- a/packages/session-persistence/src/index.ts +++ b/packages/session-persistence/src/index.ts @@ -1,7 +1,7 @@ /** * The durable session-persistence seam (`ctx.sessionPersistence`): an abstract * service defining WHAT a persistence backend does — durably store, reload, - * list, and update sessions — without saying HOW. Implementations subclass + * and list sessions — without saying HOW. Implementations subclass * {@link SessionPersistence} and register themselves as the * `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl` * (an append-only JSONL log per session) is the first and @@ -15,7 +15,7 @@ * parallel "persisted message" type the log must be converted to and from * (faithful to the event-sourced model: the log is the single source of * truth). Metadata that is NOT replayable conversation state (format version, - * cwd, lineage) travels separately as {@link SessionMeta}, which is owned by + * cwd, lineage) travels separately as {@link SessionHeader}, which is owned by * `dsh-session` and re-exported here. * * @module @deepseek-ai/dsh-session-persistence @@ -23,10 +23,10 @@ import { Context, Service } from 'cordis' import { isJsonValue } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionId, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session' // Re-export the metadata vocabulary so consumers import it from the seam. -export type { SessionHeader, SessionSummary, SessionMeta } from '@deepseek-ai/dsh-session' +export type { SessionHeader } from '@deepseek-ai/dsh-session' declare module 'cordis' { interface Context { @@ -102,7 +102,7 @@ export abstract class SessionPersistence extends Service { * created-but-never-appended session is absent from {@link has}/{@link list} * — abandoned sessions leave nothing behind. */ - abstract create(meta: SessionMeta): Promise + abstract create(meta: SessionHeader): Promise /** * Durably persist a batch of events (called from the write-behind drain at @@ -114,7 +114,7 @@ export abstract class SessionPersistence extends Service { abstract append(id: SessionId, events: readonly SessionEvent[]): Promise /** - * Reload a session: its {@link SessionMeta} plus the event log up to the last + * Reload a session: its {@link SessionHeader} plus the event log up to the last * durable checkpoint. Returns `meta` AND `events` so the live session is * reconstructed with its `cwd`/lineage, not just its log. * @@ -135,24 +135,16 @@ export abstract class SessionPersistence extends Service { * unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for * the crash-recovery contract. */ - abstract load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> + abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> /** Lightweight listing from metadata, without a full-log parse. */ - abstract list(): Promise + abstract list(): Promise /** Whether a session is durably present (materialized). */ abstract has(id: SessionId): Promise /** Remove a session and all its persisted artifacts. */ abstract delete(id: SessionId): Promise - - /** - * Update mutable metadata ({@link SessionSummary}: `updatedAt`, `title`, - * `firstPrompt`) WITHOUT touching the append-only event log. A backend - * stores the summary beside the log (a sidecar file, a header row) and - * rewrites only it. - */ - abstract update(id: SessionId, summary: Partial): Promise } export default SessionPersistence diff --git a/packages/session-persistence/tests/contract.ts b/packages/session-persistence/tests/contract.ts index 73f6f653bd..704e0abfb0 100644 --- a/packages/session-persistence/tests/contract.ts +++ b/packages/session-persistence/tests/contract.ts @@ -8,9 +8,9 @@ * @module @deepseek-ai/dsh-session-persistence/tests/contract */ -import { describe, expect, it, vi } from 'vitest' +import { describe, expect, it } from 'vitest' import { SessionId } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionMeta } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { CallId } from '@deepseek-ai/dsh-llm' import type { SessionPersistence } from '../src/index.ts' @@ -20,13 +20,12 @@ export interface ContractBackend { dispose: () => Promise } -/** Build a minimal {@link SessionMeta} for a session id. */ -export function meta(id: string, cwd?: string): SessionMeta { +/** Build a minimal {@link SessionHeader} for a session id. */ +export function meta(id: string, cwd?: string): SessionHeader { return { version: 1, id: SessionId(id), createdAt: 1000, - updatedAt: 1000, ...cwd !== undefined ? { cwd } : {}, } } @@ -242,31 +241,5 @@ export function runPersistenceContract(name: string, make: () => Promise { - const { persistence, dispose } = await make() - try { - const m = meta('s7') - const log = oneTurnLog() - await persistence.create(m) - await persistence.append(m.id, log) - const beforeUpdate = (await persistence.load(m.id)).meta.updatedAt - vi.useFakeTimers() - vi.setSystemTime(beforeUpdate + 1_000) - try { - await persistence.update(m.id, { title: 'My session', firstPrompt: 'hi' }) - } finally { - vi.useRealTimers() - } - - const loaded = await persistence.load(m.id) - expect(loaded.meta.title).toBe('My session') - expect(loaded.meta.firstPrompt).toBe('hi') - expect(loaded.meta.updatedAt).toBe(beforeUpdate + 1_000) - expect(loaded.events).toEqual(log) // log untouched - } finally { - await dispose() - } - }) }) } diff --git a/packages/session-persistence/tests/persistence.spec.ts b/packages/session-persistence/tests/persistence.spec.ts index 5c0a29131f..04ac108ed3 100644 --- a/packages/session-persistence/tests/persistence.spec.ts +++ b/packages/session-persistence/tests/persistence.spec.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from 'vitest' import { Context } from 'cordis' import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session' -import type { SessionEvent, SessionMeta, SessionSummary } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts' import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' @@ -12,10 +12,10 @@ import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' * `@deepseek-ai/dsh-session-persistence-jsonl`. */ class MemoryPersistence extends SessionPersistence { - private store = new Map() - private pending = new Map() + private store = new Map() + private pending = new Map() - async create(m: SessionMeta): Promise { + async create(m: SessionHeader): Promise { // Lazy: record the intended meta, but stay absent from has/list until the // first append materializes the session. this.pending.set(m.id, m) @@ -43,7 +43,7 @@ class MemoryPersistence extends SessionPersistence { } } - async load(id: SessionId): Promise<{ meta: SessionMeta; events: SessionEvent[] }> { + async load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> { const entry = this.store.get(id) if (!entry) throw new Error(`session "${id}" not found`) // Honor the crash-recovery contract: if the stored log ends mid-turn, close @@ -54,7 +54,7 @@ class MemoryPersistence extends SessionPersistence { return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) } } - async list(): Promise { + async list(): Promise { return [...this.store.values()].map(e => structuredClone(e.meta)) } @@ -66,11 +66,6 @@ class MemoryPersistence extends SessionPersistence { this.store.delete(id) this.pending.delete(id) } - - async update(id: SessionId, summary: Partial): Promise { - const entry = this.store.get(id) - if (entry) Object.assign(entry.meta, summary, { updatedAt: summary.updatedAt ?? Date.now() }) - } } // Run the shared contract against the in-memory backend. diff --git a/packages/session/README.md b/packages/session/README.md index 7cf443fa71..bdd825e2e7 100644 --- a/packages/session/README.md +++ b/packages/session/README.md @@ -31,9 +31,7 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`. ### Metadata types (`types.ts`) -- `SessionHeader` — immutable, written once: `{ version, id, createdAt, cwd?, parentSession? }`. -- `SessionSummary` — mutable, updateable without touching the log: `{ updatedAt, title?, firstPrompt? }`. -- `SessionMeta = SessionHeader & SessionSummary` — owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export these rather than own them (which would force a package cycle). +- `SessionHeader` — immutable session metadata, written once: `{ version, id, createdAt, cwd?, parentSession? }`. Owned here (beside `SessionId`) because `Session.header` is typed by it; persistence backends re-export it rather than own it (which would force a package cycle). ### Session event vocabulary (`types.ts`) @@ -45,7 +43,7 @@ Also defines `TurnTriggerMap` and `TurnEndReasonMap` (merge-extensible sum types ### Extension points -- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`/`SessionSummary`/`SessionMeta`, `session.header`) is what such a backend stores beside the log. +- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log. - Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. ### What is NOT here (TODO) diff --git a/packages/session/src/types.ts b/packages/session/src/types.ts index 53fca39795..cd1605d93a 100644 --- a/packages/session/src/types.ts +++ b/packages/session/src/types.ts @@ -30,29 +30,6 @@ export interface SessionHeader { parentSession?: SessionId } -/** - * Mutable session metadata — updateable without touching the append-only log. - * A persistence backend stores this beside the log (a sidecar file, a header - * row) and rewrites only it on update. - */ -export interface SessionSummary { - /** Unix epoch milliseconds of the last mutation (event append or update). */ - updatedAt: number - /** Human-facing title (derived/edited), if any. */ - title?: string - /** The first user prompt, cached for listing previews. */ - firstPrompt?: string -} - -/** - * Full session metadata: the immutable {@link SessionHeader} merged with the - * mutable {@link SessionSummary}. Owned here in `dsh-session` (beside - * {@link SessionId}) because `Session.header` is typed by it; the persistence - * package imports/re-exports these rather than owning them, which would force - * a package cycle. - */ -export type SessionMeta = SessionHeader & SessionSummary - /** * Options for creating a {@link Session} via the store. `seed` replays/forks * an existing event log; `meta` carries the caller-supplied storage fields the