refactor(session-persistence): extract a shared write coordinator
The JSONL and SQLite backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the four maps (states/buffers/chains/inits), installWritePath, initFor, onCreated's four adoption cases, flush, drain, serialize, adopt/adoptLivePrefix, assertVersion, and the create/append/load/ has/delete skeletons. Only the storage primitives (write bytes vs INSERT rows) differed, so every fix landed twice. Extract that orchestration into a PersistenceCoordinator in the seam package. Each backend composes one (new PersistenceCoordinator(ctx, this)), implements a small PersistenceBackend hook interface (loadStored, loadLive, appendBatch, commitRepair, deleteStored, list, optional close), and delegates its six public service methods to it. Composition, not inheritance — a backend exposes only the hooks, can't reach the coordinator's private state, and the public SessionPersistence API is unchanged so a third-party backend may still implement it directly. The crash-repair torn-tail token is OPAQUE: the coordinator computes the synthetic closers (it owns interruptedTurnClosers) but only tests `tornMarker !== undefined` and round-trips it to commitRepair, never inspecting it (JSONL = byte offset, SQLite = seq). loadStored vs loadLive stay distinct so HMR adoption is cwd-scoped (a same-id log at a different cwd is a collision, not a resume). appendBatch carries meta so lazy-materialize + first-batch commit atomically (no separate materialize hook). Tests: the duplicated orchestration tests (adoption, HMR, collision, dispose-drain, crash-tail) move into one runCoordinatorContract suite run once per backend (memory + jsonl + sqlite) via hook fixtures; per-backend specs keep only storage mechanics. A through-coordinator torn-tail test per real backend keeps the commitRepair-with-marker branch covered under the 100% gate. Net -112 lines (the dedup outweighs the new coordinator + shared suite); 100% coverage; backends shrank ~1200 lines of duplicated churn. Migrates the write-coordinator RFC proposed -> implemented.
This commit is contained in:
Tianyi Cui
@@ -21,11 +21,31 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable).
|
||||
- **Durability.** `append` returns only once the batch is durable.
|
||||
|
||||
## The write coordinator
|
||||
|
||||
The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows).
|
||||
|
||||
`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its six public service methods to the coordinator. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice).
|
||||
|
||||
The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinator and storage):
|
||||
|
||||
| Hook | Role |
|
||||
|---|---|
|
||||
| `name` | Backend label for the dispose-failure `AggregateError`. |
|
||||
| `loadStored(id)` | Read a stored prefix by id, scanning ANY storage scope. Used by resume/load and, via `!== undefined`, the create-collision probe. Returns an opaque `tornMarker` iff a torn tail must be truncated. |
|
||||
| `loadLive(id, cwd)` | Read a stored prefix SCOPED to `cwd` (HMR live-adoption must only adopt a log at the SAME cwd; a same-id log elsewhere is a collision, not a resume). A globally-unique-id backend ignores `cwd`. |
|
||||
| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. |
|
||||
| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). |
|
||||
| `deleteStored(id)` / `list()` | Remove a stored artifact / list all stored metadata. |
|
||||
| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. |
|
||||
|
||||
The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). The public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator. See [the write-coordinator RFC](../../docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
## Testing backends
|
||||
|
||||
Import `runPersistenceContract` from `tests/contract.ts` and call it with a factory that yields a fresh, empty backend plus a teardown. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics; a backend's own spec adds implementation-specific tests (crash repair, path sanitization) on top.
|
||||
Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top.
|
||||
|
||||
Two backends run this suite: `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). Both passing the same contract is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
|
||||
## Metadata types
|
||||
|
||||
|
||||
556
packages/session-persistence/src/coordinator.ts
Normal file
556
packages/session-persistence/src/coordinator.ts
Normal file
@@ -0,0 +1,556 @@
|
||||
/**
|
||||
* The backend-agnostic write-path orchestration shared by every first-party
|
||||
* {@link SessionPersistence} backend.
|
||||
*
|
||||
* The two durable backends (`dsh-session-persistence-jsonl` over file bytes,
|
||||
* `dsh-session-persistence-sqlite` over `node:sqlite` rows) were byte-identical
|
||||
* — or same-algorithm — for ALL of their orchestration: the in-memory
|
||||
* bookkeeping (the per-id state, the write-behind buffers, the per-id
|
||||
* serialization chains, the per-session init promises), the `session/event` →
|
||||
* buffer → `session/flush` drain, lazy materialization, crash-tail repair on
|
||||
* load, the four `session/created` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives
|
||||
* differed (write bytes vs. INSERT rows). {@link PersistenceCoordinator} owns
|
||||
* the orchestration once; a backend supplies the storage primitives as a small
|
||||
* {@link PersistenceBackend} hook object.
|
||||
*
|
||||
* The abstract {@link SessionPersistence} service's public API is unchanged: a
|
||||
* backend still IS a `SessionPersistence` (its six public methods delegate to a
|
||||
* coordinator it composes), so a third-party backend MAY implement the service
|
||||
* directly without using the coordinator at all.
|
||||
*
|
||||
* See the write-coordinator RFC (docs/rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md)
|
||||
* for the design rationale (composition over inheritance, the opaque torn marker).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/coordinator
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import { interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { assertSerializable, seedCoversPrefix } from './index.ts'
|
||||
|
||||
/**
|
||||
* A stored session's durable prefix as read back from a backend: its
|
||||
* {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix,
|
||||
* and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must
|
||||
* be truncated before further writes.
|
||||
*
|
||||
* The coordinator NEVER inspects `tornMarker`'s value — it only tests
|
||||
* `!== undefined` (is there a tail to repair?) and passes the value back to
|
||||
* {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker
|
||||
* type: the JSONL backend uses the byte offset to truncate to, the SQLite
|
||||
* backend uses the seq to delete from (both happen to be `number`).
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
tornMarker?: TornMarker
|
||||
}
|
||||
|
||||
/**
|
||||
* The storage seam between {@link PersistenceCoordinator} and a concrete
|
||||
* backend: the minimal set of durable primitives the orchestration calls. A
|
||||
* backend implements these (over files, rows, an object store, …); the
|
||||
* coordinator supplies everything else (buffering, serialization, cursors,
|
||||
* adoption, crash repair sequencing, dispose quiescence).
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token (see
|
||||
* {@link StoredPrefix}). The coordinator treats it as fully opaque.
|
||||
*/
|
||||
export interface PersistenceBackend<TornMarker = unknown> {
|
||||
/** Human-readable backend name, used in the dispose-failure AggregateError. */
|
||||
readonly name: string
|
||||
|
||||
/**
|
||||
* Read a stored prefix by id, scanning ANY storage scope (for JSONL: every
|
||||
* cwd bucket). Returns `undefined` if no stored artifact exists. Used by
|
||||
* resume/load, and — via `!== undefined` — by the create-collision probe.
|
||||
* The returned `tornMarker` is present iff there is a torn tail to truncate.
|
||||
*/
|
||||
loadStored(id: SessionId): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Read a stored prefix SCOPED to `cwd`. Deliberately distinct from
|
||||
* {@link loadStored}: HMR live-adoption must only adopt a persisted log at the
|
||||
* SAME cwd as the live session (a same-id log at a different cwd is a
|
||||
* collision, not a resume) — conflating the two reintroduces a cross-cwd
|
||||
* adoption bug. For a globally-unique-id backend (SQLite) `cwd` is ignored.
|
||||
*/
|
||||
loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Durably append a CONTIGUOUS batch, lazily materializing the session first
|
||||
* when `!isMaterialized`. The materialize-write and the first event batch MUST
|
||||
* commit ATOMICALLY (a crash between them must not leave a materialized-but-
|
||||
* empty session). Returns once the batch is durable.
|
||||
*/
|
||||
appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void>
|
||||
|
||||
/**
|
||||
* Make a crash repair durable: truncate the torn tail (iff
|
||||
* `tornMarker !== undefined`) and append `closers` (iff any). NOT required to
|
||||
* be atomic — a file backend may truncate-then-append in two fsync'd steps.
|
||||
* Used by load (truncate + synthetic closers) and by live-adoption (truncate
|
||||
* only, `closers = []`).
|
||||
*/
|
||||
commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/** Remove the stored artifact for `id` (the coordinator clears in-memory state). */
|
||||
deleteStored(id: SessionId): Promise<void>
|
||||
|
||||
/** List all stored (materialized) sessions' metadata. */
|
||||
list(): Promise<SessionHeader[]>
|
||||
|
||||
/**
|
||||
* Optional lifecycle teardown (e.g. close a database handle). Awaited by the
|
||||
* coordinator's dispose effect AFTER the quiescence drain. A stateless file
|
||||
* backend omits it.
|
||||
*/
|
||||
close?(): Promise<void>
|
||||
}
|
||||
|
||||
/** Per-session write state held by the coordinator's in-memory bookkeeping. */
|
||||
interface SessionState {
|
||||
meta: SessionHeader
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/** Whether the session has been physically materialized. */
|
||||
materialized: boolean
|
||||
/**
|
||||
* The live Session this state was bound to via `onCreated`, if any. State
|
||||
* created through the public `create()`/`load()` API has no owner; state bound
|
||||
* to a live session lets `onCreated` reject a second, unrelated session on the
|
||||
* same id (a collision) instead of silently no-opping.
|
||||
*/
|
||||
owner?: Session
|
||||
}
|
||||
|
||||
/** Collect the rejection reasons from a set of promises (none-throwing). */
|
||||
async function settledErrors(promises: Iterable<Promise<unknown>>): Promise<unknown[]> {
|
||||
const settled = await Promise.allSettled([...promises])
|
||||
const errors: unknown[] = []
|
||||
for (const result of settled) {
|
||||
if (result.status === 'rejected') errors.push(result.reason)
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the backend-agnostic session write-path orchestration. A backend
|
||||
* constructs one (`new PersistenceCoordinator(ctx, this)`), implements
|
||||
* {@link PersistenceBackend}, and delegates its six public service methods to
|
||||
* the matching coordinator methods.
|
||||
*
|
||||
* All per-id operations are serialized (a per-id promise chain) so concurrent
|
||||
* flushes / a flush racing a load never interleave storage writes. The
|
||||
* constructor installs the write-path listeners and the dispose effect.
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token.
|
||||
*/
|
||||
export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
/** Backend bookkeeping keyed by session id (NOT the live Session object). */
|
||||
private states = new Map<string, SessionState>()
|
||||
/** Write-behind buffers keyed by the live Session (write path). */
|
||||
private buffers = new Map<Session, SessionEvent[]>()
|
||||
/**
|
||||
* Per-session serialization: every operation chains onto the prior one for the
|
||||
* same id, so writes for one session never interleave. Keyed by session id.
|
||||
*/
|
||||
private chains = new Map<string, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise.
|
||||
*
|
||||
* Public (readonly) so a backend can expose it for white-box tests that await
|
||||
* a specific session's init (there is no public API to await one init); the
|
||||
* coordinator itself only ever mutates it internally.
|
||||
*/
|
||||
readonly inits = new Map<Session, Promise<void>>()
|
||||
|
||||
constructor(private ctx: Context, private backend: PersistenceBackend<TornMarker>) {
|
||||
this.installWritePath()
|
||||
}
|
||||
|
||||
// --- public surface (the backend's service methods delegate here) ---
|
||||
|
||||
/**
|
||||
* Register a new session's metadata (lazy: no physical write until the first
|
||||
* {@link append}). Rejects if the id is already tracked or already persisted.
|
||||
*/
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is stored as the lazy state, so keeping
|
||||
// the caller's object by reference would let a later mutation of `id`/`cwd`
|
||||
// register under one key but materialize under a different path/header.
|
||||
const snapshot: SessionHeader = { ...meta }
|
||||
return this.serialize(snapshot.id, () => this.createCore(snapshot))
|
||||
}
|
||||
|
||||
private async createCore(meta: SessionHeader): Promise<void> {
|
||||
// Do NOT clobber an existing session: the SessionId IS the identity.
|
||||
if (this.states.has(meta.id)) {
|
||||
throw new Error(`session "${meta.id}" already exists in this backend`)
|
||||
}
|
||||
// A persisted artifact under this id (in ANY scope) blocks creation: load/
|
||||
// has/resume identify a session by id alone, so a second artifact would make
|
||||
// resume nondeterministic.
|
||||
if (await this.backend.loadStored(meta.id) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`)
|
||||
}
|
||||
// Pure lazy: record intent only. No artifact until the first append.
|
||||
this.states.set(meta.id, { meta, cursor: 0, materialized: false })
|
||||
}
|
||||
|
||||
// `async` so the synchronous validate/clone below reject (not throw) per the
|
||||
// Promise<void> contract — callers use `await expect(...).rejects`.
|
||||
/**
|
||||
* Durably persist a batch of events. Honors the append-only and contiguous-seq
|
||||
* contracts; rejects non-JSON-serializable `event.data`.
|
||||
*/
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Validate serializability BEFORE cloning so a bad event surfaces the typed
|
||||
// error rather than an opaque DataCloneError from structuredClone.
|
||||
assertSerializable(events)
|
||||
// Deep-snapshot the batch HERE, before the op waits behind the per-session
|
||||
// chain: a caller that mutates a live array (e.g. session.events) — or an
|
||||
// event inside it — before the op runs would otherwise have those changes
|
||||
// persisted. The clone is taken synchronously (at call time).
|
||||
const batch = events.map(e => structuredClone(e))
|
||||
return this.serialize(id, () => this.appendCore(id, batch))
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
if (events.length === 0) return
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id) // calls loadCore, not load
|
||||
|
||||
// Contiguity contract: each event's seq must continue the stored log.
|
||||
for (const [i, event] of events.entries()) {
|
||||
if (event.seq !== state.cursor + i) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
await this.backend.appendBatch(state.meta, events, state.materialized)
|
||||
// The durable write is the transaction: mark materialized + advance the
|
||||
// cursor as soon as it commits (uniform across backends).
|
||||
state.materialized = true
|
||||
state.cursor += events.length
|
||||
}
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint, with any interrupted final turn durably closed (synthetic
|
||||
* boundary events) during load.
|
||||
*/
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.serialize(id, () => this.loadCore(id))
|
||||
}
|
||||
|
||||
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const stored = await this.backend.loadStored(id)
|
||||
if (stored === undefined) throw new Error(`session "${id}" not found`)
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (real, preserved events but no
|
||||
// closing turn/end), close it durably DURING load so disk, the returned log,
|
||||
// and the cursor all agree. The interrupted turn's real events are preserved,
|
||||
// never truncated (a turn can be huge — the session-persistence RFC); only a
|
||||
// never-fully-written torn tail fragment is discarded.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Make the repair durable (truncate the torn tail + append the synthetic
|
||||
// closers) BEFORE recording state — commitRepair takes `meta` directly, so
|
||||
// there is no state-path ordering dependency (uniform across backends).
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
await this.backend.commitRepair(meta, tornMarker, closers)
|
||||
}
|
||||
// The state keeps its OWN copy of the meta; the returned value is separate so
|
||||
// a consumer mutating loaded.meta cannot corrupt the backend's metadata.
|
||||
this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true })
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
// NOTE: there is deliberately no coordinator `list()`. Listing needs none of
|
||||
// the coordinator's orchestration (no per-id serialization, no cursor, no
|
||||
// in-memory state) — it is a pure read of stored metadata. A backend's public
|
||||
// `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it
|
||||
// through the coordinator would only forward to that same hook, so the
|
||||
// coordinator stays out of the listing path entirely.
|
||||
|
||||
/** Whether a session is durably present (materialized). */
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
const state = this.states.get(id)
|
||||
if (state?.materialized) return true
|
||||
// Probe storage scoped to the tracked cwd if known, else any scope. A tracked
|
||||
// lazy session has a known cwd, so loadLive(id, cwd) hits the exact artifact
|
||||
// path — a storage fault there (e.g. a non-ENOENT lookup error) must surface,
|
||||
// not be masked by an any-scope scan that filters a non-directory bucket out.
|
||||
// For an untracked id `cwd` is undefined, where loadLive scans any scope (=
|
||||
// loadStored), so this single call covers both.
|
||||
return (await this.backend.loadLive(id, state?.meta.cwd)) !== undefined
|
||||
}
|
||||
|
||||
/** Remove a session and all its persisted artifacts. */
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.serialize(id, () => this.deleteCore(id))
|
||||
}
|
||||
|
||||
private async deleteCore(id: SessionId): Promise<void> {
|
||||
await this.backend.deleteStored(id)
|
||||
this.states.delete(id)
|
||||
}
|
||||
|
||||
// --- per-id serialization + adoption helpers ---
|
||||
|
||||
/**
|
||||
* Run `op` after any in-flight operation for the same session id, so writes for
|
||||
* one session never interleave. Errors do not poison the chain. NOTE: serialized
|
||||
* public methods must NOT call each other (deadlock); they call the unserialized
|
||||
* `*Core` helpers instead.
|
||||
*/
|
||||
private serialize<T>(id: SessionId, op: () => Promise<T>): Promise<T> {
|
||||
const prior = this.chains.get(id) ?? Promise.resolve()
|
||||
const next = prior.then(op, op)
|
||||
// Keep the chain alive but swallow this op's rejection for the NEXT waiter
|
||||
// (the caller still sees the real rejection via `next`).
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
return next
|
||||
}
|
||||
|
||||
/** Build a state for a session discovered in storage but not yet in memory. */
|
||||
private async adopt(id: SessionId): Promise<SessionState> {
|
||||
// loadCore (NOT load) — adopt runs inside an already-serialized op, so
|
||||
// re-entering the chain via the public load() would deadlock.
|
||||
await this.loadCore(id)
|
||||
const state = this.states.get(id)
|
||||
/* v8 ignore next -- loadCore always sets the state for the id */
|
||||
if (!state) throw new Error(`failed to adopt session "${id}"`)
|
||||
return state
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version !== 1) {
|
||||
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v1 is supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the init
|
||||
// promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Snapshot + buffer every event (the live object is mutable; clone so a later
|
||||
// in-place mutation cannot rewrite a buffered event). Serializability is
|
||||
// guaranteed at the source (Session.append), so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every init + final drain BEFORE
|
||||
// returning, then close the backend's own resources (AFTER the drain), so no
|
||||
// write lands after teardown and a close failure never MASKS a drain error.
|
||||
ctx.effect(() => async () => {
|
||||
let disposeError: unknown
|
||||
try {
|
||||
const errors = [
|
||||
...await settledErrors(this.inits.values()),
|
||||
...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))),
|
||||
...await settledErrors(this.chains.values()),
|
||||
]
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, `${this.backend.name} dispose failed`)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
disposeError = error
|
||||
throw error
|
||||
} finally {
|
||||
try {
|
||||
await this.backend.close?.()
|
||||
} catch (closeError: unknown) {
|
||||
// A close failure can only add teardown context; keep the already-
|
||||
// captured drain AggregateError as the primary failure rather than
|
||||
// masking it. Only surface the close error if the drain succeeded.
|
||||
/* v8 ignore start -- close failure racing disposal is a defensive teardown edge */
|
||||
if (disposeError === undefined) throw closeError
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
}, `${this.backend.name} write path`)
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
if (existing) return existing
|
||||
// Snapshot the seed SYNCHRONOUSLY — initFor runs inside the `session/created`
|
||||
// emit, before any later `append` adds non-seed events. A clone freezes it
|
||||
// against later mutation of the live event objects.
|
||||
const seed = session.events.map(e => structuredClone(e))
|
||||
const p = this.onCreated(session, seed)
|
||||
// Attach a no-op rejection handler so a failing init does not surface as an
|
||||
// unhandled rejection if no flush observes `p` before it rejects. The REAL
|
||||
// error is still delivered: flush/dispose await the same `p` from the map.
|
||||
p.catch(() => { /* observed by flush/dispose via the stored promise */ })
|
||||
this.inits.set(session, p)
|
||||
return p
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's `seed` reproduces the first `cursor` persisted
|
||||
* events. A `cursor` of 0 (nothing persisted yet) trivially matches. Used when
|
||||
* a live session claims ownerless state left by a prior `load()`/`create()`.
|
||||
*/
|
||||
private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
if (cursor === 0) return true
|
||||
const stored = await this.backend.loadStored(id)
|
||||
/* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */
|
||||
if (stored === undefined) return false
|
||||
return seedCoversPrefix(seed, stored.events.slice(0, cursor))
|
||||
}
|
||||
|
||||
/**
|
||||
* On session/created: sync the backend's in-memory state to a live Session.
|
||||
*
|
||||
* Cases, by whether this backend tracks the id and whether an artifact exists:
|
||||
* 1. Already tracked → no-op (or claim ownerless state if the seed matches,
|
||||
* or reclaim a truly-abandoned id, else reject as a collision).
|
||||
* 2. Not tracked, an artifact EXISTS at this cwd and is a seq-aligned PREFIX
|
||||
* of the live events → ADOPT it (HMR/reload), persisting any live suffix.
|
||||
* 3. Not tracked, an artifact EXISTS but is NOT a prefix → REJECT (collision).
|
||||
* 4. Not tracked and NO artifact → a genuinely new session: register meta
|
||||
* (lazy) and persist its seed once.
|
||||
*/
|
||||
private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
const id = session.header.id
|
||||
const tracked = this.states.get(id)
|
||||
if (tracked !== undefined) {
|
||||
// case 1: already tracked.
|
||||
/* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */
|
||||
if (tracked.owner === session) return
|
||||
if (tracked.owner === undefined) {
|
||||
// Ownerless state from the public create()/load() API. The FIRST live
|
||||
// session claims it — but ONLY if its seed reproduces the persisted
|
||||
// prefix (else a fresh, unrelated session reusing the id would have its
|
||||
// seq 0..cursor-1 events filtered as already-written and grafted on).
|
||||
if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) {
|
||||
throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`)
|
||||
}
|
||||
tracked.owner = session
|
||||
// Persist the seed SUFFIX beyond the persisted prefix. Constructor seed
|
||||
// events never emit session/event, so the buffer never sees them.
|
||||
const suffix = seed.slice(tracked.cursor)
|
||||
if (suffix.length > 0) await this.append(id, suffix)
|
||||
return
|
||||
}
|
||||
// Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id
|
||||
// (never materialized, no pending buffer); else it is a real collision.
|
||||
const ownerBuffer = this.buffers.get(tracked.owner)
|
||||
if (!tracked.materialized && !ownerBuffer?.length) {
|
||||
this.states.delete(id)
|
||||
} else {
|
||||
throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`)
|
||||
}
|
||||
}
|
||||
|
||||
// case 2/3: an artifact at THIS cwd is adopted as a live prefix (or rejected
|
||||
// as a collision inside adoptLivePrefix). cwd-scoped (loadLive), never
|
||||
// any-scope: a same-id artifact at a different cwd is a collision, not a
|
||||
// resume.
|
||||
const live = await this.backend.loadLive(id, session.header.cwd)
|
||||
if (live !== undefined) {
|
||||
// Do NOT route through loadCore(): that crash-repairs open turns as
|
||||
// interrupted, which is wrong for HMR while the live Session is still the
|
||||
// authority and may append the real step/turn end later.
|
||||
await this.serialize(id, () => this.adoptLivePrefix(session, seed, live))
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session. Register its meta (lazy), then persist its
|
||||
// seed (events present at creation time) once.
|
||||
const meta: SessionHeader = { ...session.header }
|
||||
await this.create(meta)
|
||||
// Bind this state to the live session so a later DIFFERENT session reusing
|
||||
// the id is detected as a collision (case 1) rather than silently no-opped.
|
||||
const created = this.states.get(id)
|
||||
/* v8 ignore next -- create() always sets the state for the id */
|
||||
if (created !== undefined) created.owner = session
|
||||
if (seed.length > 0) await this.append(id, seed)
|
||||
}
|
||||
|
||||
/**
|
||||
* Adopt a stored prefix as a live session's history (HMR/reload): verify the
|
||||
* seed covers the stored prefix, truncate any torn tail (NOT the open turn —
|
||||
* the live Session is still the authority), bind ownership, and persist the
|
||||
* live suffix that was ahead of the stored prefix.
|
||||
*/
|
||||
private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix<TornMarker>): Promise<void> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
if (!seedCoversPrefix(seed, events)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
}
|
||||
// Truncate-only repair (no closers): the open turn is NOT closed here.
|
||||
if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, [])
|
||||
this.states.set(session.header.id, {
|
||||
meta: { ...meta },
|
||||
cursor: events.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
})
|
||||
const suffix = seed.slice(events.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
// Wait for the session's init (onCreated) so the state/cursor and any
|
||||
// fork-seed persistence are in place before draining. Awaiting the same
|
||||
// promise initFor stored also surfaces an init failure (e.g. a collision)
|
||||
// here, where the caller of session/flush observes it.
|
||||
await this.inits.get(session)
|
||||
// Serialize the WHOLE drain (read cursor → append → splice) on the per-session
|
||||
// chain so two concurrent flushes cannot both read the same cursor and
|
||||
// seq-mismatch on the second append.
|
||||
await this.serialize(session.header.id, () => this.drain(session))
|
||||
}
|
||||
|
||||
/** Drain a session's write buffer to the backend. Caller serializes this per id. */
|
||||
private async drain(session: Session): Promise<void> {
|
||||
const buffer = this.buffers.get(session)
|
||||
if (!buffer?.length) return
|
||||
// Copy WITHOUT removing: the buffer is the only durable-pending copy of these
|
||||
// events. Drain it only AFTER the append commits; events pushed during the
|
||||
// await sit past batch.length and survive the prefix splice, so a
|
||||
// retry/dispose re-drains the rest.
|
||||
const batch = buffer.slice()
|
||||
const state = this.states.get(session.header.id)
|
||||
// Only append events at or beyond the write cursor (a resumed session's seed
|
||||
// is already stored). flush awaits the init above, which always sets state,
|
||||
// so the `?? 0` fallback is a defensive guard that never fires in practice.
|
||||
/* v8 ignore next -- state is always set by the awaited init before flush */
|
||||
const cursor = state?.cursor ?? 0
|
||||
const fresh = batch.filter(e => e.seq >= cursor)
|
||||
// appendCore (NOT the serialized append) — drain already runs inside the
|
||||
// per-session chain, so re-entering via append() would deadlock.
|
||||
if (fresh.length > 0) await this.appendCore(session.header.id, fresh)
|
||||
buffer.splice(0, batch.length)
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,10 @@ import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-se
|
||||
// Re-export the metadata vocabulary so consumers import it from the seam.
|
||||
export type { SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
// The backend-agnostic write-path orchestration first-party backends compose.
|
||||
export { PersistenceCoordinator } from './coordinator.ts'
|
||||
export type { PersistenceBackend, StoredPrefix } from './coordinator.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
sessionPersistence: SessionPersistence
|
||||
|
||||
734
packages/session-persistence/tests/coordinator-contract.ts
Normal file
734
packages/session-persistence/tests/coordinator-contract.ts
Normal file
@@ -0,0 +1,734 @@
|
||||
/**
|
||||
* Reusable ORCHESTRATION suite for any backend that composes a
|
||||
* {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in
|
||||
* contract.ts) pins the public read/write SEMANTICS, this suite pins the
|
||||
* coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across
|
||||
* every first-party backend because it lives in the shared coordinator, not in
|
||||
* the storage primitives: the `session/created` → `session/event` →
|
||||
* `session/flush` → dispose drain, lazy materialization, fork-seed persistence,
|
||||
* the four `onCreated` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), crash-tail repair on load, and dispose-time quiescence.
|
||||
*
|
||||
* A backend imports {@link runCoordinatorContract} and calls it with a
|
||||
* {@link CoordinatorFixture} factory that knows how to (a) mount the REAL
|
||||
* backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload
|
||||
* tests can dispose one instance and mount another over the same bytes/rows),
|
||||
* and (b) inject a never-committed torn tail for one session
|
||||
* ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail
|
||||
* repair branch is exercised against real storage. The suite drives everything
|
||||
* through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore
|
||||
* write path — never the storage primitives directly — so it runs unchanged for
|
||||
* every backend (memory / jsonl / sqlite).
|
||||
*
|
||||
* Each scenario here was previously DUPLICATED in `jsonl.spec.ts` and
|
||||
* `sqlite.spec.ts`; it now lives once and runs once per backend through the
|
||||
* fixture. The per-backend specs keep ONLY their storage-mechanics tests.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
import { meta, oneTurnLog } from './contract.ts'
|
||||
|
||||
/**
|
||||
* The backend-specific capabilities the orchestration suite needs beyond the
|
||||
* public service API. A fresh fixture is created per test (isolated storage);
|
||||
* the suite mounts/disposes backend instances on it and cleans it up at the end.
|
||||
*/
|
||||
export interface CoordinatorFixture {
|
||||
/**
|
||||
* Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`,
|
||||
* over THIS fixture's shared storage scope. Returns the plugin fiber so the
|
||||
* suite can dispose a single instance (HMR/reload) while the storage — and any
|
||||
* still-live session in another fiber — survives. The caller has already
|
||||
* mounted `SessionStore` on `ctx`.
|
||||
*/
|
||||
mount: (ctx: Context) => Promise<Fiber>
|
||||
|
||||
/**
|
||||
* Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at
|
||||
* the given `cwd` (the cwd the session was created with): a half-written
|
||||
* record past the committed region (JSONL: a partial line with no newline;
|
||||
* SQLite: a row with invalid `data` JSON past the committed seq). This drives
|
||||
* the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair`
|
||||
* branch against real storage.
|
||||
*
|
||||
* OMITTED by a backend that structurally has no torn tails (memory): the
|
||||
* torn-tail scenario then self-skips (asserted explicitly in the suite).
|
||||
*/
|
||||
corruptTail?: (id: SessionId, cwd: string | undefined) => Promise<void>
|
||||
|
||||
/** Tear down the storage scope (remove the temp dir / file). */
|
||||
cleanup: () => Promise<void>
|
||||
}
|
||||
|
||||
/** A constant absolute cwd; jsonl keys directories off it, memory/sqlite ignore it. */
|
||||
const WORK = '/w'
|
||||
|
||||
/** The per-session init map a backend exposes for white-box init awaits. */
|
||||
function inits(persistence: SessionPersistence): Map<Session, Promise<void>> {
|
||||
return (persistence as unknown as { inits: Map<Session, Promise<void>> }).inits
|
||||
}
|
||||
|
||||
/** Append a whole event log to a live session, event by event (drives session/event). */
|
||||
function send(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
}
|
||||
|
||||
/** A live session created inside its OWN fiber, so it survives a backend reload. */
|
||||
async function liveSessionInFiber(
|
||||
ctx: Context, id: string, cwd: string | undefined,
|
||||
): Promise<Session> {
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(id, cwd !== undefined ? { meta: { cwd } } : undefined)
|
||||
}, { inject: ['sessions'] }))
|
||||
return session
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the coordinator orchestration suite against a backend. `makeFixture()`
|
||||
* MUST return a fresh fixture (isolated storage) each call.
|
||||
*/
|
||||
export function runCoordinatorContract(name: string, makeFixture: () => Promise<CoordinatorFixture>): void {
|
||||
describe(`PersistenceCoordinator orchestration: ${name}`, () => {
|
||||
/** Mount SessionStore + a backend instance on a fresh context over the fixture's storage. */
|
||||
async function freshCtx(fix: CoordinatorFixture): Promise<{ ctx: Context; fiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
return { ctx, fiber }
|
||||
}
|
||||
|
||||
// --- write path: live session → flush → reload ---
|
||||
|
||||
it('persists a live session driven through the store, surviving reload', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('live', { meta: { cwd: WORK } })
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('live'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
expect(loaded.meta.cwd).toBe(WORK)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('mutate', { meta: { cwd: WORK } })
|
||||
const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } })
|
||||
// Mutate the live event object AFTER it was buffered by session/event.
|
||||
;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('mutate'))
|
||||
const first = loaded.events[0]
|
||||
expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('append snapshots the batch: mutating the caller array/events after the call is ignored', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('snapshot', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const events = oneTurnLog() // seqs 0..5
|
||||
const userMsg = events[1] // the user/message event
|
||||
const p = ctx.sessionPersistence.append(m.id, events)
|
||||
// Mutate the caller's array AND an event object after the call but before
|
||||
// the queued op runs: the snapshot taken at call time must shield the copy.
|
||||
events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } })
|
||||
if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }]
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6
|
||||
const persisted = JSON.stringify(loaded.events)
|
||||
expect(persisted).toContain('hi') // original content
|
||||
expect(persisted).not.toContain('MUTATED')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- fork / resume ---
|
||||
|
||||
it('fork: a seeded new session persists its seed once (no double-write on a no-op flush)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const seed = oneTurnLog()
|
||||
// A fork: a brand-new id whose seed came from elsewhere.
|
||||
const forked = ctx.sessions.create('forked', { seed, meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(forked) // onCreated persisted the seed
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(loaded.events).toEqual(seed)
|
||||
// A flush with no NEW events must not double-write.
|
||||
await ctx.parallel('session/flush', forked)
|
||||
const reloaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(reloaded.events).toEqual(seed)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
// First lifecycle: persist a session through the store.
|
||||
const s1 = first.ctx.sessions.create('resumed', { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// Second lifecycle: a NEW backend instance + a session re-created with the
|
||||
// same id SEEDED with the loaded events. onCreated adopts the stored log
|
||||
// (does not re-persist the seed); a new turn appends at seq 6.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
const s2 = second.ctx.sessions.create('resumed', { seed: loaded.events, meta: { cwd: WORK } })
|
||||
await inits(second.ctx.sessionPersistence).get(s2) // let onCreated adopt
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await second.ctx.parallel('session/flush', s2)
|
||||
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
// 6 original + 2 new, contiguous, no duplicated seed.
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- HMR ---
|
||||
|
||||
it('HMR: applying the plugin seeds existing live sessions', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// A session exists BEFORE the persistence plugin is applied.
|
||||
const session = ctx.sessions.create('pre-existing', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
const fiber = await fix.mount(ctx)
|
||||
try {
|
||||
// The plugin seeded it on apply; a subsequent flush persists its events.
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: dispose drains remaining buffers', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
const session = await liveSessionInFiber(ctx, 'drain', WORK)
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// No explicit flush — dispose must drain.
|
||||
await fiber.dispose()
|
||||
|
||||
// A fresh backend instance reads what the disposed one drained.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('drain'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// The session lives in its OWN fiber so it survives the backend reload.
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-adopt', WORK)
|
||||
try {
|
||||
// Backend instance 1 materializes the session.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the SAME storage
|
||||
// while the session stays live. Instance 2 has an empty states map but the
|
||||
// log is materialized and is a prefix of the live events — it must ADOPT
|
||||
// (not reject). A second turn appended after reload then persists.
|
||||
await backend1.dispose()
|
||||
await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: adoption persists the live SUFFIX that was ahead of the stored prefix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-suffix', WORK)
|
||||
try {
|
||||
// Instance 1 flushes turn 1.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2: it is now ONLY in the live session's events; the new
|
||||
// backend never buffered it via session/event.
|
||||
await backend1.dispose()
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
|
||||
// Instance 2 adopts the stored prefix (turn 1) and MUST also persist the
|
||||
// live suffix (turn 2) carried in the session's events.
|
||||
await fix.mount(ctx)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR adoption does NOT crash-repair an active open turn as interrupted (truncate without closers)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-open', WORK)
|
||||
try {
|
||||
const first = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Crash-tail a torn fragment past the (open) committed turn, then reload.
|
||||
await first.dispose()
|
||||
if (fix.corruptTail) await fix.corruptTail(SessionId('hmr-open'), WORK)
|
||||
const second = await fix.mount(ctx)
|
||||
// The live session is still the authority: it appends the REAL step/turn
|
||||
// end. Adoption must truncate the torn tail but NOT synthesize closers.
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } })
|
||||
await second.dispose()
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- collision / id reuse ---
|
||||
|
||||
it('a NEW live session colliding on a persisted id is rejected, not silently adopted', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const s1 = first.ctx.sessions.create('collide', { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A FRESH backend + a NEW live session with the same id but NO explicit
|
||||
// resume. onCreated treats it as new; create() rejects because a log already
|
||||
// exists. The rejection surfaces via the init promise (flush awaits it).
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const s2 = second.ctx.sessions.create('collide', { meta: { cwd: WORK } })
|
||||
s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(inits(second.ctx.sessionPersistence).get(s2))
|
||||
.rejects.toThrow(/already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// A live session created then disposed BEFORE its first append: cursor 0,
|
||||
// never materialized. A new live session reusing the id must reclaim it.
|
||||
let firstSession!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
firstSession = inner.sessions.create('abandoned', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(firstSession) // register the lazy state
|
||||
await firstFiber.dispose() // disposed before any append → never materialized
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('abandoned', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('abandoned'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create('buffered', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(first)
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create('buffered', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create('idem', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
// Re-emit session/created for the SAME live session (idempotent initFor).
|
||||
ctx.emit('session/created', session)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('idem'))
|
||||
expect(loaded.events).toHaveLength(2) // not doubled
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- ownerless-state claim (public create()/load() then a live session arrives) ---
|
||||
|
||||
it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// create() registers ownerless state with cursor 0 (lazy, nothing persisted).
|
||||
await ctx.sessionPersistence.create(meta('lazy-claim', WORK))
|
||||
// A live session with that id arrives and claims it (cursor 0 matches
|
||||
// trivially), persisting its seed.
|
||||
const live = ctx.sessions.create('lazy-claim', { seed: oneTurnLog(), meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).resolves.toBeUndefined()
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize a log, then load() it WITHOUT a live session — ownerless
|
||||
// state, cursor at the persisted length.
|
||||
await ctx.sessionPersistence.create(meta('preview', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('preview'))
|
||||
|
||||
// A FRESH (empty-seed) live session reusing that id must be rejected: its
|
||||
// seq 0..cursor-1 events would otherwise be filtered as already-persisted.
|
||||
let fresh!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
fresh = inner.sessions.create('preview', { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(fresh))
|
||||
.rejects.toThrow(/do not match this live session|already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize and load (ownerless, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('claim', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('claim'), oneTurnLog())
|
||||
const { events } = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
|
||||
// A live session SEEDED with the loaded log PLUS a new turn claims the
|
||||
// ownerless state and persists only the suffix.
|
||||
const cont = ctx.sessions.create('claim', { seed: [
|
||||
...events,
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
], meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(cont)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- append adopts a storage-only session (fresh instance, no prior create/load) ---
|
||||
|
||||
it('append adopts a storage-only session (fresh instance) and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('adopt-append', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance appends a second turn WITHOUT a prior create/load: append
|
||||
// must adopt the stored session (cursor = stored length) and continue.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await second.ctx.sessionPersistence.append(SessionId('adopt-append'), [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('adopt-append'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- small public-API edges that the coordinator owns uniformly ---
|
||||
|
||||
it('append of an empty batch is a no-op (stays lazy)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('empty-batch', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, [])
|
||||
expect(await ctx.sessionPersistence.has(m.id)).toBe(false)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('load rejects a missing session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('delete of a non-existent session is a no-op', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
await expect(ctx.sessionPersistence.delete(SessionId('ghost'))).resolves.toBeUndefined()
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('create rejects a duplicate id (in memory and on a persisted log)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('dup', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
// Same in-memory state.
|
||||
await expect(first.ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists in this backend/)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance over the same storage sees the persisted log.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await expect(second.ctx.sessionPersistence.create(meta('dup', WORK)))
|
||||
.rejects.toThrow(/already has a persisted log on disk/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects an unknown format version on load (assertVersion)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: 2, id: SessionId('v2'), createdAt: 1, cwd: WORK }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/version/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('round-trips a header with parentSession (fork lineage)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: 1, id: SessionId('forked-child'), createdAt: 1, cwd: WORK, parentSession: SessionId('the-parent') }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta.parentSession).toBe('the-parent')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('flush before init resolves uses cursor 0', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Append directly to a live session and flush IMMEDIATELY, before the
|
||||
// async onCreated init has necessarily set state (exercises the
|
||||
// state-undefined cursor path).
|
||||
const session = ctx.sessions.create('flush-nostate', { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate'))
|
||||
expect(loaded.events).toHaveLength(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- crash-tail repair THROUGH the coordinator (real storage torn tail) ---
|
||||
|
||||
it('torn-tail load: a never-committed tail is truncated and the open turn closed during load (commitRepair w/ tornMarker)', async () => {
|
||||
const fix = await makeFixture()
|
||||
if (!fix.corruptTail) {
|
||||
// A memory-style store has no torn tails (every write is atomic in RAM),
|
||||
// so there is no tornMarker path to exercise. Assert that explicitly
|
||||
// instead of silently skipping, then bail.
|
||||
expect(fix.corruptTail).toBeUndefined()
|
||||
await fix.cleanup()
|
||||
return
|
||||
}
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('torn', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed 0..5 (balanced)
|
||||
// A second turn whose real events are durable but never closed (open turn).
|
||||
await first.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
])
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
// Inject a torn fragment past the committed region (never-committed tail).
|
||||
await fix.corruptTail(SessionId('torn'), WORK)
|
||||
|
||||
// A FRESH instance loads: the torn tail is truncated (tornMarker !==
|
||||
// undefined) AND the open turn 2 is closed with synthetic step/end +
|
||||
// turn/end {interrupted} — commitRepair runs with BOTH a torn marker and
|
||||
// closers. The preserved real events (0..7) are never truncated.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real + synthetic closers
|
||||
])
|
||||
const last = loaded.events.at(-1)!
|
||||
expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
|
||||
|
||||
// The repair is durable: the next append continues at the balanced length
|
||||
// (seq 10) and a reload round-trips identically.
|
||||
await second.ctx.sessionPersistence.append(SessionId('torn'), [
|
||||
{ type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -1,76 +1,132 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { SessionId, isJsonValue, interruptedTurnClosers } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { SessionPersistence, assertSerializable, seedCoversPrefix } from '../src/index.ts'
|
||||
import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
SessionPersistence, PersistenceCoordinator, assertSerializable, seedCoversPrefix,
|
||||
type PersistenceBackend, type StoredPrefix,
|
||||
} from '../src/index.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts'
|
||||
|
||||
/** The durable store shape: materialized sessions only (no lazy entries). */
|
||||
type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
/** Optional plugin config: an EXTERNAL store shared across backend instances. */
|
||||
interface MemoryConfig { store?: MemoryStore }
|
||||
|
||||
/**
|
||||
* A minimal in-memory {@link SessionPersistence} used to (a) cover the abstract
|
||||
* base's constructor + service registration and (b) validate the reusable
|
||||
* contract suite itself. The real durable backend is
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl`.
|
||||
* A trivial in-memory {@link SessionPersistence} that composes a
|
||||
* {@link PersistenceCoordinator} over a dependency-free `Map`-backed
|
||||
* {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle
|
||||
* (the simplest possible storage — a `Map<id, {meta, events}>` with no torn
|
||||
* tails, so `tornMarker` is always undefined) and the cover for the abstract
|
||||
* base's constructor + service registration. The real durable backends are
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`.
|
||||
*
|
||||
* The store can be supplied via config so two backend instances share one Map —
|
||||
* the in-RAM analogue of two backends over the same file/db, which the
|
||||
* coordinator orchestration suite's HMR/reload tests need (a fresh instance with
|
||||
* an empty in-memory states map adopting an already-materialized session).
|
||||
*/
|
||||
class MemoryPersistence extends SessionPersistence {
|
||||
private store = new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
private pending = new Map<string, SessionHeader>()
|
||||
class MemoryPersistence extends SessionPersistence implements PersistenceBackend<never> {
|
||||
static inject = ['sessions']
|
||||
|
||||
async create(m: SessionHeader): Promise<void> {
|
||||
// Lazy: record the intended meta, but stay absent from has/list until the
|
||||
// first append materializes the session.
|
||||
this.pending.set(m.id, m)
|
||||
override readonly name = 'session-persistence-memory'
|
||||
|
||||
/** The whole durable store: materialized sessions only (no lazy entries). */
|
||||
private store: MemoryStore
|
||||
private coordinator: PersistenceCoordinator<never>
|
||||
|
||||
constructor(ctx: Context, config?: MemoryConfig) {
|
||||
super(ctx)
|
||||
// Assign the store BEFORE constructing the coordinator: the coordinator's
|
||||
// constructor installs the write path and synchronously seeds existing live
|
||||
// sessions (onCreated → loadLive → this.store), so store must exist first.
|
||||
this.store = config?.store ?? new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
this.coordinator = new PersistenceCoordinator<never>(this.ctx, this)
|
||||
}
|
||||
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
const existing = this.store.get(id)
|
||||
const nextSeq = existing ? existing.events.length : 0
|
||||
if (events.length > 0 && events[0]!.seq !== nextSeq) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${nextSeq}, got ${events[0]!.seq}`)
|
||||
}
|
||||
for (let i = 0; i < events.length; i++) {
|
||||
const e = events[i]!
|
||||
if (e.seq !== nextSeq + i) throw new Error(`non-contiguous seq in batch for "${id}" at index ${i}`)
|
||||
if (!isJsonValue(e.data)) {
|
||||
throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
// --- service surface (delegated to the coordinator) ---
|
||||
|
||||
create(m: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(m)
|
||||
}
|
||||
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
has(id: SessionId): Promise<boolean> {
|
||||
return this.coordinator.has(id)
|
||||
}
|
||||
|
||||
delete(id: SessionId): Promise<void> {
|
||||
return this.coordinator.delete(id)
|
||||
}
|
||||
|
||||
/** White-box accessor: await a specific session's onCreated init. */
|
||||
get inits(): Map<Session, Promise<void>> {
|
||||
return this.coordinator.inits
|
||||
}
|
||||
|
||||
// --- PersistenceBackend hooks (the Map storage primitives) ---
|
||||
|
||||
// A Map-backed store has no torn tails, so `tornMarker` is never set. Ids are
|
||||
// globally unique, so loadStored and loadLive are identical (cwd is ignored).
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<never> | undefined> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) return undefined
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
}
|
||||
|
||||
loadLive(id: SessionId, _cwd: string | undefined): Promise<StoredPrefix<never> | undefined> {
|
||||
return this.loadStored(id)
|
||||
}
|
||||
|
||||
async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise<void> {
|
||||
// Defense-in-depth: the coordinator already validates serializability, but a
|
||||
// durable store must reject non-JSON data at its own boundary too.
|
||||
for (const e of events) {
|
||||
if (!isJsonValue(e.data)) throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
const existing = this.store.get(m.id)
|
||||
if (!existing) {
|
||||
const m = this.pending.get(id)
|
||||
if (!m) throw new Error(`append before create for "${id}"`)
|
||||
this.store.set(id, { meta: m, events: structuredClone(events) as SessionEvent[] })
|
||||
// First batch: `_isMaterialized` is false (the coordinator only omits
|
||||
// materialization on the first batch); writing the entry IS the materialization.
|
||||
this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] })
|
||||
} else {
|
||||
existing.events.push(...structuredClone(events) as SessionEvent[])
|
||||
}
|
||||
}
|
||||
|
||||
async load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) throw new Error(`session "${id}" not found`)
|
||||
// Honor the crash-recovery contract: if the stored log ends mid-turn, close
|
||||
// the orphaned turn durably with synthetic boundary events and continue from
|
||||
// the balanced length.
|
||||
const closers = interruptedTurnClosers(entry.events)
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers))
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
// No torn tails in a Map store, so `_tornMarker` is always undefined; only the
|
||||
// synthetic closers are appended (the same DELETE+INSERT a DB backend does,
|
||||
// minus the truncate).
|
||||
const entry = this.store.get(m.id)
|
||||
/* v8 ignore next -- commitRepair only runs for a materialized (stored) session */
|
||||
if (!entry) return
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[])
|
||||
}
|
||||
|
||||
async deleteStored(id: SessionId): Promise<void> {
|
||||
this.store.delete(id)
|
||||
}
|
||||
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
return [...this.store.values()].map(e => structuredClone(e.meta))
|
||||
}
|
||||
|
||||
async has(id: SessionId): Promise<boolean> {
|
||||
return this.store.has(id)
|
||||
}
|
||||
|
||||
async delete(id: SessionId): Promise<void> {
|
||||
this.store.delete(id)
|
||||
this.pending.delete(id)
|
||||
}
|
||||
}
|
||||
|
||||
// Run the shared contract against the in-memory backend.
|
||||
runPersistenceContract('memory', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
@@ -78,9 +134,24 @@ runPersistenceContract('memory', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the in-memory backend.
|
||||
// A per-fixture Map is the shared "storage", so two mounted instances see the
|
||||
// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store
|
||||
// writes atomically in RAM and has no torn tails, so the suite's torn-tail test
|
||||
// self-skips (and asserts the omission). The real torn-tail repair branch is
|
||||
// covered by the jsonl/sqlite fixtures, which CAN inject one.
|
||||
runCoordinatorContract('memory', async (): Promise<CoordinatorFixture> => {
|
||||
const store: MemoryStore = new Map()
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(MemoryPersistence, { store }),
|
||||
cleanup: async () => { store.clear() },
|
||||
}
|
||||
})
|
||||
|
||||
describe('SessionPersistence service registration', () => {
|
||||
it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence)
|
||||
|
||||
@@ -90,6 +161,7 @@ describe('SessionPersistence service registration', () => {
|
||||
|
||||
it('round-trips through the registered service instance', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const m = meta('reg')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
|
||||
Reference in New Issue
Block a user