Merge remote-tracking branch 'origin/master' into feat/adr0016-type-build-check
This commit is contained in:
imccyu
51
packages/session-persistence/session-persistence/README.md
Normal file
51
packages/session-persistence/session-persistence/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# @deepseek-ai/dsh-session-persistence
|
||||
|
||||
The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
|
||||
|
||||
The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here.
|
||||
|
||||
## Service API (`ctx.sessionPersistence`)
|
||||
|
||||
| Method | Contract |
|
||||
|---|---|
|
||||
| `create(meta): Promise<void>` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). |
|
||||
| `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
|
||||
| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. |
|
||||
| `list(): Promise<SessionHeader[]>` | Lightweight listing from metadata, no full-log parse. A zero-event lazily-materialized session is absent from `list`. |
|
||||
|
||||
## Invariants every backend must honor
|
||||
|
||||
- **Append-only; a crashed turn is closed, not truncated.** Committed events (at or below a flushed `turn/end`) are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}`) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded.
|
||||
- **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq.
|
||||
- **JSON-serializable data.** `append` rejects non-serializable `event.data`; backends snapshot each event when buffering (the live `session.events` object is mutable).
|
||||
- **Durability.** `append` returns only once the batch is durable.
|
||||
|
||||
## The write coordinator
|
||||
|
||||
The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows).
|
||||
|
||||
`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its four public service methods to the coordinator. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice).
|
||||
|
||||
The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinator and storage):
|
||||
|
||||
| Hook | Role |
|
||||
|---|---|
|
||||
| `name` | Backend label for the dispose-failure `AggregateError`. |
|
||||
| `loadStored(id)` | Read a stored prefix by id, scanning ANY storage scope. Used by resume/load and, via `!== undefined`, the create-collision probe. Returns an opaque `tornMarker` iff a torn tail must be truncated. |
|
||||
| `loadLive(id, cwd)` | Read a stored prefix SCOPED to `cwd` (HMR live-adoption must only adopt a log at the SAME cwd; a same-id log elsewhere is a collision, not a resume). A globally-unique-id backend ignores `cwd`. |
|
||||
| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. |
|
||||
| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined` — a marker may be falsy, e.g. seq/offset `0`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). |
|
||||
| `list()` | List all stored metadata. |
|
||||
| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. |
|
||||
|
||||
The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). The public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator. See [the write-coordinator RFC](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
## Testing backends
|
||||
|
||||
Import `runPersistenceContract` from `tests/contract.ts` (the public-API contract) and `runCoordinatorContract` from `tests/coordinator-contract.ts` (the shared write-path orchestration: adoption, HMR, collision, dispose-drain, crash-tail repair) and call each with a fixture for your backend. Every backend is held to the same append-only / contiguous-seq / lazy-materialization / serializability semantics AND the same orchestration, so a backend's own spec is left with only storage-mechanics tests (path sanitization, fsync rollback; schema version, transaction rollback) on top.
|
||||
|
||||
Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
|
||||
## Metadata types
|
||||
|
||||
Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`).
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-session-persistence",
|
||||
"description": "Abstract durable session persistence seam (ctx.sessionPersistence) for the DeepSeek Harness",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,549 @@
|
||||
/**
|
||||
* The backend-agnostic write-path orchestration shared by every first-party
|
||||
* {@link SessionPersistence} backend.
|
||||
*
|
||||
* Every durable backend needs the same orchestration: the in-memory bookkeeping
|
||||
* (the per-id state, the write-behind buffers, the per-id serialization chains,
|
||||
* the per-session init promises), the `session/event` → buffer → `session/flush`
|
||||
* drain, lazy materialization, crash-tail repair on load, the four
|
||||
* `session/created` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives are
|
||||
* backend-specific (file bytes for `dsh-session-persistence-jsonl`, `node:sqlite`
|
||||
* rows for `dsh-session-persistence-sqlite`). {@link PersistenceCoordinator} owns
|
||||
* the orchestration; a backend supplies the storage primitives as a small
|
||||
* {@link PersistenceBackend} hook object.
|
||||
*
|
||||
* The abstract {@link SessionPersistence} service's public API is independent of
|
||||
* this: a backend IS a `SessionPersistence` (its four public methods delegate to
|
||||
* a coordinator it composes), so a third-party backend MAY implement the service
|
||||
* directly without using the coordinator at all.
|
||||
*
|
||||
* See the write-coordinator RFC (docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md)
|
||||
* for the design rationale (composition over inheritance, the opaque torn marker).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/coordinator
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import { interruptedTurnClosers, SESSION_FORMAT_VERSION } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { assertSerializable, seedCoversPrefix } from './index.ts'
|
||||
|
||||
/**
|
||||
* A stored session's durable prefix as read back from a backend: its
|
||||
* {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix,
|
||||
* and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must
|
||||
* be truncated before further writes.
|
||||
*
|
||||
* The coordinator NEVER inspects `tornMarker`'s value — it only tests
|
||||
* `!== undefined` (is there a tail to repair?) and passes the value back to
|
||||
* {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker
|
||||
* type: the JSONL backend uses the byte offset to truncate to, the SQLite
|
||||
* backend uses the seq to delete from (both happen to be `number`).
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
tornMarker?: TornMarker
|
||||
}
|
||||
|
||||
/**
|
||||
* The storage seam between {@link PersistenceCoordinator} and a concrete
|
||||
* backend: the minimal set of durable primitives the orchestration calls. A
|
||||
* backend implements these (over files, rows, an object store, …); the
|
||||
* coordinator supplies everything else (buffering, serialization, cursors,
|
||||
* adoption, crash repair sequencing, dispose quiescence).
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token (see
|
||||
* {@link StoredPrefix}). The coordinator treats it as fully opaque.
|
||||
*/
|
||||
export interface PersistenceBackend<TornMarker = unknown> {
|
||||
/** Human-readable backend name, used in the dispose-failure AggregateError. */
|
||||
readonly name: string
|
||||
|
||||
/**
|
||||
* Read a stored prefix by id, scanning ANY storage scope (for JSONL: every
|
||||
* cwd bucket). Returns `undefined` if no stored artifact exists. Used by
|
||||
* resume/load, and — via `!== undefined` — by the create-collision probe.
|
||||
* The returned `tornMarker` is present iff there is a torn tail to truncate.
|
||||
*/
|
||||
loadStored(id: SessionId): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Read a stored prefix SCOPED to `cwd`. Deliberately distinct from
|
||||
* {@link loadStored}: HMR live-adoption must only adopt a persisted log at the
|
||||
* SAME cwd as the live session (a same-id log at a different cwd is a
|
||||
* collision, not a resume) — conflating the two reintroduces a cross-cwd
|
||||
* adoption bug. For a globally-unique-id backend (SQLite) `cwd` is ignored.
|
||||
*/
|
||||
loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Durably append a CONTIGUOUS batch, lazily materializing the session first
|
||||
* when `!isMaterialized`. The materialize-write and the first event batch MUST
|
||||
* commit ATOMICALLY (a crash between them must not leave a materialized-but-
|
||||
* empty session). Returns once the batch is durable.
|
||||
*/
|
||||
appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void>
|
||||
|
||||
/**
|
||||
* Make a crash repair durable: truncate the torn tail (iff
|
||||
* `tornMarker !== undefined`) and append `closers` (iff any). NOT required to
|
||||
* be atomic — a file backend may truncate-then-append in two fsync'd steps.
|
||||
* Used by load (truncate + synthetic closers) and by live-adoption (truncate
|
||||
* only, `closers = []`).
|
||||
*/
|
||||
commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/** List all stored (materialized) sessions' metadata. */
|
||||
list(): Promise<SessionHeader[]>
|
||||
|
||||
/**
|
||||
* Optional lifecycle teardown (e.g. close a database handle). Awaited by the
|
||||
* coordinator's dispose effect AFTER the quiescence drain. A stateless file
|
||||
* backend omits it.
|
||||
*/
|
||||
close?(): Promise<void>
|
||||
}
|
||||
|
||||
/** Per-session write state held by the coordinator's in-memory bookkeeping. */
|
||||
interface SessionState {
|
||||
meta: SessionHeader
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/**
|
||||
* Whether the backend has physically written this session (a JSONL file /
|
||||
* SQLite row exists). `create()` registers state LAZILY — cursor 0,
|
||||
* materialized false, nothing on disk — so an empty session leaves no
|
||||
* artifact and the FIRST `appendBatch` writes the header + its events in ONE
|
||||
* transaction (the "a row exists ⇔ it has events" invariant `list`
|
||||
* relies on; a separate up-front materialize could crash leaving a row with
|
||||
* zero events). The flag is the only signal that distinguishes a session
|
||||
* registered-but-never-written from one durably present, which the reclaim
|
||||
* path needs (an abandoned id with no artifact AND no buffered events is free
|
||||
* to reuse; a materialized one is a real collision).
|
||||
*/
|
||||
materialized: boolean
|
||||
/**
|
||||
* The live Session this state was bound to via `onCreated`, if any. State
|
||||
* created through the public `create()`/`load()` API has no owner; state bound
|
||||
* to a live session lets `onCreated` reject a second, unrelated session on the
|
||||
* same id (a collision) instead of silently no-opping.
|
||||
*/
|
||||
owner?: Session
|
||||
}
|
||||
|
||||
/** Collect the rejection reasons from a set of promises (none-throwing). */
|
||||
async function settledErrors(promises: Iterable<Promise<unknown>>): Promise<unknown[]> {
|
||||
const settled = await Promise.allSettled([...promises])
|
||||
const errors: unknown[] = []
|
||||
for (const result of settled) {
|
||||
if (result.status === 'rejected') errors.push(result.reason)
|
||||
}
|
||||
return errors
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the backend-agnostic session write-path orchestration. A backend
|
||||
* constructs one (`new PersistenceCoordinator(ctx, this)`), implements
|
||||
* {@link PersistenceBackend}, and delegates its four public service methods to
|
||||
* the matching coordinator methods.
|
||||
*
|
||||
* All per-id operations are serialized (a per-id promise chain) so concurrent
|
||||
* flushes / a flush racing a load never interleave storage writes. The
|
||||
* constructor installs the write-path listeners and the dispose effect.
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token.
|
||||
*/
|
||||
export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
/** Backend bookkeeping keyed by session id (NOT the live Session object). */
|
||||
private states = new Map<SessionId, SessionState>()
|
||||
/** Write-behind buffers keyed by the live Session (write path). */
|
||||
private buffers = new Map<Session, SessionEvent[]>()
|
||||
/**
|
||||
* Per-session serialization: every operation chains onto the prior one for the
|
||||
* same id, so writes for one session never interleave. Keyed by session id.
|
||||
*/
|
||||
private chains = new Map<SessionId, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise.
|
||||
*
|
||||
* Public (readonly) so a backend can expose it for white-box tests that await
|
||||
* a specific session's init (there is no public API to await one init); the
|
||||
* coordinator itself only ever mutates it internally.
|
||||
*/
|
||||
readonly inits = new Map<Session, Promise<void>>()
|
||||
|
||||
constructor(private ctx: Context, private backend: PersistenceBackend<TornMarker>) {
|
||||
this.installWritePath()
|
||||
}
|
||||
|
||||
// --- public surface (the backend's service methods delegate here) ---
|
||||
|
||||
/**
|
||||
* Register a new session's metadata (lazy: no physical write until the first
|
||||
* {@link append}). Rejects if the id is already tracked or already persisted.
|
||||
*/
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is stored as the lazy state, so keeping
|
||||
// the caller's object by reference would let a later mutation of `id`/`cwd`
|
||||
// register under one key but materialize under a different path/header.
|
||||
const snapshot: SessionHeader = { ...meta }
|
||||
return this.serialize(snapshot.id, () => this.createCore(snapshot))
|
||||
}
|
||||
|
||||
private async createCore(meta: SessionHeader): Promise<void> {
|
||||
// Do NOT clobber an existing session: the SessionId IS the identity.
|
||||
if (this.states.has(meta.id)) {
|
||||
throw new Error(`session "${meta.id}" already exists in this backend`)
|
||||
}
|
||||
// A persisted artifact under this id (in ANY scope) blocks creation: load/
|
||||
// resume identify a session by id alone, so a second artifact would make
|
||||
// resume nondeterministic.
|
||||
if (await this.backend.loadStored(meta.id) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`)
|
||||
}
|
||||
// Pure lazy: record intent only. No artifact until the first append.
|
||||
this.states.set(meta.id, { meta, cursor: 0, materialized: false })
|
||||
}
|
||||
|
||||
// `async` so the synchronous validate/clone below reject (not throw) per the
|
||||
// Promise<void> contract — callers use `await expect(...).rejects`.
|
||||
/**
|
||||
* Durably persist a batch of events. Honors the append-only and contiguous-seq
|
||||
* contracts; rejects non-JSON-serializable `event.data`.
|
||||
*/
|
||||
async append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Validate serializability BEFORE cloning so a bad event surfaces the typed
|
||||
// error rather than an opaque DataCloneError from structuredClone.
|
||||
assertSerializable(events)
|
||||
// Deep-snapshot the batch HERE, before the op waits behind the per-session
|
||||
// chain: a caller that mutates a live array (e.g. session.events) — or an
|
||||
// event inside it — before the op runs would otherwise have those changes
|
||||
// persisted. The clone is taken synchronously (at call time).
|
||||
const batch = events.map(e => structuredClone(e))
|
||||
return this.serialize(id, () => this.appendCore(id, batch))
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
if (events.length === 0) return
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id) // calls loadCore, not load
|
||||
|
||||
// Contiguity contract: each event's seq must continue the stored log.
|
||||
for (const [i, event] of events.entries()) {
|
||||
if (event.seq !== state.cursor + i) {
|
||||
throw new Error(`append seq mismatch for "${id}": expected ${state.cursor + i} at index ${i}, got ${event.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
await this.backend.appendBatch(state.meta, events, state.materialized)
|
||||
// The durable write is the transaction: mark materialized + advance the
|
||||
// cursor as soon as it commits (uniform across backends).
|
||||
state.materialized = true
|
||||
state.cursor += events.length
|
||||
}
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint, with any interrupted final turn durably closed (synthetic
|
||||
* boundary events) during load.
|
||||
*/
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.serialize(id, () => this.loadCore(id))
|
||||
}
|
||||
|
||||
private async loadCore(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const stored = await this.backend.loadStored(id)
|
||||
if (stored === undefined) throw new Error(`session "${id}" not found`)
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (real, preserved events but no
|
||||
// closing turn/end), close it durably DURING load so disk, the returned log,
|
||||
// and the cursor all agree. The interrupted turn's real events are preserved,
|
||||
// never truncated (a turn can be huge — the session-persistence RFC); only a
|
||||
// never-fully-written torn tail fragment is discarded.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Make the repair durable (truncate the torn tail + append the synthetic
|
||||
// closers) BEFORE recording state — commitRepair takes `meta` directly, so
|
||||
// there is no state-path ordering dependency (uniform across backends).
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
await this.backend.commitRepair(meta, tornMarker, closers)
|
||||
}
|
||||
// The state keeps its OWN copy of the meta; the returned value is separate so
|
||||
// a consumer mutating loaded.meta cannot corrupt the backend's metadata.
|
||||
this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true })
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
// NOTE: there is deliberately no coordinator `list()`. Listing needs none of
|
||||
// the coordinator's orchestration (no per-id serialization, no cursor, no
|
||||
// in-memory state) — it is a pure read of stored metadata. A backend's public
|
||||
// `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it
|
||||
// through the coordinator would only forward to that same hook, so the
|
||||
// coordinator stays out of the listing path entirely.
|
||||
|
||||
// --- per-id serialization + adoption helpers ---
|
||||
|
||||
/**
|
||||
* Run `op` after any in-flight operation for the same session id, so writes for
|
||||
* one session never interleave. Errors do not poison the chain. NOTE: serialized
|
||||
* public methods must NOT call each other (deadlock); they call the unserialized
|
||||
* `*Core` helpers instead.
|
||||
*/
|
||||
private serialize<T>(id: SessionId, op: () => Promise<T>): Promise<T> {
|
||||
const prior = this.chains.get(id) ?? Promise.resolve()
|
||||
const next = prior.then(op, op)
|
||||
// Keep the chain alive but swallow this op's rejection for the NEXT waiter
|
||||
// (the caller still sees the real rejection via `next`).
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
return next
|
||||
}
|
||||
|
||||
/** Build a state for a session discovered in storage but not yet in memory. */
|
||||
private async adopt(id: SessionId): Promise<SessionState> {
|
||||
// loadCore (NOT load) — adopt runs inside an already-serialized op, so
|
||||
// re-entering the chain via the public load() would deadlock.
|
||||
await this.loadCore(id)
|
||||
const state = this.states.get(id)
|
||||
/* v8 ignore next -- loadCore always sets the state for the id */
|
||||
if (!state) throw new Error(`failed to adopt session "${id}"`)
|
||||
return state
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version !== SESSION_FORMAT_VERSION) {
|
||||
throw new Error(`unsupported session format version ${meta.version} for "${meta.id}" (only v${SESSION_FORMAT_VERSION} is supported)`)
|
||||
}
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the init
|
||||
// promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Snapshot + buffer every event (the live object is mutable; clone so a later
|
||||
// in-place mutation cannot rewrite a buffered event). Serializability is
|
||||
// guaranteed at the source (Session.append), so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every init + final drain BEFORE
|
||||
// returning, then close the backend's own resources (AFTER the drain), so no
|
||||
// write lands after teardown and a close failure never MASKS a drain error.
|
||||
ctx.effect(() => async () => {
|
||||
let disposeError: unknown
|
||||
try {
|
||||
const errors = [
|
||||
...await settledErrors(this.inits.values()),
|
||||
...await settledErrors([...this.buffers.keys()].map(s => this.flush(s))),
|
||||
...await settledErrors(this.chains.values()),
|
||||
]
|
||||
if (errors.length > 0) {
|
||||
throw new AggregateError(errors, `${this.backend.name} dispose failed`)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
disposeError = error
|
||||
throw error
|
||||
} finally {
|
||||
try {
|
||||
await this.backend.close?.()
|
||||
} catch (closeError: unknown) {
|
||||
// A close failure can only add teardown context; keep the already-
|
||||
// captured drain AggregateError as the primary failure rather than
|
||||
// masking it. Only surface the close error if the drain succeeded.
|
||||
/* v8 ignore start -- close failure racing disposal is a defensive teardown edge */
|
||||
if (disposeError === undefined) throw closeError
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
}, `${this.backend.name} write path`)
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
if (existing) return existing
|
||||
// Snapshot the seed SYNCHRONOUSLY — initFor runs inside the `session/created`
|
||||
// emit, before any later `append` adds non-seed events. A clone freezes it
|
||||
// against later mutation of the live event objects.
|
||||
const seed = session.events.map(e => structuredClone(e))
|
||||
const p = this.onCreated(session, seed)
|
||||
// Attach a no-op rejection handler so a failing init does not surface as an
|
||||
// unhandled rejection if no flush observes `p` before it rejects. The REAL
|
||||
// error is still delivered: flush/dispose await the same `p` from the map.
|
||||
p.catch(() => { /* observed by flush/dispose via the stored promise */ })
|
||||
this.inits.set(session, p)
|
||||
return p
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's `seed` reproduces the first `cursor` persisted
|
||||
* events. A `cursor` of 0 (nothing persisted yet) trivially matches. Used when
|
||||
* a live session claims ownerless state left by a prior `load()`/`create()`.
|
||||
*/
|
||||
private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
if (cursor === 0) return true
|
||||
const stored = await this.backend.loadStored(id)
|
||||
/* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */
|
||||
if (stored === undefined) return false
|
||||
return seedCoversPrefix(seed, stored.events.slice(0, cursor))
|
||||
}
|
||||
|
||||
/**
|
||||
* On session/created: sync the backend's in-memory state to a live Session.
|
||||
*
|
||||
* Cases, by whether this backend tracks the id and whether an artifact exists:
|
||||
* 1. Already tracked → no-op (or claim ownerless state if the seed matches,
|
||||
* or reclaim a truly-abandoned id, else reject as a collision).
|
||||
* 2. Not tracked, an artifact EXISTS at this cwd and is a seq-aligned PREFIX
|
||||
* of the live events → ADOPT it (HMR/reload), persisting any live suffix.
|
||||
* 3. Not tracked, an artifact EXISTS but is NOT a prefix → REJECT (collision).
|
||||
* 4. Not tracked and NO artifact → a genuinely new session: register meta
|
||||
* (lazy) and persist its seed once.
|
||||
*/
|
||||
private async onCreated(session: Session, seed: readonly SessionEvent[]): Promise<void> {
|
||||
const id = session.header.id
|
||||
const tracked = this.states.get(id)
|
||||
if (tracked !== undefined) {
|
||||
// case 1: already tracked.
|
||||
/* v8 ignore next -- initFor dedupes per session object; same-object re-entry can't occur */
|
||||
if (tracked.owner === session) return
|
||||
if (tracked.owner === undefined) {
|
||||
// Ownerless state from the public create()/load() API. The FIRST live
|
||||
// session claims it — but ONLY if BOTH the cwd scope and the seed match.
|
||||
// The cwd guard mirrors case-2's cwd-scoped loadLive(): a same-id
|
||||
// ownerless artifact at a DIFFERENT cwd is a collision, not a claim
|
||||
// (claiming it would append the live cwd's events under the stored
|
||||
// header's cwd, the exact cross-cwd corruption the loadLive scope
|
||||
// prevents). The seed guard then ensures the live events reproduce the
|
||||
// persisted prefix (else a fresh, unrelated session reusing the id would
|
||||
// have its seq 0..cursor-1 events filtered as already-written and
|
||||
// grafted on).
|
||||
if (tracked.meta.cwd !== session.header.cwd) {
|
||||
throw new Error(`session "${id}" is already persisted at a different cwd (persisted: ${String(tracked.meta.cwd)}, live: ${String(session.header.cwd)}) (id collision)`)
|
||||
}
|
||||
if (!await this.seedMatchesPersisted(id, seed, tracked.cursor)) {
|
||||
throw new Error(`session "${id}" is already persisted with ${tracked.cursor} event(s) that do not match this live session (id collision)`)
|
||||
}
|
||||
tracked.owner = session
|
||||
// Persist the seed SUFFIX beyond the persisted prefix. Constructor seed
|
||||
// events never emit session/event, so the buffer never sees them.
|
||||
const suffix = seed.slice(tracked.cursor)
|
||||
if (suffix.length > 0) await this.append(id, suffix)
|
||||
return
|
||||
}
|
||||
// Owned by a DIFFERENT live session. Reclaim ONLY a truly-abandoned id
|
||||
// (never materialized, no pending buffer); else it is a real collision.
|
||||
const ownerBuffer = this.buffers.get(tracked.owner)
|
||||
if (!tracked.materialized && !ownerBuffer?.length) {
|
||||
this.states.delete(id)
|
||||
} else {
|
||||
throw new Error(`session "${id}" is already bound to a different live session in this backend (id collision)`)
|
||||
}
|
||||
}
|
||||
|
||||
// case 2/3: an artifact at THIS cwd is adopted as a live prefix (or rejected
|
||||
// as a collision inside adoptLivePrefix). cwd-scoped (loadLive), never
|
||||
// any-scope: a same-id artifact at a different cwd is a collision, not a
|
||||
// resume.
|
||||
const live = await this.backend.loadLive(id, session.header.cwd)
|
||||
if (live !== undefined) {
|
||||
// Do NOT route through loadCore(): that crash-repairs open turns as
|
||||
// interrupted, which is wrong for HMR while the live Session is still the
|
||||
// authority and may append the real step/turn end later.
|
||||
await this.serialize(id, () => this.adoptLivePrefix(session, seed, live))
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session. Register its meta (lazy), then persist its
|
||||
// seed (events present at creation time) once.
|
||||
const meta: SessionHeader = { ...session.header }
|
||||
await this.create(meta)
|
||||
// Bind this state to the live session so a later DIFFERENT session reusing
|
||||
// the id is detected as a collision (case 1) rather than silently no-opped.
|
||||
const created = this.states.get(id)
|
||||
/* v8 ignore next -- create() always sets the state for the id */
|
||||
if (created !== undefined) created.owner = session
|
||||
if (seed.length > 0) await this.append(id, seed)
|
||||
}
|
||||
|
||||
/**
|
||||
* Adopt a stored prefix as a live session's history (HMR/reload): verify the
|
||||
* seed covers the stored prefix, truncate any torn tail (NOT the open turn —
|
||||
* the live Session is still the authority), bind ownership, and persist the
|
||||
* live suffix that was ahead of the stored prefix.
|
||||
*/
|
||||
private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix<TornMarker>): Promise<void> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
if (!seedCoversPrefix(seed, events)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
}
|
||||
// Truncate-only repair (no closers): the open turn is NOT closed here.
|
||||
if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, [])
|
||||
this.states.set(session.header.id, {
|
||||
meta: { ...meta },
|
||||
cursor: events.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
})
|
||||
const suffix = seed.slice(events.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
// Wait for the session's init (onCreated) so the state/cursor and any
|
||||
// fork-seed persistence are in place before draining. Awaiting the same
|
||||
// promise initFor stored also surfaces an init failure (e.g. a collision)
|
||||
// here, where the caller of session/flush observes it.
|
||||
await this.inits.get(session)
|
||||
// Serialize the WHOLE drain (read cursor → append → splice) on the per-session
|
||||
// chain so two concurrent flushes cannot both read the same cursor and
|
||||
// seq-mismatch on the second append.
|
||||
await this.serialize(session.header.id, () => this.drain(session))
|
||||
}
|
||||
|
||||
/** Drain a session's write buffer to the backend. Caller serializes this per id. */
|
||||
private async drain(session: Session): Promise<void> {
|
||||
const buffer = this.buffers.get(session)
|
||||
if (!buffer?.length) return
|
||||
// Copy WITHOUT removing: the buffer is the only durable-pending copy of these
|
||||
// events. Drain it only AFTER the append commits; events pushed during the
|
||||
// await sit past batch.length and survive the prefix splice, so a
|
||||
// retry/dispose re-drains the rest.
|
||||
const batch = buffer.slice()
|
||||
const state = this.states.get(session.header.id)
|
||||
// Only append events at or beyond the write cursor (a resumed session's seed
|
||||
// is already stored). flush awaits the init above, which always sets state,
|
||||
// so the `?? 0` fallback is a defensive guard that never fires in practice.
|
||||
/* v8 ignore next -- state is always set by the awaited init before flush */
|
||||
const cursor = state?.cursor ?? 0
|
||||
const fresh = batch.filter(e => e.seq >= cursor)
|
||||
// appendCore (NOT the serialized append) — drain already runs inside the
|
||||
// per-session chain, so re-entering via append() would deadlock.
|
||||
if (fresh.length > 0) await this.appendCore(session.header.id, fresh)
|
||||
buffer.splice(0, batch.length)
|
||||
}
|
||||
}
|
||||
148
packages/session-persistence/session-persistence/src/index.ts
Normal file
148
packages/session-persistence/session-persistence/src/index.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
/**
|
||||
* The durable session-persistence seam (`ctx.sessionPersistence`): an abstract
|
||||
* service defining WHAT a persistence backend does — durably store, reload,
|
||||
* and list sessions — without saying HOW. Implementations subclass
|
||||
* {@link SessionPersistence} and register themselves as the
|
||||
* `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl`
|
||||
* (an append-only JSONL log per session) is the first and
|
||||
* `@deepseek-ai/dsh-session-persistence-sqlite` (`node:sqlite`, one row per
|
||||
* event) is a second that validates the seam is backend-agnostic by passing
|
||||
* the same `runPersistenceContract` suite. Further backends swap in an object
|
||||
* store or a remote service without touching the consumers (the write-path
|
||||
* plugin, the agent-loop resume seam).
|
||||
*
|
||||
* The persisted unit IS the existing {@link SessionEvent} — there is no
|
||||
* parallel "persisted message" type the log must be converted to and from
|
||||
* (faithful to the event-sourced model: the log is the single source of
|
||||
* truth). Metadata that is NOT replayable conversation state (format version,
|
||||
* cwd, lineage) travels separately as {@link SessionHeader}, which is owned by
|
||||
* `dsh-session` and re-exported here.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import { isJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
// Re-export the metadata vocabulary so consumers import it from the seam.
|
||||
export type { SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
|
||||
// The backend-agnostic write-path orchestration first-party backends compose.
|
||||
export { PersistenceCoordinator } from './coordinator.ts'
|
||||
export type { PersistenceBackend, StoredPrefix } from './coordinator.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
sessionPersistence: SessionPersistence
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's seed reproduces a persisted prefix exactly. Backends
|
||||
* use this collision check to distinguish a legitimate resume/HMR rebind from a
|
||||
* different live session reusing an existing session id.
|
||||
*
|
||||
* The comparison includes the full event payload, not just seq/type/time, so a
|
||||
* mutated seed cannot be grafted onto a durable log with the same envelope.
|
||||
*/
|
||||
export function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly SessionEvent[]): boolean {
|
||||
return prefix.length <= seed.length
|
||||
&& prefix.every((event, index) => {
|
||||
const seedEvent = seed[index]
|
||||
return seedEvent !== undefined && JSON.stringify(seedEvent) === JSON.stringify(event)
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject non-JSON-serializable event data before a backend serializes a batch.
|
||||
* Live session appends already enforce this; persistence append paths also
|
||||
* accept replay/fork batches that may bypass a live session instance.
|
||||
*/
|
||||
export function assertSerializable(events: readonly SessionEvent[]): void {
|
||||
for (const event of events) {
|
||||
if (!isJsonValue(event.data)) {
|
||||
throw new Error(`event "${event.type}" carries non-JSON-serializable data (seq ${event.seq})`)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract durable session-persistence service. Subclass, implement the
|
||||
* abstract methods, and load the subclass as a plugin — it registers as
|
||||
* `ctx.sessionPersistence` (one implementation per context; loading a second
|
||||
* throws, cordis' standard duplicate-service behavior).
|
||||
*
|
||||
* Contracts every implementation MUST honor (a DB backend asserts them inside
|
||||
* a transaction; a file backend appends at EOF):
|
||||
*
|
||||
* - **Append-only; a crashed turn is closed, not truncated.** Committed events
|
||||
* — those at or below a flushed `turn/end` — are never rewritten. A crash can
|
||||
* leave an unclosed final turn whose events are real (and possibly large);
|
||||
* {@link load} preserves them and closes the orphaned turn with synthetic
|
||||
* boundary events (see {@link load}). Only a never-fully-written torn tail
|
||||
* fragment is discarded.
|
||||
* - **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`.
|
||||
* {@link load} rejects a parse error or a `seq` gap in the COMMITTED region
|
||||
* (unloadable); {@link append}'s first event `seq` MUST equal the backend's
|
||||
* stored next-seq (after `load` has balanced any interrupted turn).
|
||||
* - **JSON-serializable data.** `SessionEventMap` is merge-extensible and
|
||||
* `event.data` is typed only as `SessionEventMap[K]`, so {@link append}
|
||||
* REJECTS non-JSON-serializable data with an error naming the offending
|
||||
* event type. A backend snapshots (serializes/clones) each event when it
|
||||
* buffers, since `session.events` hands out the live mutable object.
|
||||
* - **Durability.** {@link append} returns only once the batch is durable
|
||||
* (the file backend fsyncs; a DB commits). {@link create} MAY defer the
|
||||
* physical write until the first {@link append} (lazy materialization).
|
||||
*/
|
||||
export abstract class SessionPersistence extends Service {
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'sessionPersistence')
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a new session's metadata. A backend MAY defer the physical write
|
||||
* until the first {@link append} (lazy materialization), in which case a
|
||||
* created-but-never-appended session is absent from {@link list}
|
||||
* — abandoned sessions leave nothing behind.
|
||||
*/
|
||||
abstract create(meta: SessionHeader): Promise<void>
|
||||
|
||||
/**
|
||||
* Durably persist a batch of events (called from the write-behind drain at
|
||||
* the `session/flush` checkpoint). Honors the append-only and contiguous-seq
|
||||
* contracts: the first event's `seq` MUST equal the stored next-seq (after
|
||||
* `load` has durably closed any interrupted turn). Rejects non-JSON-
|
||||
* serializable `event.data` with an error naming the offending event type.
|
||||
*/
|
||||
abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint. Returns `meta` AND `events` so the live session is
|
||||
* reconstructed with its `cwd`/lineage, not just its log.
|
||||
*
|
||||
* The loop only flushes at `turn/end`, so a crash can leave a durable log
|
||||
* whose final turn never closed: real, fully-written events sit after the last
|
||||
* `turn/end`. Those events are PRESERVED — a single turn can be huge in a
|
||||
* long-horizon task, so truncating it would destroy real work — and `load`
|
||||
* CLOSES the orphaned turn by durably appending the minimal synthetic boundary
|
||||
* events: an error `tool/result` for every `tool-call` the crash left
|
||||
* unanswered (so the rehydrated history is a valid provider transcript — a
|
||||
* dangling assistant tool-call is otherwise rejected), then a `step/end` if a
|
||||
* step was open, then a `turn/end` carrying the `{ kind: 'interrupted' }`
|
||||
* reason. The returned `events` therefore end on a balanced `turn/end` and are
|
||||
* immediately usable as a session seed. Only a never-fully-written TORN tail
|
||||
* fragment (a half-written final record) is discarded. Returned events are
|
||||
* contiguous (`events[i].seq === i`); a parse error or a `seq` gap in the
|
||||
* COMMITTED region (at or before the last real `turn/end`) makes the session
|
||||
* unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for
|
||||
* the crash-recovery contract.
|
||||
*/
|
||||
abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
/** Lightweight listing from metadata, without a full-log parse. */
|
||||
abstract list(): Promise<SessionHeader[]>
|
||||
}
|
||||
|
||||
export default SessionPersistence
|
||||
@@ -0,0 +1,229 @@
|
||||
/**
|
||||
* Reusable contract test for any {@link SessionPersistence} backend. A backend
|
||||
* package imports {@link runPersistenceContract} and calls it with a factory
|
||||
* that yields a fresh, empty backend (and a teardown), so every backend is held
|
||||
* to the same append-only / contiguous-seq / lazy-materialization / crash
|
||||
* semantics. The JSONL backend's own spec adds file-specific tests on top.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/contract
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { SessionPersistence } from '../src/index'
|
||||
|
||||
/** A backend under test plus its teardown. */
|
||||
export interface ContractBackend {
|
||||
persistence: SessionPersistence
|
||||
dispose: () => Promise<void>
|
||||
}
|
||||
|
||||
/** Build a minimal {@link SessionHeader} for a session id. */
|
||||
export function meta(id: string, cwd?: string): SessionHeader {
|
||||
return {
|
||||
version: SESSION_FORMAT_VERSION,
|
||||
id: SessionId(id),
|
||||
createdAt: 1000,
|
||||
...cwd !== undefined ? { cwd } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/** A well-formed one-turn event log (contiguous seqs from 0). */
|
||||
export function oneTurnLog(): SessionEvent[] {
|
||||
return [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } } },
|
||||
{ type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } },
|
||||
{ type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the backend-agnostic contract suite. `make()` MUST return a fresh, empty
|
||||
* backend each call.
|
||||
*/
|
||||
export function runPersistenceContract(name: string, make: () => Promise<ContractBackend>): void {
|
||||
describe(`SessionPersistence contract: ${name}`, () => {
|
||||
it('round-trips a session: create + append → load returns identical meta and byte-identical events', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('s1', '/work')
|
||||
const log = oneTurnLog()
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, log)
|
||||
|
||||
const loaded = await persistence.load(m.id)
|
||||
expect(loaded.meta).toMatchObject({ version: SESSION_FORMAT_VERSION, id: m.id, cwd: '/work' })
|
||||
expect(loaded.events).toEqual(log)
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('crash recovery: load preserves an interrupted (unclosed) turn and closes it with turn/end {interrupted}', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('interrupted')
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, oneTurnLog()) // turn 1, committed (seqs 0..5)
|
||||
// A second turn that crashed mid-flight: turn/start + step/start were
|
||||
// durably written, but no step/end / turn/end ever arrived.
|
||||
await persistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
])
|
||||
|
||||
// load PRESERVES the interrupted turn's events (a turn can be huge — they
|
||||
// must not be truncated) and closes the orphaned turn with synthetic
|
||||
// boundary events: step/end (the step was open) then turn/end {interrupted}.
|
||||
const loaded = await persistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers
|
||||
])
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
const last = loaded.events.at(-1)!
|
||||
expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
|
||||
|
||||
// The closed log is durable and continuable: a fresh append continues at
|
||||
// the balanced length (seq 10), and a reload round-trips identically.
|
||||
await persistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await persistence.load(m.id)
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('crash recovery: an interrupted tool call gets a synthetic error result so resume is a valid transcript', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('interrupted-toolcall')
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, oneTurnLog()) // turn 1, committed (seqs 0..5)
|
||||
// Turn 2 crashed AFTER the assistant message asked for a tool call but
|
||||
// BEFORE the tool/result was written (the loop runs tools after logging
|
||||
// the assistant message — a process killed mid-tool lands exactly here).
|
||||
await persistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 8, time: 9, data: { turn: 2, step: 1, content: [
|
||||
{ type: 'tool-call', id: CallId('call-x'), name: 'bash', arguments: '{}' },
|
||||
] } },
|
||||
])
|
||||
|
||||
const loaded = await persistence.load(m.id)
|
||||
// The orphaned call is answered by a synthetic error tool/result BEFORE
|
||||
// step/end + turn/end {interrupted}, so the step (and turn) are balanced
|
||||
// and a resumed session derives a valid transcript (no dangling call).
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'assistant/message', 'tool/result', 'step/end', 'turn/end', // turn 2
|
||||
])
|
||||
const synthetic = loaded.events.find(e => e.type === 'tool/result')
|
||||
expect(synthetic?.type === 'tool/result' && synthetic.data).toMatchObject({
|
||||
callId: CallId('call-x'), isError: true, error: { code: 'interrupted' },
|
||||
})
|
||||
// The synthetic result carries the SAME callId as the orphaned tool-call,
|
||||
// so deriveMessages() pairs them — no provider-invalid dangling call.
|
||||
const call = loaded.events.findLast(e => e.type === 'assistant/message')
|
||||
const callId = call?.type === 'assistant/message'
|
||||
&& call.data.content.find(b => b.type === 'tool-call')
|
||||
expect(callId && callId.type === 'tool-call' && callId.id).toBe(CallId('call-x'))
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('list() excludes a created-but-never-appended (zero-event) session', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
await persistence.create(meta('empty'))
|
||||
expect((await persistence.list()).map(m => m.id)).not.toContain(SessionId('empty'))
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('list() includes a session once it has events', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('s2')
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, oneTurnLog())
|
||||
expect((await persistence.list()).map(x => x.id)).toContain(m.id)
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('append rejects a batch whose first seq does not match the stored next-seq', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('s3')
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, oneTurnLog()) // seqs 0..5, next-seq = 6
|
||||
// A re-append of an already-stored seq must be rejected, not duplicated.
|
||||
const restated = oneTurnLog()
|
||||
await expect(persistence.append(m.id, restated)).rejects.toThrow()
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('append rejects a mid-batch seq gap', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('s4')
|
||||
await persistence.create(m)
|
||||
const gapped: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // gap: missing seq 1
|
||||
]
|
||||
await expect(persistence.append(m.id, gapped)).rejects.toThrow()
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('append rejects non-JSON-serializable event data, naming the event type', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
// Every value `isJsonValue` rejects must be rejected by the backend, not
|
||||
// just BigInt — otherwise a backend could pass this contract while still
|
||||
// accepting values that corrupt the durable round-trip. Each is a
|
||||
// plugin-added `extra` field on a single user/message (seq 0).
|
||||
const cyclic: Record<string, unknown> = { type: 'text', text: 'x' }
|
||||
cyclic['self'] = cyclic
|
||||
const badValues: unknown[] = [
|
||||
1n, // BigInt
|
||||
undefined, // dropped by JSON.stringify
|
||||
Infinity, // → null
|
||||
() => 0, // function
|
||||
Symbol('s'), // symbol
|
||||
new Map(), // exotic object
|
||||
cyclic, // circular ref
|
||||
]
|
||||
for (const [i, bad] of badValues.entries()) {
|
||||
// A fresh session per value isolates each rejection (a rejected append
|
||||
// must leave no state behind, but isolating keeps the assertion clean).
|
||||
const mi = meta(`s5-${i}`)
|
||||
await persistence.create(mi)
|
||||
const events = [
|
||||
{ type: 'user/message', seq: 0, time: 1, data: { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: bad } },
|
||||
] as unknown as SessionEvent[]
|
||||
await expect(persistence.append(mi.id, events)).rejects.toThrow(/user\/message/)
|
||||
}
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,775 @@
|
||||
/**
|
||||
* Reusable ORCHESTRATION suite for any backend that composes a
|
||||
* {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in
|
||||
* contract.ts) pins the public read/write SEMANTICS, this suite pins the
|
||||
* coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across
|
||||
* every first-party backend because it lives in the shared coordinator, not in
|
||||
* the storage primitives: the `session/created` → `session/event` →
|
||||
* `session/flush` → dispose drain, lazy materialization, fork-seed persistence,
|
||||
* the four `onCreated` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), crash-tail repair on load, and dispose-time quiescence.
|
||||
*
|
||||
* A backend imports {@link runCoordinatorContract} and calls it with a
|
||||
* {@link CoordinatorFixture} factory that knows how to (a) mount the REAL
|
||||
* backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload
|
||||
* tests can dispose one instance and mount another over the same bytes/rows),
|
||||
* and (b) inject a never-committed torn tail for one session
|
||||
* ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail
|
||||
* repair branch is exercised against real storage. The suite drives everything
|
||||
* through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore
|
||||
* write path — never the storage primitives directly — so it runs unchanged for
|
||||
* every backend (memory / jsonl / sqlite).
|
||||
*
|
||||
* Each scenario lives here once and runs once per backend through the fixture;
|
||||
* the per-backend specs keep ONLY their storage-mechanics tests.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
import { meta, oneTurnLog } from './contract.ts'
|
||||
|
||||
/**
|
||||
* The backend-specific capabilities the orchestration suite needs beyond the
|
||||
* public service API. A fresh fixture is created per test (isolated storage);
|
||||
* the suite mounts/disposes backend instances on it and cleans it up at the end.
|
||||
*/
|
||||
export interface CoordinatorFixture {
|
||||
/**
|
||||
* Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`,
|
||||
* over THIS fixture's shared storage scope. Returns the plugin fiber so the
|
||||
* suite can dispose a single instance (HMR/reload) while the storage — and any
|
||||
* still-live session in another fiber — survives. The caller has already
|
||||
* mounted `SessionStore` on `ctx`.
|
||||
*/
|
||||
mount: (ctx: Context) => Promise<Fiber>
|
||||
|
||||
/**
|
||||
* Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at
|
||||
* the given `cwd` (the cwd the session was created with): a half-written
|
||||
* record past the committed region (JSONL: a partial line with no newline;
|
||||
* SQLite: a row with invalid `data` JSON past the committed seq). This drives
|
||||
* the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair`
|
||||
* branch against real storage.
|
||||
*
|
||||
* OMITTED by a backend that structurally has no torn tails (memory): the
|
||||
* torn-tail scenario then self-skips (asserted explicitly in the suite).
|
||||
*/
|
||||
corruptTail?: (id: SessionId, cwd: string | undefined) => Promise<void>
|
||||
|
||||
/** Tear down the storage scope (remove the temp dir / file). */
|
||||
cleanup: () => Promise<void>
|
||||
}
|
||||
|
||||
/** A constant absolute cwd; jsonl keys directories off it, memory/sqlite ignore it. */
|
||||
const WORK = '/w'
|
||||
const OTHER = '/other'
|
||||
|
||||
/** The per-session init map a backend exposes for white-box init awaits. */
|
||||
function inits(persistence: SessionPersistence): Map<Session, Promise<void>> {
|
||||
return (persistence as unknown as { inits: Map<Session, Promise<void>> }).inits
|
||||
}
|
||||
|
||||
/** Append a whole event log to a live session, event by event (drives session/event). */
|
||||
function send(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
}
|
||||
|
||||
/** A live session created inside its OWN fiber, so it survives a backend reload. */
|
||||
async function liveSessionInFiber(
|
||||
ctx: Context, id: string, cwd: string | undefined,
|
||||
): Promise<Session> {
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId(id), cwd !== undefined ? { meta: { cwd } } : undefined)
|
||||
}, { inject: ['sessions'] }))
|
||||
return session
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the coordinator orchestration suite against a backend. `makeFixture()`
|
||||
* MUST return a fresh fixture (isolated storage) each call.
|
||||
*/
|
||||
export function runCoordinatorContract(name: string, makeFixture: () => Promise<CoordinatorFixture>): void {
|
||||
describe(`PersistenceCoordinator orchestration: ${name}`, () => {
|
||||
/** Mount SessionStore + a backend instance on a fresh context over the fixture's storage. */
|
||||
async function freshCtx(fix: CoordinatorFixture): Promise<{ ctx: Context; fiber: Fiber }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
return { ctx, fiber }
|
||||
}
|
||||
|
||||
// --- write path: live session → flush → reload ---
|
||||
|
||||
it('persists a live session driven through the store, surviving reload', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('live'), { meta: { cwd: WORK } })
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('live'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
expect(loaded.meta.cwd).toBe(WORK)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('snapshot-on-buffer: mutating an event after session/event does not corrupt the persisted copy', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('mutate'), { meta: { cwd: WORK } })
|
||||
const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } })
|
||||
// Mutate the live event object AFTER it was buffered by session/event.
|
||||
;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('mutate'))
|
||||
const first = loaded.events[0]
|
||||
expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('append snapshots the batch: mutating the caller array/events after the call is ignored', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('snapshot', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
const events = oneTurnLog() // seqs 0..5
|
||||
const userMsg = events[1] // the user/message event
|
||||
const p = ctx.sessionPersistence.append(m.id, events)
|
||||
// Mutate the caller's array AND an event object after the call but before
|
||||
// the queued op runs: the snapshot taken at call time must shield the copy.
|
||||
events.push({ type: 'turn/start', seq: 6, time: 99, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } })
|
||||
if (userMsg?.type === 'user/message') userMsg.data.content = [{ type: 'text', text: 'MUTATED' }]
|
||||
await p
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5]) // not 0..6
|
||||
const persisted = JSON.stringify(loaded.events)
|
||||
expect(persisted).toContain('hi') // original content
|
||||
expect(persisted).not.toContain('MUTATED')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- fork / resume ---
|
||||
|
||||
it('fork: a seeded new session persists its seed once (no double-write on a no-op flush)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const seed = oneTurnLog()
|
||||
// A fork: a brand-new id whose seed came from elsewhere.
|
||||
const forked = ctx.sessions.create(SessionId('forked'), { seed, meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(forked) // onCreated persisted the seed
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(loaded.events).toEqual(seed)
|
||||
// A flush with no NEW events must not double-write.
|
||||
await ctx.parallel('session/flush', forked)
|
||||
const reloaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(reloaded.events).toEqual(seed)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
// First lifecycle: persist a session through the store.
|
||||
const s1 = first.ctx.sessions.create(SessionId('resumed'), { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// Second lifecycle: a NEW backend instance + a session re-created with the
|
||||
// same id SEEDED with the loaded events. onCreated adopts the stored log
|
||||
// (does not re-persist the seed); a new turn appends at seq 6.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
const s2 = second.ctx.sessions.create(SessionId('resumed'), { seed: loaded.events, meta: { cwd: WORK } })
|
||||
await inits(second.ctx.sessionPersistence).get(s2) // let onCreated adopt
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await second.ctx.parallel('session/flush', s2)
|
||||
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
// 6 original + 2 new, contiguous, no duplicated seed.
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- HMR ---
|
||||
|
||||
it('HMR: applying the plugin seeds existing live sessions', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// A session exists BEFORE the persistence plugin is applied.
|
||||
const session = ctx.sessions.create(SessionId('pre-existing'), { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
const fiber = await fix.mount(ctx)
|
||||
try {
|
||||
// The plugin seeded it on apply; a subsequent flush persists its events.
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: dispose drains remaining buffers', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
const session = await liveSessionInFiber(ctx, 'drain', WORK)
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// No explicit flush — dispose must drain.
|
||||
await fiber.dispose()
|
||||
|
||||
// A fresh backend instance reads what the disposed one drained.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('drain'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: reloading the backend adopts a still-live, already-materialized session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// The session lives in its OWN fiber so it survives the backend reload.
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-adopt', WORK)
|
||||
try {
|
||||
// Backend instance 1 materializes the session.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the SAME storage
|
||||
// while the session stays live. Instance 2 has an empty states map but the
|
||||
// log is materialized and is a prefix of the live events — it must ADOPT
|
||||
// (not reject). A second turn appended after reload then persists.
|
||||
await backend1.dispose()
|
||||
await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR: adoption persists the live SUFFIX that was ahead of the stored prefix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-suffix', WORK)
|
||||
try {
|
||||
// Instance 1 flushes turn 1.
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2: it is now ONLY in the live session's events; the new
|
||||
// backend never buffered it via session/event.
|
||||
await backend1.dispose()
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
|
||||
// Instance 2 adopts the stored prefix (turn 1) and MUST also persist the
|
||||
// live suffix (turn 2) carried in the session's events.
|
||||
await fix.mount(ctx)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('HMR adoption does NOT crash-repair an active open turn as interrupted (truncate without closers)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const session = await liveSessionInFiber(ctx, 'hmr-open', WORK)
|
||||
try {
|
||||
const first = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Crash-tail a torn fragment past the (open) committed turn, then reload.
|
||||
await first.dispose()
|
||||
if (fix.corruptTail) await fix.corruptTail(SessionId('hmr-open'), WORK)
|
||||
const second = await fix.mount(ctx)
|
||||
// The live session is still the authority: it appends the REAL step/turn
|
||||
// end. Adoption must truncate the torn tail but NOT synthesize closers.
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
expect(loaded.events.at(-1)).toMatchObject({ type: 'turn/end', data: { reason: { kind: 'completed' } } })
|
||||
await second.dispose()
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- collision / id reuse ---
|
||||
|
||||
it('a NEW live session colliding on a persisted id is rejected, not silently adopted', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const s1 = first.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A FRESH backend + a NEW live session with the same id but NO explicit
|
||||
// resume. onCreated treats it as new; create() rejects because a log already
|
||||
// exists. The rejection surfaces via the init promise (flush awaits it).
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const s2 = second.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } })
|
||||
s2.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await expect(inits(second.ctx.sessionPersistence).get(s2))
|
||||
.rejects.toThrow(/already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('an abandoned lazy session (never materialized) releases its id for reuse', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// A live session created then disposed BEFORE its first append: cursor 0,
|
||||
// never materialized. A new live session reusing the id must reclaim it.
|
||||
let firstSession!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
firstSession = inner.sessions.create(SessionId('abandoned'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(firstSession) // register the lazy state
|
||||
await firstFiber.dispose() // disposed before any append → never materialized
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create(SessionId('abandoned'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('abandoned'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create(SessionId('buffered'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await inits(ctx.sessionPersistence).get(first)
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create(SessionId('buffered'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('initFor is idempotent: re-emitting session/created does not re-initialize', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('idem'), { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
// Re-emit session/created for the SAME live session (idempotent initFor).
|
||||
ctx.emit('session/created', session)
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('idem'))
|
||||
expect(loaded.events).toHaveLength(2) // not doubled
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- ownerless-state claim (public create()/load() then a live session arrives) ---
|
||||
|
||||
it('a live session claims cursor-0 ownerless state created via the public API and persists its seed', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// create() registers ownerless state with cursor 0 (lazy, nothing persisted).
|
||||
await ctx.sessionPersistence.create(meta('lazy-claim', WORK))
|
||||
// A live session with that id arrives and claims it (cursor 0 matches
|
||||
// trivially), persisting its seed.
|
||||
const live = ctx.sessions.create(SessionId('lazy-claim'), { seed: oneTurnLog(), meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).resolves.toBeUndefined()
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('lazy-claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a fresh session reusing a previously-loaded id is rejected (ownerless guard)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize a log, then load() it WITHOUT a live session — ownerless
|
||||
// state, cursor at the persisted length.
|
||||
await ctx.sessionPersistence.create(meta('preview', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('preview'), oneTurnLog())
|
||||
await ctx.sessionPersistence.load(SessionId('preview'))
|
||||
|
||||
// A FRESH (empty-seed) live session reusing that id must be rejected: its
|
||||
// seq 0..cursor-1 events would otherwise be filtered as already-persisted.
|
||||
let fresh!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
fresh = inner.sessions.create(SessionId('preview'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(inits(ctx.sessionPersistence).get(fresh))
|
||||
.rejects.toThrow(/do not match this live session|already has a persisted log|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a live session whose seed matches the loaded prefix claims ownerless state and persists the suffix', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize and load (ownerless, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('claim', WORK))
|
||||
await ctx.sessionPersistence.append(SessionId('claim'), oneTurnLog())
|
||||
const { events } = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
|
||||
// A live session SEEDED with the loaded log PLUS a new turn claims the
|
||||
// ownerless state and persists only the suffix.
|
||||
const cont = ctx.sessions.create(SessionId('claim'), { seed: [
|
||||
...events,
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
], meta: { cwd: WORK } })
|
||||
await inits(ctx.sessionPersistence).get(cont)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('claim'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a live session at a DIFFERENT cwd cannot claim cursor-0 ownerless state (cwd scope)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// create() registers ownerless state at cwd /a (cursor 0 — claims would
|
||||
// otherwise match trivially on the seed).
|
||||
await ctx.sessionPersistence.create(meta('wrong-cwd-claim', OTHER))
|
||||
// A live session reusing the id but at cwd WORK must NOT claim it — the
|
||||
// cwd scope is the fence (without it, WORK events would append under the
|
||||
// OTHER header). Rejected as a collision.
|
||||
const live = ctx.sessions.create(SessionId('wrong-cwd-claim'), { seed: oneTurnLog(), meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a live session at a DIFFERENT cwd cannot claim loaded-prefix ownerless state (cwd scope)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Materialize + load at cwd OTHER (ownerless, cursor = 6).
|
||||
await ctx.sessionPersistence.create(meta('wrong-cwd-load', OTHER))
|
||||
await ctx.sessionPersistence.append(SessionId('wrong-cwd-load'), oneTurnLog())
|
||||
const { events } = await ctx.sessionPersistence.load(SessionId('wrong-cwd-load'))
|
||||
// A live session whose SEED matches the loaded prefix but whose cwd is
|
||||
// WORK must still be rejected — the cwd guard runs before the seed check.
|
||||
const live = ctx.sessions.create(SessionId('wrong-cwd-load'), { seed: events, meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('a no-cwd ownerless state cannot be claimed by a live session WITH a cwd (cwd scope, undefined side)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Ownerless state created WITHOUT a cwd (the no-cwd bucket).
|
||||
await ctx.sessionPersistence.create(meta('no-cwd-state'))
|
||||
// A live session reusing the id but WITH cwd WORK is a cwd mismatch
|
||||
// (undefined vs WORK) and must be rejected.
|
||||
const live = ctx.sessions.create(SessionId('no-cwd-state'), { seed: oneTurnLog(), meta: { cwd: WORK } })
|
||||
await expect(inits(ctx.sessionPersistence).get(live)).rejects.toThrow(/different cwd|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- append adopts a storage-only session (fresh instance, no prior create/load) ---
|
||||
|
||||
it('append adopts a storage-only session (fresh instance) and continues the seq', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('adopt-append', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance appends a second turn WITHOUT a prior create/load: append
|
||||
// must adopt the stored session (cursor = stored length) and continue.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await second.ctx.sessionPersistence.append(SessionId('adopt-append'), [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('adopt-append'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- small public-API edges that the coordinator owns uniformly ---
|
||||
|
||||
it('append of an empty batch is a no-op (stays lazy)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('empty-batch', WORK)
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, [])
|
||||
expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('load rejects a missing session', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
await expect(ctx.sessionPersistence.load(SessionId('nope'))).rejects.toThrow(/not found/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('create rejects a duplicate id (in memory and on a persisted log)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('dup', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
// Same in-memory state.
|
||||
await expect(first.ctx.sessionPersistence.create(m)).rejects.toThrow(/already exists in this backend/)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A fresh instance over the same storage sees the persisted log.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
await expect(second.ctx.sessionPersistence.create(meta('dup', WORK)))
|
||||
.rejects.toThrow(/already has a persisted log on disk/)
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects an unknown format version on load (assertVersion)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: 99, id: SessionId('v99'), createdAt: 1, cwd: WORK }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/version/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('round-trips a header with parentSession (fork lineage)', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const m = { version: SESSION_FORMAT_VERSION, id: SessionId('forked-child'), createdAt: 1, cwd: WORK, parentSession: SessionId('the-parent') }
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta.parentSession).toBe('the-parent')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('flush before init resolves uses cursor 0', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
// Append directly to a live session and flush IMMEDIATELY, before the
|
||||
// async onCreated init has necessarily set state (exercises the
|
||||
// state-undefined cursor path).
|
||||
const session = ctx.sessions.create(SessionId('flush-nostate'), { meta: { cwd: WORK } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate'))
|
||||
expect(loaded.events).toHaveLength(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
// --- crash-tail repair THROUGH the coordinator (real storage torn tail) ---
|
||||
|
||||
it('torn-tail load: a never-committed tail is truncated and the open turn closed during load (commitRepair w/ tornMarker)', async () => {
|
||||
const fix = await makeFixture()
|
||||
if (!fix.corruptTail) {
|
||||
// A memory-style store has no torn tails (every write is atomic in RAM),
|
||||
// so there is no tornMarker path to exercise. Assert that explicitly
|
||||
// instead of silently skipping, then bail.
|
||||
expect(fix.corruptTail).toBeUndefined()
|
||||
await fix.cleanup()
|
||||
return
|
||||
}
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
const m = meta('torn', WORK)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed 0..5 (balanced)
|
||||
// A second turn whose real events are durable but never closed (open turn).
|
||||
await first.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
])
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
// Inject a torn fragment past the committed region (never-committed tail).
|
||||
await fix.corruptTail(SessionId('torn'), WORK)
|
||||
|
||||
// A FRESH instance loads: the torn tail is truncated (tornMarker !==
|
||||
// undefined) AND the open turn 2 is closed with synthetic step/end +
|
||||
// turn/end {interrupted} — commitRepair runs with BOTH a torn marker and
|
||||
// closers. The preserved real events (0..7) are never truncated.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real + synthetic closers
|
||||
])
|
||||
const last = loaded.events.at(-1)!
|
||||
expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
|
||||
|
||||
// The repair is durable: the next append continues at the balanced length
|
||||
// (seq 10) and a reload round-trips identically.
|
||||
await second.ctx.sessionPersistence.append(SessionId('torn'), [
|
||||
{ type: 'turn/start', seq: 10, time: 9, data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('torn'))
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
SessionPersistence, PersistenceCoordinator, assertSerializable, seedCoversPrefix,
|
||||
type PersistenceBackend, type StoredPrefix,
|
||||
} from '../src/index'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from './contract'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract'
|
||||
|
||||
/** The durable store shape: materialized sessions only (no lazy entries). */
|
||||
type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
/** Optional plugin config: an EXTERNAL store shared across backend instances. */
|
||||
interface MemoryConfig { store?: MemoryStore }
|
||||
|
||||
/**
|
||||
* A trivial in-memory {@link SessionPersistence} that composes a
|
||||
* {@link PersistenceCoordinator} over a dependency-free `Map`-backed
|
||||
* {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle
|
||||
* (the simplest possible storage — a `Map<id, {meta, events}>` with no torn
|
||||
* tails, so `tornMarker` is always undefined) and the cover for the abstract
|
||||
* base's constructor + service registration. The real durable backends are
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`.
|
||||
*
|
||||
* The store can be supplied via config so two backend instances share one Map —
|
||||
* the in-RAM analogue of two backends over the same file/db, which the
|
||||
* coordinator orchestration suite's HMR/reload tests need (a fresh instance with
|
||||
* an empty in-memory states map adopting an already-materialized session).
|
||||
*/
|
||||
class MemoryPersistence extends SessionPersistence implements PersistenceBackend<never> {
|
||||
static inject = ['sessions']
|
||||
|
||||
override readonly name = 'session-persistence-memory'
|
||||
|
||||
/** The whole durable store: materialized sessions only (no lazy entries). */
|
||||
private store: MemoryStore
|
||||
private coordinator: PersistenceCoordinator<never>
|
||||
|
||||
constructor(ctx: Context, config?: MemoryConfig) {
|
||||
super(ctx)
|
||||
// Assign the store BEFORE constructing the coordinator: the coordinator's
|
||||
// constructor installs the write path and synchronously seeds existing live
|
||||
// sessions (onCreated → loadLive → this.store), so store must exist first.
|
||||
this.store = config?.store ?? new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
this.coordinator = new PersistenceCoordinator<never>(this.ctx, this)
|
||||
}
|
||||
|
||||
// --- service surface (delegated to the coordinator) ---
|
||||
|
||||
create(m: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(m)
|
||||
}
|
||||
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
/** White-box accessor: await a specific session's onCreated init. */
|
||||
get inits(): Map<Session, Promise<void>> {
|
||||
return this.coordinator.inits
|
||||
}
|
||||
|
||||
// --- PersistenceBackend hooks (the Map storage primitives) ---
|
||||
|
||||
// A Map-backed store has no torn tails, so `tornMarker` is never set. Ids are
|
||||
// globally unique, so loadStored and loadLive are identical (cwd is ignored).
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<never> | undefined> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) return undefined
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
}
|
||||
|
||||
loadLive(id: SessionId, _cwd: string | undefined): Promise<StoredPrefix<never> | undefined> {
|
||||
return this.loadStored(id)
|
||||
}
|
||||
|
||||
async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise<void> {
|
||||
// Defense-in-depth: the coordinator already validates serializability, but a
|
||||
// durable store must reject non-JSON data at its own boundary too.
|
||||
for (const e of events) {
|
||||
if (!isJsonValue(e.data)) throw new Error(`event "${e.type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
const existing = this.store.get(m.id)
|
||||
if (!existing) {
|
||||
// First batch: `_isMaterialized` is false (the coordinator only omits
|
||||
// materialization on the first batch); writing the entry IS the materialization.
|
||||
this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] })
|
||||
} else {
|
||||
existing.events.push(...structuredClone(events) as SessionEvent[])
|
||||
}
|
||||
}
|
||||
|
||||
async commitRepair(m: SessionHeader, _tornMarker: undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
// No torn tails in a Map store, so `_tornMarker` is always undefined; only the
|
||||
// synthetic closers are appended (the same DELETE+INSERT a DB backend does,
|
||||
// minus the truncate).
|
||||
const entry = this.store.get(m.id)
|
||||
/* v8 ignore next -- commitRepair only runs for a materialized (stored) session */
|
||||
if (!entry) return
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[])
|
||||
}
|
||||
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
return [...this.store.values()].map(e => structuredClone(e.meta))
|
||||
}
|
||||
}
|
||||
|
||||
// Run the shared contract against the in-memory backend.
|
||||
runPersistenceContract('memory', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
dispose: async () => { await fiber.dispose() },
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the in-memory backend.
|
||||
// A per-fixture Map is the shared "storage", so two mounted instances see the
|
||||
// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store
|
||||
// writes atomically in RAM and has no torn tails, so the suite's torn-tail test
|
||||
// self-skips (and asserts the omission). The real torn-tail repair branch is
|
||||
// covered by the jsonl/sqlite fixtures, which CAN inject one.
|
||||
runCoordinatorContract('memory', async (): Promise<CoordinatorFixture> => {
|
||||
const store: MemoryStore = new Map()
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(MemoryPersistence, { store }),
|
||||
cleanup: async () => { store.clear() },
|
||||
}
|
||||
})
|
||||
|
||||
describe('SessionPersistence service registration', () => {
|
||||
it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
expect(ctx.sessionPersistence).toBeInstanceOf(SessionPersistence)
|
||||
|
||||
await fiber.dispose()
|
||||
expect(ctx.sessionPersistence).toBeUndefined()
|
||||
})
|
||||
|
||||
it('round-trips through the registered service instance', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const m = meta('reg')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('shared persistence helpers', () => {
|
||||
it('accepts a seed that reproduces the persisted prefix exactly', () => {
|
||||
const log = oneTurnLog()
|
||||
expect(seedCoversPrefix(log, log.slice(0, 3))).toBe(true)
|
||||
expect(seedCoversPrefix(log, [])).toBe(true)
|
||||
})
|
||||
|
||||
it('rejects a prefix longer than the seed', () => {
|
||||
const log = oneTurnLog()
|
||||
expect(seedCoversPrefix(log.slice(0, 2), log)).toBe(false)
|
||||
})
|
||||
|
||||
it('rejects a same-envelope event with mutated data', () => {
|
||||
const log = oneTurnLog()
|
||||
const tampered = structuredClone(log)
|
||||
const event = tampered[1]!
|
||||
tampered[1] = {
|
||||
...event,
|
||||
data: { ...event.data, content: [{ type: 'text', text: 'tampered' }] },
|
||||
} as SessionEvent
|
||||
expect(seedCoversPrefix(tampered, log.slice(0, 2))).toBe(false)
|
||||
})
|
||||
|
||||
it('accepts JSON-serializable event data', () => {
|
||||
expect(() => { assertSerializable(oneTurnLog()) }).not.toThrow()
|
||||
})
|
||||
|
||||
it('rejects non-JSON-serializable event data with type and seq context', () => {
|
||||
const bad = [
|
||||
{ type: 'user/message', seq: 0, time: 1, data: { content: 1n } },
|
||||
] as unknown as SessionEvent[]
|
||||
expect(() => { assertSerializable(bad) }).toThrow(/"user\/message".*seq 0/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user