fix review findings: skip collided SCHEMA_VERSION 3; reject marker-less surface events
P1: both merge parents shipped SCHEMA_VERSION=3 for different layouts (surface columns vs seed_length), so an on-disk 3 was ambiguous and wrongly accepted. Bump to 4 (merged layout) so the version check rejects both sibling v3s. P2: a surface-eligible event with no surfaceOp lands in the log but vanishes from deriveMessages() (surface is the sole derivation path). The typed append overload enforces the marker only when the type arg is a literal; it collapses to optional when widened to the union (a caller iterating raw events). Guard at runtime in both append() and the seed constructor — no backward-compat for surface-less logs. Shared seed fixtures carry surfaceOp explicitly and the appendLog helper forwards it verbatim (no synthesized default). Exports isSurfaceEligibleType. Regression tests for all three, each verified to fail on the unfixed code. Gates: typecheck, test (1115), snapshot (14), doc-sync, lint, build, hygiene green.
This commit is contained in:
@@ -412,7 +412,7 @@ get(id: SessionId): Session | undefined
|
||||
list(): Session[]
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:300`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:321`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `ctx.subagents` — `SubagentService`
|
||||
|
||||
|
||||
@@ -51,9 +51,11 @@ The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls
|
||||
|
||||
The dev-mode invariants plugin validates: `sourceEventSeqs` references (non-empty, no duplicates, references earlier events, references known seqs) and `surfaceOp` (replace `start ≤ end`, both endpoints are on the tracked surface, the range is non-reversed in surface position, and `sourceEventSeqs` includes every node the range shadows).
|
||||
|
||||
Because the surface is the SOLE derivation path, a surface-eligible event that carries no `surfaceOp` marker is invisible to `deriveMessages()` — it would land in the log yet silently drop from history on resume/fork. `append`'s typed overload makes the marker mandatory for `SurfaceEventType` events at compile time, but only when the type argument is a SPECIFIC literal; when it widens to the `SessionEventType` union (a caller iterating raw events, e.g. `for (const e of log) append(e.type, e.data)`) the conditional rest collapses to optional and the compiler stops enforcing it. The marker requirement is therefore ALSO checked at runtime in two places: `append` itself throws on a marker-less surface-eligible event (covering the union-widening loophole), and the `Session` seed constructor re-checks the same invariant (alongside its seq-contiguity and JSON-serializability checks) so a seed/load/fork — which arrives as raw `SessionEvent[]`, bypassing `append` — is REJECTED rather than constructing a session that resumes with missing history. (No backward-compat path for surface-less logs: per the pre-release stance there is no persisted user data to preserve, so such a log is rejected, not upgraded.)
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`packages/core/session`**: New `surface.ts` (`SurfaceManager`), new types (`SurfaceOp`, `SurfaceIntent`), new fields on `SessionEvent`, modified `append()` (third required `SurfaceIntent` param), refactored `deriveMessages()` (surface path + legacy fallback), surface-aware `repair.ts`.
|
||||
- **`packages/core/session`**: New `surface.ts` (`SurfaceManager`), new types (`SurfaceOp`, `SurfaceIntent`), new fields on `SessionEvent`, modified `append()` (third required `SurfaceIntent` param), refactored `deriveMessages()` (walks the surface as the sole derivation path), surface-aware `repair.ts`. The seed constructor rejects a surface-eligible seed event missing its `surfaceOp` marker (see § Invariants).
|
||||
- **`packages/core/agent-loop`**: All surface-capable appends pass surface opts. Chunk seqs are collected for `assistant/message` provenance; `tool/call` seqs are captured for `tool/result` provenance.
|
||||
- **`packages/session-persistence/session-persistence-sqlite`**: Two new nullable TEXT columns (`source_event_seqs`, `surface_op`) on the `events` table; `SCHEMA_VERSION` bumped (bump-and-reject, no migration).
|
||||
- **`packages/support/invariants`**: Surface-related validation rules.
|
||||
|
||||
@@ -27,7 +27,7 @@ Record where a session's **inherited** prefix ends, persist it, and have the rep
|
||||
- **JSONL**: a `seedLength` field on the header line (`toHeaderLine`/`fromHeaderLine`).
|
||||
- **SQLite**: a `seed_length` column on the `sessions` table.
|
||||
|
||||
The SQLite change is a breaking table-layout change, so `SCHEMA_VERSION` bumps **2 → 3**. Per the repo's pre-release stance (§ "Pre-release stance" in AGENTS.md) the backend **rejects** a non-current `user_version` on open rather than migrating it — there is no persisted user data to preserve, so no migration code is written (the existing reject-not-migrate path at `openDatabase` already enforces this; v1 and now v2 are both rejected).
|
||||
The SQLite change is a breaking table-layout change, so `SCHEMA_VERSION` bumps. This branch added `seed_length` under version **3**; it later merged with the session-surface branch, which had independently shipped its OWN version-3 layout (the `source_event_seqs`/`surface_op` columns). Because an on-disk `3` is ambiguous between the two sibling layouts, the merged build is version **4** (every column), and an on-disk `3` is rejected like any other non-current version. Per the repo's pre-release stance (§ "Pre-release stance" in AGENTS.md) the backend **rejects** a non-current `user_version` on open rather than migrating it — there is no persisted user data to preserve, so no migration code is written (the existing reject-not-migrate path at `openDatabase` already enforces this; v1, v2, and the collided v3 are all rejected).
|
||||
|
||||
### 3. Replay derives a child script after the boundary
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ Creates and holds event-sourced `Session` instances. Persistence is intentionall
|
||||
|
||||
Plain class (not a Cordis Service). Create via `ctx.sessions.create()`.
|
||||
|
||||
- `session.append(type, data, opts?): SessionEvent` — synchronous, never blocks on I/O. **Throws** if `data` is not losslessly JSON-serializable (BigInt, function, symbol, undefined, non-finite number, circular ref, or an exotic object like Map/Set/Date) — the event log is the durable source of truth, so this invariant is enforced at the source (exported as `isJsonValue` for backends to reuse on their replay/fork entry points). A third parameter `opts: SurfaceIntent` carries surface metadata: `surfaceOp` controls how the event enters the surface linked list, and `sourceEventSeqs` records provenance (the seq numbers of events this one derives from). It is **required** for the five `SurfaceEventType` events (every message-producing event must declare how it joins the surface) and rejected by the compiler for non-surface types.
|
||||
- `session.append(type, data, opts?): SessionEvent` — synchronous, never blocks on I/O. **Throws** if `data` is not losslessly JSON-serializable (BigInt, function, symbol, undefined, non-finite number, circular ref, or an exotic object like Map/Set/Date) — the event log is the durable source of truth, so this invariant is enforced at the source (exported as `isJsonValue` for backends to reuse on their replay/fork entry points). A third parameter `opts: SurfaceIntent` carries surface metadata: `surfaceOp` controls how the event enters the surface linked list, and `sourceEventSeqs` records provenance (the seq numbers of events this one derives from). It is **required** for the five `SurfaceEventType` events (every message-producing event must declare how it joins the surface) and rejected by the compiler for non-surface types. The marker requirement is enforced two ways: the typed overload makes `opts` mandatory when `type` is a specific `SurfaceEventType` literal, AND `append` **throws** at runtime if a surface-eligible event arrives with no `surfaceOp` — covering the case where `type` widens to the `SessionEventType` union (a caller iterating raw events, where the conditional overload collapses to optional) so a marker-less message event can never silently land in the log and vanish from `deriveMessages()`.
|
||||
- `session.deriveMessages(): Message[]` — derive the LLM message history by walking the surface linked list (skipping non-surface events like chunks and boundaries; a `replace` shadows the nodes it covers). The surface is the single source of derived history — there is no raw-log fallback.
|
||||
- `session.surface: SurfaceManager` — the derived surface, lazily rebuilt from `surfaceOp` markers in the log. Processes only new events (delta) on each access — the log is append-only, so prior events never change.
|
||||
- `session.events`, `session.seq`, `session.id`
|
||||
@@ -45,6 +45,7 @@ Plain class (not a Cordis Service). Create via `ctx.sessions.create()`.
|
||||
- `SurfaceOp` — how a surface node entered the linked list: `'append'` (normal tail append) or `{ op: 'replace', start, end }` (replace nodes from `start` through `end` inclusive — both must be valid surface node seqs; `start === end` replaces a single node). Used by compaction to shadow old nodes without deleting them.
|
||||
- `SurfaceIntent` — `{ surfaceOp: SurfaceOp; sourceEventSeqs?: number[] }`, the required third parameter to `session.append()` for surface-eligible types.
|
||||
- `SurfaceNode` — `{ seq: number; prev: number | null; next: number | null }`, one node in the surface linked list.
|
||||
- `isSurfaceEvent(event)` / `isSurfaceEligibleType(type)` — the first narrows a `SessionEvent` to a fully-formed surface node (type is surface-eligible AND `surfaceOp` present); the second is the type-only check (is this one of the five `SurfaceEventType` values?), used to detect a surface-eligible event MISSING its marker — e.g. when validating a seed/load log.
|
||||
|
||||
### Session event vocabulary (`types.ts`)
|
||||
|
||||
@@ -66,7 +67,7 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata)
|
||||
### Extension points
|
||||
|
||||
- Persistence plugins: subscribe to `session/event` (write-behind) and drain on `session/flush` (awaited) and fiber dispose. A durable backend reads the log and reloads it into a live session; the metadata seam (`SessionHeader`, `session.header`) is what such a backend stores beside the log.
|
||||
- Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. The surface rebuilds deterministically from `surfaceOp` markers in the seeded events.
|
||||
- Replay/fork: `ctx.sessions.create(id, { seed })` seeds a new session with an existing event log. The surface rebuilds deterministically from `surfaceOp` markers in the seeded events. The seed is validated to the SAME invariants `append` enforces — including that every surface-eligible event (`SurfaceEventType`) carries a `surfaceOp` marker — so a marker-less message event is rejected at construction rather than silently vanishing from `deriveMessages()` (the surface is the sole derivation path) on resume.
|
||||
- Compaction: a future plugin appends a new event with `surfaceOp: { op: 'replace', start, end }` to shadow old surface nodes.
|
||||
|
||||
### What is NOT here (TODO)
|
||||
|
||||
@@ -12,13 +12,13 @@ import type { ContentBlock, Message, MessageSource } from '@deepseek-ai/dsh-llm'
|
||||
import { SESSION_FORMAT_VERSION, SessionId } from './types.ts'
|
||||
import type { CreateSessionOptions, SessionEvent, SessionEventMap, SessionEventType, SessionHeader, SurfaceIntent, SurfaceEventType } from './types.ts'
|
||||
import { isJsonValue } from './json.ts'
|
||||
import { SurfaceManager } from './surface.ts'
|
||||
import { SurfaceManager, isSurfaceEligibleType } from './surface.ts'
|
||||
|
||||
export * from './types.ts'
|
||||
export { isJsonValue } from './json.ts'
|
||||
export { interruptedTurnClosers } from './repair.ts'
|
||||
export type { SurfaceNode } from './surface.ts'
|
||||
export { isSurfaceEvent } from './surface.ts'
|
||||
export { isSurfaceEvent, isSurfaceEligibleType } from './surface.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
@@ -120,6 +120,16 @@ export class Session {
|
||||
if (!isJsonValue(event.data)) {
|
||||
throw new Error(`seed event "${event.type}" (seq ${event.seq}) carries non-JSON-serializable data`)
|
||||
}
|
||||
// Surface-eligible events MUST carry a surfaceOp marker — the surface is
|
||||
// the sole source of derived history, so a marker-less message event
|
||||
// would load fine yet vanish from deriveMessages(). `append` enforces
|
||||
// this at compile time via its typed overload; a seed arrives as raw
|
||||
// SessionEvent[] (replay/fork/load), bypassing that, so re-check at
|
||||
// runtime here rather than silently resuming with empty history.
|
||||
if (isSurfaceEligibleType(event.type)
|
||||
&& (event as SessionEvent<SurfaceEventType>).surfaceOp === undefined) {
|
||||
throw new Error(`seed event "${event.type}" (seq ${event.seq}) is surface-eligible but carries no surfaceOp marker`)
|
||||
}
|
||||
})
|
||||
// Deep-clone each seed event, NOT just the array: the seed events and
|
||||
// their `data` are still owned by the caller (or the source session of a
|
||||
@@ -172,6 +182,18 @@ export class Session {
|
||||
if (!isJsonValue(data)) {
|
||||
throw new Error(`session event "${type}" carries non-JSON-serializable data`)
|
||||
}
|
||||
const surfaceOpts: SurfaceIntent | undefined = opts[0]
|
||||
// Surface-eligible events MUST carry a surfaceOp marker — the surface is the
|
||||
// sole source of derived history, so a marker-less message event would be
|
||||
// logged yet vanish from deriveMessages(). The typed `opts` overload makes
|
||||
// the marker mandatory only when `T` is a SPECIFIC SurfaceEventType literal;
|
||||
// when `T` widens to the SessionEventType union (a caller iterating raw
|
||||
// events: `for (const e of log) append(e.type, e.data)`), the conditional
|
||||
// rest collapses to optional and the compiler stops enforcing it. Re-check
|
||||
// at runtime so that loophole can't silently drop history.
|
||||
if (isSurfaceEligibleType(type) && surfaceOpts?.surfaceOp === undefined) {
|
||||
throw new Error(`session event "${type}" is surface-eligible and requires a surfaceOp marker`)
|
||||
}
|
||||
// Snapshot `data` into the log, NOT the caller's reference: the validation
|
||||
// above proves it is JSON-serializable AT THIS MOMENT, but the caller still
|
||||
// owns the object and could mutate it afterwards (before a persistence
|
||||
@@ -185,7 +207,6 @@ export class Session {
|
||||
// Surface metadata is snapshot separately: sourceEventSeqs (number[] —
|
||||
// primitives, so array spread is a complete copy) and surfaceOp (a string
|
||||
// primitive, or cloned if it's a replace object).
|
||||
const surfaceOpts: SurfaceIntent | undefined = opts[0]
|
||||
// Build the event shape with conditional surface fields via spreading.
|
||||
// The result is cast through `unknown` because the conditional spreads
|
||||
// produce an intersection type that the assignability checker can't
|
||||
|
||||
@@ -22,6 +22,18 @@ const SURFACE_EVENT_TYPES = new Set<string>([
|
||||
'steering/message',
|
||||
])
|
||||
|
||||
/**
|
||||
* Whether an event's `type` is surface-eligible (one of the five
|
||||
* message-producing {@link SurfaceEventType} values). This is the TYPE check
|
||||
* only — it does NOT require `surfaceOp` to be present. Use it to detect a
|
||||
* surface-eligible event that is MISSING its mandatory marker (e.g. validating
|
||||
* a seed/load log); use {@link isSurfaceEvent} to narrow to a fully-formed
|
||||
* {@link SurfaceEvent} with `surfaceOp` present.
|
||||
*/
|
||||
export function isSurfaceEligibleType(type: string): boolean {
|
||||
return SURFACE_EVENT_TYPES.has(type)
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow a {@link SessionEvent} to {@link SurfaceEvent}: checks that the
|
||||
* event's `type` is surface-eligible AND that `surfaceOp` is present.
|
||||
|
||||
@@ -11,22 +11,29 @@ import { describe, expect, it } from 'vitest'
|
||||
import fc from 'fast-check'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEventMap, SessionEventType } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEventMap, SessionEventType, SurfaceIntent } from '@deepseek-ai/dsh-session'
|
||||
|
||||
type Appendable = { [T in SessionEventType]: { type: T; data: SessionEventMap[T] } }[SessionEventType]
|
||||
// An appendable event: its type/data plus, for surface-eligible types, the
|
||||
// explicit surface intent the generator declares (mirroring how a real caller
|
||||
// passes it). The intent is part of the generated fixture, NOT synthesized by
|
||||
// `build`, so each arbitrary states the marker it produces.
|
||||
type Appendable = {
|
||||
[T in SessionEventType]: { type: T; data: SessionEventMap[T]; intent?: SurfaceIntent }
|
||||
}[SessionEventType]
|
||||
|
||||
const textContentArb = fc.array(
|
||||
fc.record({ type: fc.constant<'text'>('text'), text: fc.string() }),
|
||||
{ maxLength: 3 },
|
||||
)
|
||||
|
||||
// A message-producing event (these DO affect derived history).
|
||||
// A message-producing event (these DO affect derived history). Each carries an
|
||||
// explicit `surfaceOp: 'append'` intent — the marker the real loop passes.
|
||||
const messageEventArb: fc.Arbitrary<Appendable> = fc.oneof(
|
||||
textContentArb.map((content): Appendable => ({ type: 'user/message', data: { content, source: { kind: 'user' } } })),
|
||||
textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content } })),
|
||||
textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content, usage: { inputTokens: 1, outputTokens: 1 } } })),
|
||||
textContentArb.map((content): Appendable => ({ type: 'user/message', data: { content, source: { kind: 'user' } }, intent: { surfaceOp: 'append' } })),
|
||||
textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content }, intent: { surfaceOp: 'append' } })),
|
||||
textContentArb.map((content): Appendable => ({ type: 'assistant/message', data: { turn: 1, step: 1, content, usage: { inputTokens: 1, outputTokens: 1 } }, intent: { surfaceOp: 'append' } })),
|
||||
fc.record({ id: fc.string({ minLength: 1 }), content: textContentArb, isError: fc.boolean() })
|
||||
.map((r): Appendable => ({ type: 'tool/result', data: { turn: 1, step: 1, callId: CallId(r.id), content: r.content, isError: r.isError } })),
|
||||
.map((r): Appendable => ({ type: 'tool/result', data: { turn: 1, step: 1, callId: CallId(r.id), content: r.content, isError: r.isError }, intent: { surfaceOp: 'append' } })),
|
||||
)
|
||||
|
||||
// A non-message event (trace/replay data — must NOT affect derived history).
|
||||
@@ -44,7 +51,11 @@ const logArb = fc.array(anyEventArb, { maxLength: 25 })
|
||||
let counter = 0
|
||||
function build(events: Appendable[]): Session {
|
||||
const session = new Session(SessionId(`prop-${counter++}`))
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
for (const e of events) {
|
||||
// Forward the generated intent verbatim; non-surface events carry none.
|
||||
if (e.intent !== undefined) session.append(e.type, e.data, e.intent)
|
||||
else session.append(e.type, e.data)
|
||||
}
|
||||
return session
|
||||
}
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, Session, SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEventType } from '@deepseek-ai/dsh-session'
|
||||
|
||||
describe('Session', () => {
|
||||
it('derives message history from the event log', () => {
|
||||
@@ -120,6 +121,21 @@ describe('Session', () => {
|
||||
expect(session.events).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('rejects a surface-eligible append with no surfaceOp marker (runtime guard for the union-widening loophole)', () => {
|
||||
const session = new Session(SessionId('s5b'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
// The typed overload makes surfaceOp mandatory only when the type argument is
|
||||
// a SPECIFIC SurfaceEventType literal. A caller iterating raw events widens it
|
||||
// to the SessionEventType union, where the conditional rest collapses to
|
||||
// optional — the exact shape `for (const e of log) append(e.type, e.data)`
|
||||
// produces. Reproduce that here and assert the runtime guard rejects it.
|
||||
const widenedType = 'user/message' as SessionEventType
|
||||
expect(() => session.append(widenedType, { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }))
|
||||
.toThrow(/surface-eligible and requires a surfaceOp marker/)
|
||||
// The rejected append never entered the log (only turn/start is present).
|
||||
expect(session.events).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('accepts dense arrays and nested plain objects', () => {
|
||||
const session = new Session(SessionId('s6'))
|
||||
expect(() => session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: [1, 2, [3, { a: null, b: true }]] } as never, { surfaceOp: 'append' })).not.toThrow()
|
||||
@@ -143,10 +159,23 @@ describe('Session', () => {
|
||||
expect(() => new Session(SessionId('seed-gap'), gapSeed)).toThrow(/contiguous|seq/)
|
||||
})
|
||||
|
||||
it('validates seed events: rejects a surface-eligible event missing its surfaceOp marker', () => {
|
||||
// A surface-eligible event (user/message) with no surfaceOp would load fine
|
||||
// but vanish from deriveMessages() (the surface is the sole derivation path),
|
||||
// so a resume/fork would silently lose history. append() forbids this at
|
||||
// compile time; a raw seed must be rejected at runtime to match.
|
||||
const markerlessSeed = [
|
||||
{ type: 'turn/start' as const, seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message' as const, source: { kind: 'user' as const } } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 2, data: { content: [{ type: 'text' as const, text: 'hi' }], source: { kind: 'user' as const } } },
|
||||
{ type: 'turn/end' as const, seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' as const } } },
|
||||
] as SessionEvent[]
|
||||
expect(() => new Session(SessionId('seed-no-marker'), markerlessSeed)).toThrow(/surface-eligible but carries no surfaceOp/)
|
||||
})
|
||||
|
||||
it('accepts a well-formed contiguous serializable seed', () => {
|
||||
const goodSeed = [
|
||||
{ type: 'turn/start' as const, seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message' as const, source: { kind: 'user' as const } } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 2, data: { content: [{ type: 'text' as const, text: 'hi' }], source: { kind: 'user' as const } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 2, data: { content: [{ type: 'text' as const, text: 'hi' }], source: { kind: 'user' as const } }, surfaceOp: 'append' as const },
|
||||
{ type: 'turn/end' as const, seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' as const } } },
|
||||
] as SessionEvent[]
|
||||
const session = new Session(SessionId('seed-ok'), goodSeed)
|
||||
@@ -156,7 +185,7 @@ describe('Session', () => {
|
||||
it('snapshots the seed: mutating the original after construction does not affect session.events', () => {
|
||||
const seed = [
|
||||
{ type: 'turn/start' as const, seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message' as const, source: { kind: 'user' as const } } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 2, data: { content: [{ type: 'text' as const, text: 'original' }], source: { kind: 'user' as const } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 2, data: { content: [{ type: 'text' as const, text: 'original' }], source: { kind: 'user' as const } }, surfaceOp: 'append' as const },
|
||||
{ type: 'turn/end' as const, seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' as const } } },
|
||||
] as SessionEvent[]
|
||||
const session = new Session(SessionId('seed-snapshot'), seed)
|
||||
|
||||
@@ -7,7 +7,7 @@ import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
let root: string
|
||||
@@ -121,7 +121,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'he' } } },
|
||||
{ type: 'assistant/chunk', seq: 3, time: 4, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'llo' } } },
|
||||
{ type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } },
|
||||
{ type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] }, surfaceOp: 'append', sourceEventSeqs: [2, 3] },
|
||||
{ type: 'step/end', seq: 5, time: 6, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 6, time: 7, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
@@ -452,7 +452,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
// Session A materializes a log under id "reuse".
|
||||
const sessFiberA = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
const a = inner.sessions.create(SessionId('reuse'), { meta: { cwd: '/a' } })
|
||||
for (const e of oneTurnLog()) a.append(e.type, e.data)
|
||||
appendLog(a, oneTurnLog())
|
||||
}, { inject: ['sessions'] }))
|
||||
// Drain A, then dispose ITS fiber (the live session A is gone) while the
|
||||
// backend stays loaded.
|
||||
|
||||
@@ -15,7 +15,7 @@ import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepsee
|
||||
* layout; orthogonal to a session's own `version` (which versions the EVENT
|
||||
* vocabulary, stored per session in the `sessions` row).
|
||||
*/
|
||||
export const SCHEMA_VERSION = 3
|
||||
export const SCHEMA_VERSION = 4
|
||||
|
||||
/**
|
||||
* A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}).
|
||||
@@ -56,9 +56,15 @@ export interface EventRow {
|
||||
* current {@link SCHEMA_VERSION}; an existing database whose version is NOT the
|
||||
* current one (written by a different, incompatible build — older or newer) is
|
||||
* REJECTED rather than opened against a layout this build does not understand.
|
||||
* There are no migrations: an earlier layout (v1's different `sessions` shape,
|
||||
* v2 without the `seed_length`/`source_event_seqs`/`surface_op` columns) is not
|
||||
* upgraded in place — it is rejected.
|
||||
* There are no migrations: an earlier layout is not upgraded in place — it is
|
||||
* rejected. v1 had a different `sessions` shape; v2 lacked all of
|
||||
* `seed_length`/`source_event_seqs`/`surface_op`. v3 is SKIPPED: two unmerged
|
||||
* branches each shipped a DISTINCT v3 (one adding only `seed_length`, the other
|
||||
* adding only the surface columns), so an on-disk v3 is ambiguous — it could be
|
||||
* either sibling layout, neither of which has all of this build's columns. v4
|
||||
* is the merged layout carrying every column; bumping past the collided v3
|
||||
* makes the version check reject both sibling v3 databases instead of opening
|
||||
* one against columns it does not have.
|
||||
*/
|
||||
export function openDatabase(path: string): DatabaseSync {
|
||||
const db = new DatabaseSync(path)
|
||||
|
||||
@@ -7,7 +7,7 @@ import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SurfaceEvent, SurfaceEventType } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import { openDatabase, rowToEvent, scanRows, type EventRow } from '../src/schema.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
const dirs: string[] = []
|
||||
@@ -66,9 +66,17 @@ runCoordinatorContract('sqlite', async (): Promise<CoordinatorFixture> => {
|
||||
|
||||
describe('scanRows', () => {
|
||||
// scanRows works off EventRows (data is a JSON string column); build them from
|
||||
// SessionEvents so the unit tests read in terms of the event vocabulary.
|
||||
// SessionEvents so the unit tests read in terms of the event vocabulary. Surface
|
||||
// fields are serialized to their nullable columns so a round trip is faithful.
|
||||
const rows = (events: SessionEvent[]): EventRow[] =>
|
||||
events.map(e => ({ seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data), source_event_seqs: null, surface_op: null }))
|
||||
events.map((e) => {
|
||||
const se = e as SessionEvent<SurfaceEventType>
|
||||
return {
|
||||
seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data),
|
||||
source_event_seqs: se.sourceEventSeqs !== undefined ? JSON.stringify(se.sourceEventSeqs) : null,
|
||||
surface_op: se.surfaceOp !== undefined ? JSON.stringify(se.surfaceOp) : null,
|
||||
}
|
||||
})
|
||||
|
||||
it('preserves the full log when it ends exactly on a turn/end (no torn tail)', () => {
|
||||
const { preserved, tornFrom } = scanRows(rows(oneTurnLog()))
|
||||
@@ -246,6 +254,20 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
expect(() => openDatabase(olderPath)).toThrow(/incompatible with this build/)
|
||||
})
|
||||
|
||||
it('rejects a sibling v3 database (the merge-collided version) rather than opening it against missing columns', async () => {
|
||||
// Two unmerged branches each shipped a DISTINCT layout under user_version 3
|
||||
// (one added only `seed_length`, the other only the surface columns). The
|
||||
// merged build is v4; an on-disk v3 is ambiguous and is missing at least one
|
||||
// of this build's columns, so it MUST be rejected, not opened. Stamp a v3
|
||||
// database and confirm the version check refuses it.
|
||||
const path = await freshDbPath()
|
||||
openDatabase(path).close() // creates + stamps user_version = SCHEMA_VERSION (4)
|
||||
const db = openDatabase(path)
|
||||
db.exec('PRAGMA user_version = 3')
|
||||
db.close()
|
||||
expect(() => openDatabase(path)).toThrow(/schema version 3, incompatible with this build/)
|
||||
})
|
||||
|
||||
it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('corrupt-tail')
|
||||
@@ -315,7 +337,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
})
|
||||
|
||||
it('exposes the schema version constant', () => {
|
||||
expect(SCHEMA_VERSION).toBe(3)
|
||||
expect(SCHEMA_VERSION).toBe(4)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -353,7 +375,7 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
// Instance 1 materializes a session and disposes.
|
||||
const b1 = await backend(path)
|
||||
const s1 = b1.ctx.sessions.create(SessionId('hmr-collide'))
|
||||
for (const e of oneTurnLog()) s1.append(e.type, e.data)
|
||||
appendLog(s1, oneTurnLog())
|
||||
await b1.ctx.parallel('session/flush', s1)
|
||||
await b1.dispose()
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader, SurfaceEventType, SurfaceIntent } from '@deepseek-ai/dsh-session'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
|
||||
@@ -34,14 +34,40 @@ export function meta(id: string, cwd?: string): SessionHeader {
|
||||
export function oneTurnLog(): SessionEvent[] {
|
||||
return [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, surfaceOp: 'append' },
|
||||
{ type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] } },
|
||||
{ type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] }, surfaceOp: 'append' },
|
||||
{ type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* Append a whole event log to a LIVE session, event by event, forwarding the
|
||||
* surface metadata each event already carries. A bare `append(e.type, e.data)`
|
||||
* over a `SessionEvent[]` widens the type argument to the union, where the
|
||||
* typed overload's mandatory-marker rule collapses to optional — and `append`'s
|
||||
* runtime guard then rejects a surface-eligible event with no marker. This
|
||||
* helper forwards the `surfaceOp`/`sourceEventSeqs` VERBATIM from the source
|
||||
* event (it does not synthesize a default), so a well-formed recorded log
|
||||
* round-trips through a live session intact and a fixture that forgot a marker
|
||||
* still trips the guard.
|
||||
*/
|
||||
export function appendLog(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) {
|
||||
const se = e as SessionEvent<SurfaceEventType>
|
||||
if (se.surfaceOp !== undefined) {
|
||||
const intent: SurfaceIntent = {
|
||||
surfaceOp: se.surfaceOp,
|
||||
...se.sourceEventSeqs !== undefined ? { sourceEventSeqs: se.sourceEventSeqs } : {},
|
||||
}
|
||||
session.append(e.type, e.data, intent)
|
||||
} else {
|
||||
session.append(e.type, e.data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the backend-agnostic contract suite. `make()` MUST return a fresh, empty
|
||||
* backend each call.
|
||||
|
||||
@@ -31,7 +31,7 @@ import { Context, type Fiber } from 'cordis'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
import { meta, oneTurnLog } from './contract.ts'
|
||||
import { meta, oneTurnLog, appendLog } from './contract.ts'
|
||||
|
||||
/**
|
||||
* The backend-specific capabilities the orchestration suite needs beyond the
|
||||
@@ -76,7 +76,7 @@ function inits(persistence: SessionPersistence): Map<Session, Promise<void>> {
|
||||
|
||||
/** Append a whole event log to a live session, event by event (drives session/event). */
|
||||
function send(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) session.append(e.type, e.data)
|
||||
appendLog(session, events)
|
||||
}
|
||||
|
||||
/** A live session created inside its OWN fiber, so it survives a backend reload. */
|
||||
|
||||
@@ -306,7 +306,7 @@ describe('dev-freeze', () => {
|
||||
const { ctx } = await setup()
|
||||
const seed = [
|
||||
{ type: 'turn/start' as const, seq: 0, time: 0, data: { turn: 1, trigger: { kind: 'message' as const, source: { kind: 'user' as const } } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 0, data: { content: [{ type: 'text' as const, text: 'seeded' }], source: { kind: 'user' as const } } },
|
||||
{ type: 'user/message' as const, seq: 1, time: 0, data: { content: [{ type: 'text' as const, text: 'seeded' }], source: { kind: 'user' as const } }, surfaceOp: 'append' as const },
|
||||
]
|
||||
const session = ctx.sessions.create(undefined, { seed })
|
||||
expect(Object.isFrozen(session.events[0])).toBe(true)
|
||||
|
||||
Reference in New Issue
Block a user