Add per-session snapshot replay for nested agents (PR2.5)

The snapshot tier was built single-session: dsh-llm-replay served calls from
one global positional cursor, and the harness harvested one session log. A
subagent runs as a second agent with its own session, so a parent→child
scenario could neither replay deterministically nor harvest the child's log.
This resolves the TODO(subagent-snapshots) deferral from the subagent RFC.

- Stamp the calling session id onto the model request: GenerateOptions.sessionId
  (typed Branded<'SessionId'> to avoid the dsh-llm↔dsh-session cycle), set by the
  agent loop from agent.session.id. Adapters ignore it; an llm/stream listener
  routes by it.
- Key replay per session: dsh-llm-replay loads the parent log plus one per child
  (childFiles / $DSH_SNAPSHOT_CHILD_FILES), derives a script per recorded session,
  and binds each live (freshly-random) session to a recorded script by first-call
  order — parent first (earliest createdAt, first to stream). Keys by WHO calls,
  so it survives a future concurrent/backgrounded subagent; a global cursor would
  not. An unrecorded extra session fails loud.
- Harvest every log: the harness collects all .jsonl across cwd buckets, ordered
  primary-first (top-level, then children by createdAt), and RunResult exposes the
  plural sessionLogs. The spec writes each back on record (session.jsonl +
  session.<n>.jsonl) and diffs each against its fixture on replay.
- Wire the subagent seam + spawn + fork + tool into the acp-agent example (both
  cordis configs) and add two nested scenarios recorded against the real API:
  subagent-spawn (parent + 1 child) and subagent-multi (parent + 2 children, 3
  sessions). Both replay keyless in the default gate.

A new RFC documents the design (docs/rfc/implemented/testing/). Single-session
replay is unchanged (a call with no sessionId is one anonymous primary session).

TODO follow-up: a dedicated branded-ids package could own the SessionId brand and
dissolve the cross-package cycle note; out of scope for this testing PR.
This commit is contained in:
Tianyi Cui
2026-06-22 08:39:36 +08:00
parent 9c1048f2b5
commit e68496fd79
22 changed files with 1392 additions and 88 deletions

View File

@@ -570,6 +570,7 @@ async function runStep(
messages: session.deriveMessages(),
...system ? { system } : {},
...assembly.tools.length > 0 ? { tools: assembly.tools } : {},
sessionId: session.id,
signal,
}
request = await ctx.waterfall('agent/request', agent, turn, step, request, () => Promise.resolve(request))

View File

@@ -19,6 +19,7 @@
* ```
*/
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId } from './brand.ts'
/** Cache hint attached to a content block (provider-interpreted). */
@@ -192,4 +193,18 @@ export interface GenerateOptions {
*/
stop?: string[]
signal?: AbortSignal
/**
* The id of the session this request belongs to — stamped by the agent loop
* from `agent.session.id`. Adapters ignore it; it lets an `llm/stream` listener
* route a call by WHICH session issued it (the replay adapter keys its per-call
* cursor by session, so a parent and its in-process subagent — each with its
* own session on one context — replay from their own recorded scripts).
*
* Typed as `Branded<'SessionId'>` rather than importing `SessionId` from
* `dsh-session`: that package imports `Message` from here, so importing its
* `SessionId` back would cycle. `SessionId` IS `Branded<'SessionId'>`, so a
* real session id assigns with no cast. (A future ids package could own the
* brand and dissolve this note.)
*/
sessionId?: Branded<'SessionId'>
}

View File

@@ -10,26 +10,35 @@ The fixture IS the persisted session log (`<scenario>/session.jsonl`). Its `assi
Two failure modes are not reconstructable from `assistant/chunk` alone — a pure throw before any chunk (e.g. an HTTP 401, where the log holds only a `turn/end {error}` and no chunks) and a cancel/hang (timing, not chunk content). A scenario that needs those supplies an optional sidecar (`<scenario>/replay.override.json`: a `ReplayEntry[]`) that REPLACES the derived script.
## Nested agents: per-session keying
A scenario where a parent agent delegates to in-process subagents records more than one log: the parent (`session.jsonl`) plus one per child (`session.1.jsonl`, …). Each agent runs as its own `Session` on the same context, so replay must serve each one its own script.
Replay keys every call by its calling session id (`GenerateOptions.sessionId`, stamped by the agent loop). Live session ids are freshly random each run and never equal the recorded ones, so a live session binds to a recorded script by **first-call order**: scripts are ordered by header `createdAt` (parent first — it streams before it can delegate), and the first live session to make any call claims the first script, the next new session the next, and so on. Each session then advances its own cursor. A call with no `sessionId` is one anonymous session bound to the primary script, so single-session scenarios behave exactly as before. More distinct live sessions than recorded scripts fails loud.
## Config
| Key | Type | Default | Notes |
|---|---|---|---|
| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the per-scenario `session.jsonl` fixture. Required (config or env). |
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional path to a `ReplayEntry[]` sidecar that replaces the derived script. |
| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the primary (parent) `session.jsonl` fixture. Required (config or env). |
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional path to a `ReplayEntry[]` sidecar that replaces the PRIMARY session's derived script. |
| `childFiles` | string[] | `$DSH_SNAPSHOT_CHILD_FILES` (path-delimited) | Recorded subagent child-session logs for a nested scenario; empty for a single-session scenario. |
```yaml
- id: llm-replay
name: '@deepseek-ai/dsh-llm-replay'
# file/overrideFile default to $DSH_SNAPSHOT_FILE / $DSH_SNAPSHOT_OVERRIDE,
# set by the snapshot harness per scenario.
# file/overrideFile/childFiles default to $DSH_SNAPSHOT_FILE /
# $DSH_SNAPSHOT_OVERRIDE / $DSH_SNAPSHOT_CHILD_FILES, set by the snapshot
# harness per scenario.
```
## Exports
- `installLlmReplay(ctx, config)` — install the `llm/stream` listener; returns the disposer (HMR safety). Use this in tests to drive replay without the Loader or env vars.
- `loadReplayScript(config)` — resolve the `ReplayEntry[]` for a scenario (sidecar override if present, else derived from the JSONL; fail-loud if the fixture is missing).
- `deriveReplayScript(events)` / `parseSessionLog(text)` — the pure helpers that turn a recorded session log into a script. A derived group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar.
- Types `ReplayEntry` / `ReplayConfig` / `Config`.
- `loadSessionScripts(config)` — resolve the ordered `SessionScript[]` (primary + children) for a scenario, ready to bind to live sessions in first-call order.
- `loadReplayScript(config)` — resolve the `ReplayEntry[]` for the PRIMARY session only (sidecar override if present, else derived from the JSONL; fail-loud if the fixture is missing).
- `deriveReplayScript(events)` / `parseSessionLog(text)` / `parseSessionHeader(text)` — the pure helpers that turn a recorded session log into a script and read its header `id`/`createdAt`. A derived group must end in a `finish` chunk; a group without one is the fingerprint of a thrown `stream()` and must instead be expressed via an override sidecar.
- Types `ReplayEntry` / `SessionScript` / `ReplayConfig` / `Config`.
## Plugin export shape

View File

@@ -14,6 +14,15 @@
* therefore "run the real agent once and harvest the `.jsonl`", done by the
* snapshot harness — this plugin does not record.
*
* A NESTED-agent scenario records more than one log: the parent plus one per
* in-process subagent (each subagent runs as its own {@link Session} on the same
* context). Replay loads them all ({@link loadSessionScripts}), derives a script
* per recorded session, and keys each live call by its calling session id
* (`GenerateOptions.sessionId`, stamped by the loop). Live session ids are fresh
* random values, so a live session binds to a recorded script by FIRST-CALL
* order (parent first — it streams before it delegates); see
* {@link installLlmReplay}.
*
* Two failure modes are NOT reconstructable from `assistant/chunk` alone — a
* pure throw before any chunk (e.g. an HTTP 401: the log holds only a
* `turn/end {error}`, no chunks) and a cancel/hang (timing, not chunk content).
@@ -36,6 +45,7 @@
*/
import { existsSync, readFileSync } from 'node:fs'
import { delimiter as pathDelimiter } from 'node:path'
import type { Context } from 'cordis'
import type { SessionEvent } from '@deepseek-ai/dsh-session'
import type { GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
@@ -65,14 +75,49 @@ export type ReplayEntry =
/** Resolved plugin configuration. */
export interface ReplayConfig {
/** Path to the per-scenario `session.jsonl` fixture (the recorded log). */
/**
* Path to the PRIMARY (parent) `session.jsonl` fixture. For a single-session
* scenario this is the only log; for a nested-agent scenario it is the parent,
* and the child logs ride in {@link childFiles}.
*/
file: string
/**
* Optional path to a `ReplayEntry[]` sidecar that REPLACES the derived
* script. Used by the two scenarios not expressible as `assistant/chunk`
* (pure throw-before-chunk, cancel/hang). Absent for normal scenarios.
* Optional `ReplayEntry[]` sidecar that REPLACES the derived script for the
* PRIMARY session. Used by the two single-session scenarios not expressible as
* `assistant/chunk` (pure throw-before-chunk, cancel/hang). Absent for normal
* and nested scenarios.
*/
overrideFile?: string
/**
* Additional recorded child-session logs (a nested-agent scenario's subagent
* sessions). Each is derived independently; the full set is ordered by
* `createdAt` so the parent (earliest) binds to the first live session. Empty
* for a single-session scenario.
*/
childFiles?: string[]
}
/**
* One recorded session's replay script: the per-call entries plus the header
* facts needed to ORDER and key it. Live session ids are freshly random at
* replay time and never equal the recorded `id`, so the recorded id is only a
* diagnostic; `createdAt` is the load-bearing field — scripts are ordered by it
* (a parent is created before its children) and each newly-seen live session is
* bound to the next script in that order (= first-call order in the synchronous
* nested cut, where the parent streams before it delegates).
*/
export interface SessionScript {
/** The recorded session id (diagnostics only — the live id differs). */
recordedId: string
/** Session creation time; the deterministic ordering key (parent < child). */
createdAt: number
/** The per-`stream()`-call replay entries, in recorded call order. */
entries: ReplayEntry[]
/**
* Whether this is the PRIMARY (parent) session. Breaks a `createdAt` tie in
* favor of the parent, which always issues the first model call.
*/
primary: boolean
}
/**
@@ -93,6 +138,23 @@ export function parseSessionLog(text: string): SessionEvent[] {
return events
}
/**
* Read the identifying facts off a session log's header line (line 0): the
* recorded session `id` (diagnostics) and `createdAt` (the deterministic
* ordering key that binds a recorded script to a live session — see
* {@link SessionScript}). A header missing either field falls back to a stable
* default (`''` / `0`) rather than throwing: a no-model fixture is header-only
* and still orders fine as the single (primary) script.
*/
export function parseSessionHeader(text: string): { id: string; createdAt: number } {
const firstLine = text.split('\n').find(line => line.trim().length > 0) ?? '{}'
const parsed = JSON.parse(firstLine) as { id?: unknown; createdAt?: unknown }
return {
id: typeof parsed.id === 'string' ? parsed.id : '',
createdAt: typeof parsed.createdAt === 'number' ? parsed.createdAt : 0,
}
}
/**
* Reconstruct the per-`stream()` replay script from a recorded session log.
*
@@ -144,11 +206,11 @@ export function deriveReplayScript(events: SessionEvent[]): ReplayEntry[] {
}
/**
* Build the replay script for a scenario: the sidecar override if present,
* otherwise the script derived from the recorded session JSONL. Fail-loud if
* the JSONL fixture is missing (the scenario was never recorded) — never
* silently returns an empty script, so a coverage hole can't masquerade as a
* passing replay.
* Build the replay script for the PRIMARY session: the sidecar override if
* present, otherwise the script derived from the recorded session JSONL.
* Fail-loud if the JSONL fixture is missing (the scenario was never recorded) —
* never silently returns an empty script, so a coverage hole can't masquerade
* as a passing replay.
*/
export function loadReplayScript(config: ReplayConfig): ReplayEntry[] {
if (config.overrideFile !== undefined && existsSync(config.overrideFile)) {
@@ -164,6 +226,54 @@ export function loadReplayScript(config: ReplayConfig): ReplayEntry[] {
return deriveReplayScript(parseSessionLog(readFileSync(config.file, 'utf8')))
}
/**
* Load every recorded session's script for a scenario, ordered by `createdAt`
* (earliest first), ready to bind to live sessions in first-call order.
*
* The PRIMARY session (`config.file`, with its optional `overrideFile`) is the
* parent; each `config.childFiles` entry is a recorded subagent session. A
* single-session scenario has no `childFiles`, so this returns one script and
* behaves exactly like the old single-cursor replay. The primary always sorts
* first when ties occur (a sub-millisecond parent/child `createdAt` collision):
* the parent issues the FIRST model call (it must stream before it can delegate
* in the synchronous nested cut), so binding it to the first live session is
* correct regardless of a timestamp tie.
*/
export function loadSessionScripts(config: ReplayConfig): SessionScript[] {
const primaryEntries = loadReplayScript(config)
// The override path replaces the derived script but carries no header; read
// the header off the JSONL when it exists, else use a stable default so an
// override-only fixture (header-less) still orders first as the primary.
const primaryHeader = existsSync(config.file)
? parseSessionHeader(readFileSync(config.file, 'utf8'))
: { id: '', createdAt: 0 }
const primary: SessionScript = {
recordedId: primaryHeader.id, createdAt: primaryHeader.createdAt, entries: primaryEntries, primary: true,
}
const children: SessionScript[] = []
for (const childFile of config.childFiles ?? []) {
if (!existsSync(childFile)) {
throw new Error(`llm-replay: child fixture not found: ${childFile} — re-record the scenario`)
}
const text = readFileSync(childFile, 'utf8')
const header = parseSessionHeader(text)
children.push({
recordedId: header.id,
createdAt: header.createdAt,
entries: deriveReplayScript(parseSessionLog(text)),
primary: false,
})
}
// The primary (parent) always binds first — it issues the first model call,
// because it must run a turn before it can delegate. Children follow in
// createdAt order (the order they were spawned in the synchronous nested cut),
// ties broken by recorded id for determinism. Keeping the primary at the head
// rather than sorting it among the children means a sub-millisecond
// parent/child createdAt collision can never reorder it behind a child.
children.sort((a, b) => a.createdAt - b.createdAt || a.recordedId.localeCompare(b.recordedId))
return [primary, ...children]
}
/** Yield a recorded stream back, honoring abort like a real adapter. */
async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined): AsyncIterable<StreamChunk> {
switch (entry.kind) {
@@ -206,32 +316,70 @@ async function* replayEntry(entry: ReplayEntry, signal: AbortSignal | undefined)
* disposer (so a fiber dispose removes it — HMR safety). Exported separately
* from {@link apply} so unit tests can drive it without the Loader or env vars.
*
* Replay is POSITIONAL: the Nth `stream()` call serves the Nth script entry.
* This is deterministic only with at most one model stream in flight at a time;
* the snapshot harness runs one ACP session per scenario to guarantee that. The
* cursor is advanced synchronously at listener-invocation time (not lazily
* inside the generator) so call ORDER, not iteration order, fixes the mapping.
* Replay is PER-SESSION POSITIONAL: each recorded session has its own script
* (parent + any subagent children, loaded by {@link loadSessionScripts} ordered
* by `createdAt`), and the Nth `stream()` call FROM A GIVEN SESSION serves that
* session's Nth entry. The calling session is read off `options.sessionId` (the
* agent loop stamps it from `agent.session.id`).
*
* TODO(subagent-snapshots): this single global cursor cannot route calls to the
* right agent when a parent and an in-process subagent both stream on one ctx.
* Snapshot coverage of nested agents needs either per-session-keyed replay (a
* `Map<sessionId, cursor>` fed by the calling agent on the `agent/request`
* waterfall, which carries the agent) or a call-ordered merge of the parent and
* child session logs (sound because subagent execution is strictly nested —
* the parent blocks on the child). Tracked as a stacked follow-up to the
* in-process subagent backends; see the subagent RFC's "Snapshot coverage of
* nested agents" deferral.
* Live session ids are freshly random and never equal the recorded ones, so a
* live session binds to a recorded script by FIRST-CALL ORDER: the first live
* session to make any call takes the first ordered script (the parent — earliest
* `createdAt`, and the first to stream because it must run before it delegates),
* the next new live session takes the next script, and so on. This keys by WHO
* calls rather than global call order, so it stays correct even if subagents
* ever run concurrently/backgrounded (a global cursor would interleave them).
*
* A call with no `sessionId` (a direct unit-test `ctx.llm.stream` that omits it)
* is treated as one anonymous session — it binds to the first script, so the
* single-session path behaves exactly as the old global cursor did.
*
* Each per-session cursor advances synchronously at listener-invocation time
* (not lazily inside the generator) so call ORDER within a session, not
* iteration order, fixes the mapping.
*/
export function installLlmReplay(ctx: Context, config: ReplayConfig): () => void {
const entries = loadReplayScript(config)
let cursor = 0
const scripts = loadSessionScripts(config)
// Live-session → its bound script + cursor. A new live session id claims the
// next not-yet-bound script (scripts are in bind order); `nextScript` is the
// index of the next unclaimed one.
const bound = new Map<string, { entries: ReplayEntry[]; cursor: number }>()
let nextScript = 0
const ANON = '\0anon\0' // the key for a call that carries no sessionId
return ctx.on('llm/stream', (options: GenerateOptions, _next) => {
const index = cursor++
const entry: ReplayEntry | undefined = entries[index]
const key = options.sessionId ?? ANON
let state = bound.get(key)
let unrecorded = false
if (state === undefined) {
const script = scripts[nextScript]
if (script === undefined) {
// More distinct live sessions made calls than the scenario recorded —
// an unrecorded subagent appeared. Defer the throw into the returned
// generator (the listener must return an AsyncIterable, not throw).
unrecorded = true
state = { entries: [], cursor: 0 }
} else {
nextScript++
state = { entries: script.entries, cursor: 0 }
bound.set(key, state)
}
}
const boundState = state
const seenSessions = nextScript
const totalScripts = scripts.length
const index = boundState.cursor++
const entry: ReplayEntry | undefined = boundState.entries[index]
return (async function* () {
if (unrecorded) {
throw new Error(
`llm-replay: a model call arrived from an unrecorded session (#${seenSessions + 1}); `
+ `the scenario recorded only ${totalScripts} session(s) — re-record it`,
)
}
if (entry === undefined) {
throw new Error(
`llm-replay: script exhausted — requested model call #${index + 1} but the fixture has only ${entries.length}; re-record the scenario`,
`llm-replay: script exhausted — session requested model call #${index + 1} `
+ `but its script has only ${boundState.entries.length}; re-record the scenario`,
)
}
yield* replayEntry(entry, options.signal)
@@ -247,6 +395,12 @@ export interface Config {
file?: string
/** Override the sidecar path; defaults to `$DSH_SNAPSHOT_OVERRIDE`. */
overrideFile?: string
/**
* Override the child-log paths; defaults to `$DSH_SNAPSHOT_CHILD_FILES` (a
* path-separator-delimited list). Each is a recorded subagent session log for
* a nested-agent scenario; absent/empty for a single-session scenario.
*/
childFiles?: string[]
}
export function apply(ctx: Context, config: Config = {}): void {
@@ -255,5 +409,12 @@ export function apply(ctx: Context, config: Config = {}): void {
throw new Error('llm-replay: a fixture path is required (Config.file or $DSH_SNAPSHOT_FILE)')
}
const overrideFile = config.overrideFile ?? process.env.DSH_SNAPSHOT_OVERRIDE
installLlmReplay(ctx, overrideFile === undefined || overrideFile.length === 0 ? { file } : { file, overrideFile })
const childEnv = process.env.DSH_SNAPSHOT_CHILD_FILES
const childFiles = config.childFiles
?? (childEnv !== undefined && childEnv.length > 0 ? childEnv.split(pathDelimiter) : [])
installLlmReplay(ctx, {
file,
...overrideFile !== undefined && overrideFile.length > 0 ? { overrideFile } : {},
...childFiles.length > 0 ? { childFiles } : {},
})
}

View File

@@ -7,12 +7,15 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session'
import LlmService, { GenerateOptions, LlmAdapter, StreamChunk } from '@deepseek-ai/dsh-llm'
import {
type ReplayEntry,
type SessionScript,
apply,
deriveReplayScript,
inject,
installLlmReplay,
loadReplayScript,
loadSessionScripts,
name,
parseSessionHeader,
parseSessionLog,
} from '../src/index.ts'
@@ -32,9 +35,14 @@ const TEXT_CHUNKS: StreamChunk[] = [
]
/** Build a minimal session-JSONL string: a header line + the given events. */
function sessionJsonl(events: SessionEvent[]): string {
const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 })
return [header, ...events.map(e => JSON.stringify(e))].join('\n') + '\n'
function sessionJsonl(events: SessionEvent[], header?: { id?: string; createdAt?: number }): string {
const headerLine = JSON.stringify({
type: 'session',
version: 0,
id: header?.id ?? 's1',
createdAt: header?.createdAt ?? 0,
})
return [headerLine, ...events.map(e => JSON.stringify(e))].join('\n') + '\n'
}
/** A SessionEvent of type assistant/chunk for (turn, step). */
@@ -361,13 +369,188 @@ describe('installLlmReplay (through the real waterfall)', () => {
})
})
describe('parseSessionHeader', () => {
it('reads id and createdAt off the header line', () => {
expect(parseSessionHeader(sessionJsonl([], { id: 'abc', createdAt: 42 })))
.toEqual({ id: 'abc', createdAt: 42 })
})
it('falls back to id="" / createdAt=0 when the header lacks them', () => {
expect(parseSessionHeader('{"type":"session","version":0}\n')).toEqual({ id: '', createdAt: 0 })
})
it('falls back on an empty buffer (no header line)', () => {
expect(parseSessionHeader('')).toEqual({ id: '', createdAt: 0 })
})
})
describe('loadSessionScripts', () => {
/** Write a session log file and return its path. */
function writeSession(filename: string, header: { id: string; createdAt: number }, calls: StreamChunk[][]): string {
let seq = 1
const events: SessionEvent[] = []
calls.forEach((chunks, step) => { for (const c of chunks) events.push(chunkEvent(seq++, 1, step + 1, c)) })
const path = join(dir, filename)
writeFileSync(path, sessionJsonl(events, header), 'utf8')
return path
}
it('returns one primary script for a single-session scenario', () => {
const f = writeSession('session.jsonl', { id: 'p', createdAt: 100 }, [TEXT_CHUNKS])
const scripts: SessionScript[] = loadSessionScripts({ file: f })
expect(scripts).toHaveLength(1)
expect(scripts[0]).toMatchObject({ recordedId: 'p', createdAt: 100, primary: true })
expect(scripts[0]?.entries).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }])
})
it('orders parent + children by createdAt with the primary first on a tie', () => {
const f = writeSession('session.jsonl', { id: 'parent', createdAt: 100 }, [TEXT_CHUNKS])
// One child created LATER, one child sharing the parent's createdAt (tie).
const later = writeSession('session.1.jsonl', { id: 'late', createdAt: 200 }, [TEXT_CHUNKS])
const tie = writeSession('session.2.jsonl', { id: 'tie', createdAt: 100 }, [TEXT_CHUNKS])
const scripts = loadSessionScripts({ file: f, childFiles: [later, tie] })
// parent (100, primary) → tie (100, non-primary) → late (200).
expect(scripts.map(s => s.recordedId)).toEqual(['parent', 'tie', 'late'])
expect(scripts[0]?.primary).toBe(true)
})
it('throws when a declared child fixture is missing', () => {
const f = writeSession('session.jsonl', { id: 'p', createdAt: 1 }, [TEXT_CHUNKS])
expect(() => loadSessionScripts({ file: f, childFiles: [join(dir, 'absent.jsonl')] }))
.toThrow(/child fixture not found/)
})
it('uses the override for the primary and still derives children', () => {
writeFileSync(file, sessionJsonl([], { id: 'p', createdAt: 1 }), 'utf8')
const overrideFile = join(dir, 'replay.override.json')
const override: ReplayEntry[] = [{ kind: 'hang' }]
writeFileSync(overrideFile, JSON.stringify(override), 'utf8')
const child = writeSession('session.1.jsonl', { id: 'c', createdAt: 2 }, [TEXT_CHUNKS])
const scripts = loadSessionScripts({ file, overrideFile, childFiles: [child] })
expect(scripts[0]?.entries).toEqual(override)
expect(scripts[1]?.entries).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }])
})
it('defaults the primary header to id="" / createdAt=0 when only an override (no JSONL) exists', () => {
// An override-only fixture: config.file does NOT exist, the override drives
// the primary script, so the header default branch applies.
const overrideFile = join(dir, 'replay.override.json')
writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8')
const scripts = loadSessionScripts({ file: join(dir, 'absent.jsonl'), overrideFile })
expect(scripts).toHaveLength(1)
expect(scripts[0]).toMatchObject({ recordedId: '', createdAt: 0, primary: true })
})
it('orders two same-createdAt children deterministically after the primary', () => {
// Two children sharing a createdAt (both non-primary): exercises the sort
// tie-break\'s "both same primary-ness" arm and a non-primary-vs-primary arm.
const f = writeSession('session.jsonl', { id: 'parent', createdAt: 100 }, [TEXT_CHUNKS])
const c1 = writeSession('session.1.jsonl', { id: 'c1', createdAt: 100 }, [TEXT_CHUNKS])
const c2 = writeSession('session.2.jsonl', { id: 'c2', createdAt: 100 }, [TEXT_CHUNKS])
const scripts = loadSessionScripts({ file: f, childFiles: [c1, c2] })
// Primary first (its createdAt ties the children but primary wins); the two
// children keep a stable relative order.
expect(scripts[0]?.recordedId).toBe('parent')
expect(scripts.every(s => s.createdAt === 100)).toBe(true)
expect(scripts.map(s => s.primary)).toEqual([true, false, false])
})
it('keeps the primary first even when a child sorts BEFORE it in input order', () => {
// The primary is appended first internally but the child has an EARLIER
// createdAt — the primary must still win on the tie-break against a
// later-but-equal child, and lose only to a genuinely earlier child via
// createdAt (here the child is earlier, so order is child-then-primary only
// if createdAt strictly less; equal createdAt keeps primary first).
const f = writeSession('session.jsonl', { id: 'parent', createdAt: 100 }, [TEXT_CHUNKS])
const earlier = writeSession('session.1.jsonl', { id: 'early', createdAt: 100 }, [TEXT_CHUNKS])
const scripts = loadSessionScripts({ file: f, childFiles: [earlier] })
// Equal createdAt → primary first.
expect(scripts.map(s => s.recordedId)).toEqual(['parent', 'early'])
})
})
describe('installLlmReplay (per-session keying)', () => {
const second: StreamChunk[] = [
{ type: 'block-start', index: 0, blockType: 'text' },
{ type: 'text-delta', index: 0, text: 'child' },
{ type: 'finish', reason: { kind: 'stop' } },
]
/** Write a session log file and return its path. */
function writeSession(filename: string, header: { id: string; createdAt: number }, calls: StreamChunk[][]): string {
let seq = 1
const events: SessionEvent[] = []
calls.forEach((chunks, step) => { for (const c of chunks) events.push(chunkEvent(seq++, 1, step + 1, c)) })
const path = join(dir, filename)
writeFileSync(path, sessionJsonl(events, header), 'utf8')
return path
}
const live = (id: string): GenerateOptions =>
({ model: 'm', messages: [], sessionId: id as NonNullable<GenerateOptions['sessionId']> })
it('routes each live session to its own script by FIRST-CALL order', async () => {
const parentFile = writeSession('session.jsonl', { id: 'rec-parent', createdAt: 100 }, [TEXT_CHUNKS])
const childFile = writeSession('session.1.jsonl', { id: 'rec-child', createdAt: 200 }, [second])
const ctx = new Context()
await ctx.plugin(LlmService)
installLlmReplay(ctx, { file: parentFile, childFiles: [childFile] })
// The first live session to call binds to the parent script; a different
// live session id binds to the child script — regardless of recorded ids.
expect(await drain(ctx.llm.stream(live('live-A')))).toEqual(TEXT_CHUNKS)
expect(await drain(ctx.llm.stream(live('live-B')))).toEqual(second)
// The first session's SECOND call would exhaust its 1-entry script.
await expect(drain(ctx.llm.stream(live('live-A')))).rejects.toThrow(/exhausted/)
})
it('keeps each session\'s cursor independent (interleaved calls)', async () => {
const a2: StreamChunk[] = [{ type: 'text-delta', index: 0, text: 'a2' }, { type: 'finish', reason: { kind: 'stop' } }]
const b2: StreamChunk[] = [{ type: 'text-delta', index: 0, text: 'b2' }, { type: 'finish', reason: { kind: 'stop' } }]
const parentFile = writeSession('session.jsonl', { id: 'p', createdAt: 1 }, [TEXT_CHUNKS, a2])
const childFile = writeSession('session.1.jsonl', { id: 'c', createdAt: 2 }, [second, b2])
const ctx = new Context()
await ctx.plugin(LlmService)
installLlmReplay(ctx, { file: parentFile, childFiles: [childFile] })
// Interleave: A#1, B#1, A#2, B#2 — each cursor advances per-session.
expect(await drain(ctx.llm.stream(live('A')))).toEqual(TEXT_CHUNKS)
expect(await drain(ctx.llm.stream(live('B')))).toEqual(second)
expect(await drain(ctx.llm.stream(live('A')))).toEqual(a2)
expect(await drain(ctx.llm.stream(live('B')))).toEqual(b2)
})
it('treats a call with no sessionId as the single anonymous (primary) session', async () => {
const parentFile = writeSession('session.jsonl', { id: 'p', createdAt: 1 }, [TEXT_CHUNKS])
const ctx = new Context()
await ctx.plugin(LlmService)
installLlmReplay(ctx, { file: parentFile })
// No sessionId at all — the legacy single-session path.
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
})
it('fails loud when more distinct live sessions call than were recorded', async () => {
const parentFile = writeSession('session.jsonl', { id: 'p', createdAt: 1 }, [TEXT_CHUNKS])
const ctx = new Context()
await ctx.plugin(LlmService)
installLlmReplay(ctx, { file: parentFile }) // only ONE recorded session
expect(await drain(ctx.llm.stream(live('first')))).toEqual(TEXT_CHUNKS)
// A SECOND distinct live session has no script to bind to.
await expect(drain(ctx.llm.stream(live('second')))).rejects.toThrow(/unrecorded session/)
})
})
describe('apply (the plugin entry)', () => {
const ORIG = { file: process.env.DSH_SNAPSHOT_FILE, override: process.env.DSH_SNAPSHOT_OVERRIDE }
const ORIG = {
file: process.env.DSH_SNAPSHOT_FILE,
override: process.env.DSH_SNAPSHOT_OVERRIDE,
children: process.env.DSH_SNAPSHOT_CHILD_FILES,
}
afterEach(() => {
if (ORIG.file === undefined) delete process.env.DSH_SNAPSHOT_FILE
else process.env.DSH_SNAPSHOT_FILE = ORIG.file
if (ORIG.override === undefined) delete process.env.DSH_SNAPSHOT_OVERRIDE
else process.env.DSH_SNAPSHOT_OVERRIDE = ORIG.override
if (ORIG.children === undefined) delete process.env.DSH_SNAPSHOT_CHILD_FILES
else process.env.DSH_SNAPSHOT_CHILD_FILES = ORIG.children
})
it('exposes the namespace plugin shape (name/inject, no default export)', () => {
@@ -418,4 +601,52 @@ describe('apply (the plugin entry)', () => {
await ctx.plugin(LlmService)
expect(() => { apply(ctx, { file: '' }) }).toThrow(/a fixture path is required/)
})
it('loads child fixtures from config.childFiles (per-session routing)', async () => {
const childSecond: StreamChunk[] = [
{ type: 'block-start', index: 0, blockType: 'text' },
{ type: 'text-delta', index: 0, text: 'kid' },
{ type: 'finish', reason: { kind: 'stop' } },
]
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8')
const childFile = join(dir, 'session.1.jsonl')
writeFileSync(childFile, sessionJsonl(childSecond.map((c, i) => chunkEvent(i + 1, 1, 1, c)), { id: 'c', createdAt: 2 }), 'utf8')
const ctx = new Context()
await ctx.plugin(LlmService)
apply(ctx, { file, childFiles: [childFile] })
const live = (id: string): GenerateOptions =>
({ model: 'm', messages: [], sessionId: id as NonNullable<GenerateOptions['sessionId']> })
expect(await drain(ctx.llm.stream(live('A')))).toEqual(TEXT_CHUNKS)
expect(await drain(ctx.llm.stream(live('B')))).toEqual(childSecond)
})
it('falls back to $DSH_SNAPSHOT_CHILD_FILES (path-delimited) when config omits childFiles', async () => {
const childChunks: StreamChunk[] = [
{ type: 'block-start', index: 0, blockType: 'text' },
{ type: 'text-delta', index: 0, text: 'env-kid' },
{ type: 'finish', reason: { kind: 'stop' } },
]
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8')
const childFile = join(dir, 'session.1.jsonl')
writeFileSync(childFile, sessionJsonl(childChunks.map((c, i) => chunkEvent(i + 1, 1, 1, c)), { id: 'c', createdAt: 2 }), 'utf8')
process.env.DSH_SNAPSHOT_FILE = file
process.env.DSH_SNAPSHOT_CHILD_FILES = childFile // single entry, no delimiter needed
const ctx = new Context()
await ctx.plugin(LlmService)
apply(ctx)
const live = (id: string): GenerateOptions =>
({ model: 'm', messages: [], sessionId: id as NonNullable<GenerateOptions['sessionId']> })
expect(await drain(ctx.llm.stream(live('A')))).toEqual(TEXT_CHUNKS)
expect(await drain(ctx.llm.stream(live('B')))).toEqual(childChunks)
})
it('ignores an empty $DSH_SNAPSHOT_CHILD_FILES (single-session)', async () => {
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8')
process.env.DSH_SNAPSHOT_FILE = file
process.env.DSH_SNAPSHOT_CHILD_FILES = ''
const ctx = new Context()
await ctx.plugin(LlmService)
apply(ctx)
expect(await drain(ctx.llm.stream({ model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
})
})