Merge remote-tracking branch 'origin/master' into session-query-search
# Conflicts: # .agents/notes/implemented/feature/2026-07-10-session-query-service.md # .agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md # .agents/notes/proposed/feature/2026-07-10-sqlite-session-query-provider.md # docs/architecture.md # docs/capability-seams.md # docs/config-catalog.md # docs/cordis-catalog/services.md # docs/core-data-structures/core.md # docs/core-data-structures/persistence.md # docs/core-data-structures/session-query.md # docs/module-graph.md # docs/rfc/INDEX.md # packages/README.md # packages/cordis/tool-cordis/src/api-catalog.ts # packages/hooks/hooks-claude/tests/coverage.spec.ts # packages/session-persistence/session-persistence-jsonl/src/index.ts # packages/session-persistence/session-persistence-jsonl/tests/jsonl.spec.ts # packages/session-persistence/session-persistence-sqlite/README.md # packages/session-persistence/session-persistence-sqlite/src/index.ts # packages/session-persistence/session-persistence-sqlite/src/schema.ts # packages/session-persistence/session-persistence-sqlite/tests/sqlite.spec.ts # packages/session-persistence/session-persistence/README.md # packages/session-persistence/session-persistence/package.json # packages/session-query/README.md # packages/session-query/session-query/README.md # packages/session-query/session-query/package.json # packages/session-query/session-query/src/config.ts # packages/session-query/session-query/src/index.ts # packages/session-query/session-query/src/types.ts # pnpm-lock.yaml # scripts/gen-doc-graphs.ts # scripts/type-equiv.manifest.json # tsconfig.host.json # tsconfig.json
This commit is contained in:
@@ -5,7 +5,8 @@ The durable session-persistence seam and its storage backends. The interface pac
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
| `session-persistence/` | Persistence seam + shared write coordinator | `ctx.sessionPersistence` |
|
||||
| `session-checkpoint-policy/` | Semantic durability barriers for agent requests and tool execution | (wraps `ctx.llm` / `ctx.tools`, listens on agent events) |
|
||||
| `session-persistence-jsonl/` | JSONL-sidecar persistence backend | (registers `ctx.sessionPersistence`) |
|
||||
| `session-persistence-sqlite/` | SQLite persistence backend | (registers `ctx.sessionPersistence`) |
|
||||
|
||||
The interface lives at `session-persistence/session-persistence/`; backends are flat siblings. A new storage backend joins here and registers on `ctx.sessionPersistence`. See [session persistence](../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md).
|
||||
The interface lives at `session-persistence/session-persistence/`; backends are flat siblings. A new storage backend joins here and registers on `ctx.sessionPersistence`. See [session persistence](../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md).
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# dsh-session-checkpoint-policy
|
||||
|
||||
Semantic durability policy for persisted agents. It checkpoints the event-sourced session before a model adapter receives a request, before a top-level tool body may produce an external side effect, and after a step has recorded its complete assistant message and ordered tool results. The final `turn/end` checkpoint remains owned by `dsh-agent-loop`.
|
||||
|
||||
## Plugin (namespace: `session-checkpoint-policy`)
|
||||
|
||||
This zero-config function plugin consumes `ctx.sessions`, `ctx.llm`, `ctx.tools`, and the presence of `ctx.sessionPersistence`. Load it beside one persistence backend:
|
||||
|
||||
```yaml
|
||||
- id: session-persistence
|
||||
name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
|
||||
- id: session-checkpoints
|
||||
name: '@deepseek-ai/dsh-session-checkpoint-policy'
|
||||
```
|
||||
|
||||
Persistence and checkpoint scheduling are intentionally separate Cordis plugins. A persistence backend makes each requested `session/flush` durable; this policy chooses the request, tool-dispatch, and completed-step checkpoints. Loading a backend without this policy is valid and retains checkpoints requested by the loop, including final `turn/end`, but crash recovery may lose the rest of an in-flight turn. First-party persisted apps and runtimes mount both plugins explicitly; a specialized deployment may deliberately omit or replace the policy.
|
||||
|
||||
The policy wraps `llm/stream` lazily, so the downstream stream is not constructed until the live session's buffered request events are durable. It wraps `tools/execute` after pre-execute policy and guards; a top-level tool body runs only after its recorded call is durable. If cancellation lands while that flush is pending, the wrapper returns the canonical `ABORTED_BEFORE_DISPATCH` result without entering the tool body. Nested tool dispatches reuse the outer model-visible call's checkpoint. `agent/post-step` persists the complete response/result batch before continuation work.
|
||||
|
||||
The loop records its assistant message and ordered tool results before dispatching `agent/post-step`, so the policy always captures that core batch. An event appended by another `agent/post-step` listener is captured at this checkpoint only when that listener is registered before the policy; Cordis registration order is the explicit composition rule for such extensions.
|
||||
|
||||
Checkpoint rejection is fail-closed at the model and tool boundaries: neither the adapter nor the top-level tool body runs. A post-step rejection fails the turn before another request starts. Concurrent tool checkpoints share the session store's serialized persistence drain and cannot duplicate sequence numbers.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Interrupted calls
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The plugin adds no prompt or tool schema. A hard crash after a tool checkpoint but before its result leaves a durable unmatched call; session recovery supplies the model-visible `TOOL_OUTCOME_UNKNOWN` result owned by `dsh-session`. The message permits retry for read-only or idempotent work and requires state verification or user confirmation for calls that may have side effects.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Successful checkpoints add no tokens and do not change the request. Recovery adds one short tool-result message to balance the interrupted transcript.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
The repair result is appended after the reusable prefix, so it does not invalidate earlier cache entries.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- The policy durably records execution intent, not generic exactly-once effects. Side-effecting tools should forward `exec.callId` as an idempotency key when their provider supports one.
|
||||
- Streaming `assistant/chunk` events have no per-chunk checkpoint. They reach storage with the next semantic checkpoint, so a hard crash may lose the current partial response.
|
||||
- A persisted call without a result cannot prove whether its external effect completed. Recovery therefore records an unknown outcome instead of retrying automatically.
|
||||
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-session-checkpoint-policy",
|
||||
"description": "Semantic session durability checkpoints before model requests and tool side effects",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cordisjs/plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop-testkit": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
/**
|
||||
* Semantic durability checkpoints for model requests, top-level tool dispatch,
|
||||
* and completed agent steps.
|
||||
* @module @deepseek-ai/dsh-session-checkpoint-policy
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { TOOL_ABORTED_BEFORE_DISPATCH, type ToolExecutionResult } from '@deepseek-ai/dsh-tools'
|
||||
import type {} from '@deepseek-ai/dsh-agent'
|
||||
import type {} from '@deepseek-ai/dsh-session-persistence'
|
||||
|
||||
/** Cordis plugin name used by Loader diagnostics. */
|
||||
export const name = 'session-checkpoint-policy'
|
||||
|
||||
/** Services whose request, tool, session, and persistence boundaries this policy joins. */
|
||||
export const inject = ['llm', 'sessionPersistence', 'sessions', 'tools']
|
||||
|
||||
/**
|
||||
* Delay construction of the downstream model stream until the complete logged
|
||||
* request prefix is durable. A checkpoint rejection prevents adapter dispatch.
|
||||
*
|
||||
* @param ctx - plugin context that owns the session store.
|
||||
* @param session - live session named by the model request.
|
||||
* @param next - downstream `llm/stream` chain.
|
||||
* @returns a stream that checkpoints before requesting its first chunk.
|
||||
*/
|
||||
function afterCheckpoint(
|
||||
ctx: Context,
|
||||
session: Session,
|
||||
next: () => AsyncIterable<StreamChunk>,
|
||||
): AsyncIterable<StreamChunk> {
|
||||
return (async function* (): AsyncIterable<StreamChunk> {
|
||||
await ctx.sessions.flush(session)
|
||||
yield* next()
|
||||
})()
|
||||
}
|
||||
|
||||
/** Materialize the canonical result for a call cancelled before tool dispatch. */
|
||||
function abortedBeforeDispatchResult(): ToolExecutionResult {
|
||||
return {
|
||||
content: [{ type: 'text', text: 'Error: tool call aborted before dispatch' }],
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: TOOL_ABORTED_BEFORE_DISPATCH },
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Install semantic checkpoint listeners. Loop-built model calls checkpoint the
|
||||
* logged request before adapter dispatch; top-level tool calls checkpoint their
|
||||
* recorded call before the tool body; post-step checkpoints retain the complete
|
||||
* response/result batch. Nested tool dispatches reuse the durable outer call.
|
||||
*
|
||||
* Checkpoint failures are fail-closed at the model and tool side-effect
|
||||
* boundaries: the downstream adapter or tool body is not invoked.
|
||||
*
|
||||
* @param ctx - plugin context that owns the listeners.
|
||||
*/
|
||||
export function apply(ctx: Context): void {
|
||||
ctx.on('llm/stream', (options, next): AsyncIterable<StreamChunk> => {
|
||||
if (options.sessionId === undefined) return next()
|
||||
const session = ctx.sessions.get(options.sessionId)
|
||||
return session === undefined ? next() : afterCheckpoint(ctx, session, next)
|
||||
})
|
||||
|
||||
ctx.on('tools/execute', async (exec, next): Promise<ToolExecutionResult> => {
|
||||
if (exec.agent === undefined || exec.parent !== undefined) return next()
|
||||
await ctx.sessions.flush(exec.agent.session)
|
||||
if (exec.signal.aborted) return abortedBeforeDispatchResult()
|
||||
return next()
|
||||
})
|
||||
|
||||
ctx.on('agent/post-step', (agent): Promise<void> => ctx.sessions.flush(agent.session))
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-checkpoint-policy`.
|
||||
* @module @deepseek-ai/dsh-session-checkpoint-policy/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-checkpoint-policy'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-checkpoint-policy-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: checkpoint ordering is enforced at the intercepted waterfall and
|
||||
* persistence seams; this stateless policy owns no independent mutable relation.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -0,0 +1,106 @@
|
||||
import { spawn } from 'node:child_process'
|
||||
import { access, mkdtemp, readFile, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { Context } from 'cordis'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import SessionStore, {
|
||||
SessionId, TOOL_OUTCOME_UNKNOWN,
|
||||
type SessionEvent,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
|
||||
const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url))
|
||||
const childScript = fileURLToPath(new URL('./fixtures/crash-child.ts', import.meta.url))
|
||||
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
|
||||
const sessionId = SessionId('semantic-checkpoint-crash')
|
||||
const roots: string[] = []
|
||||
const CHILD_FAILPOINT_TIMEOUT_MS = 30_000
|
||||
|
||||
async function waitForFile(path: string): Promise<void> {
|
||||
const deadline = Date.now() + CHILD_FAILPOINT_TIMEOUT_MS
|
||||
for (;;) {
|
||||
try {
|
||||
await access(path)
|
||||
return
|
||||
} catch (error: unknown) {
|
||||
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
|
||||
}
|
||||
if (Date.now() >= deadline) throw new Error(`crash child did not reach failpoint ${path}`)
|
||||
await new Promise(resolve => setTimeout(resolve, 10))
|
||||
}
|
||||
}
|
||||
|
||||
async function crashAt(mode: 'request' | 'tool'): Promise<{ root: string; markerText: string }> {
|
||||
const root = await mkdtemp(join(tmpdir(), `dsh-semantic-${mode}-`))
|
||||
roots.push(root)
|
||||
const marker = join(root, 'failpoint')
|
||||
const child = spawn(process.execPath, ['--import', tsxLoader, childScript, mode, root, marker], {
|
||||
cwd: repoRoot,
|
||||
env: { ...process.env, TSX_TSCONFIG_PATH: join(repoRoot, 'tsconfig.json') },
|
||||
stdio: ['ignore', 'ignore', 'pipe'],
|
||||
})
|
||||
let stderr = ''
|
||||
child.stderr.setEncoding('utf8')
|
||||
child.stderr.on('data', (chunk: string) => { stderr += chunk })
|
||||
try {
|
||||
await waitForFile(marker)
|
||||
const markerText = await readFile(marker, 'utf8')
|
||||
const closed = new Promise<{ code: number | null; signal: NodeJS.Signals | null }>((resolve) => {
|
||||
child.once('close', (code, signal) => { resolve({ code, signal }) })
|
||||
})
|
||||
child.kill('SIGKILL')
|
||||
const exit = await closed
|
||||
expect(exit).toEqual({ code: null, signal: 'SIGKILL' })
|
||||
return { root, markerText }
|
||||
} catch (error: unknown) {
|
||||
if (child.exitCode === null && child.signalCode === null) child.kill('SIGKILL')
|
||||
throw new Error(`crash child failed: ${stderr}`, { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
async function load(root: string): Promise<SessionEvent[]> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
try {
|
||||
return (await ctx.sessionPersistence.load(sessionId)).events
|
||||
} finally {
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe.skipIf(process.platform === 'win32')('semantic checkpoint hard-crash recovery', () => {
|
||||
it('persists the complete request before model dispatch', async () => {
|
||||
const crashed = await crashAt('request')
|
||||
expect(crashed.markerText).toBe('request-dispatched')
|
||||
const events = await load(crashed.root)
|
||||
expect(events.map(event => event.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'request/header', 'step/end', 'turn/end',
|
||||
])
|
||||
expect(events.at(-1)).toMatchObject({
|
||||
type: 'turn/end', data: { reason: { kind: 'interrupted' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('persists tool intent before a side effect and repairs its missing result as unknown', async () => {
|
||||
const crashed = await crashAt('tool')
|
||||
expect(crashed.markerText).toBe('tool-side-effect')
|
||||
const events = await load(crashed.root)
|
||||
expect(events.some(event => event.type === 'assistant/message')).toBe(true)
|
||||
expect(events.some(event => event.type === 'tool/call')).toBe(true)
|
||||
const result = events.find(event => event.type === 'tool/result')
|
||||
expect(result?.type === 'tool/result' && result.data.error).toEqual({
|
||||
name: 'ToolOutcomeUnknownError', code: TOOL_OUTCOME_UNKNOWN,
|
||||
})
|
||||
if (result?.type !== 'tool/result' || result.data.content[0]?.type !== 'text') {
|
||||
throw new Error('expected a text tool result')
|
||||
}
|
||||
expect(result.data.content[0].text).toContain('Do not retry blindly.')
|
||||
})
|
||||
})
|
||||
59
packages/session-persistence/session-checkpoint-policy/tests/fixtures/crash-child.ts
vendored
Normal file
59
packages/session-persistence/session-checkpoint-policy/tests/fixtures/crash-child.ts
vendored
Normal file
@@ -0,0 +1,59 @@
|
||||
import { writeFile } from 'node:fs/promises'
|
||||
import { Context } from 'cordis'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit'
|
||||
import { CallId, type GenerateOptions, LlmAdapter, type StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import * as checkpointPolicy from '../../src/index.ts'
|
||||
|
||||
function waitForCrash(): Promise<never> {
|
||||
return new Promise(() => { setInterval(() => {}, 60_000) })
|
||||
}
|
||||
|
||||
const [mode, root, marker] = process.argv.slice(2)
|
||||
if ((mode !== 'request' && mode !== 'tool') || root === undefined || marker === undefined) {
|
||||
throw new Error('usage: crash-child.ts <request|tool> <persistence-root> <marker>')
|
||||
}
|
||||
const persistenceRoot = root
|
||||
const failpoint = marker
|
||||
|
||||
class CrashAdapter extends LlmAdapter {
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
if (mode === 'request') {
|
||||
await writeFile(failpoint, 'request-dispatched')
|
||||
await waitForCrash()
|
||||
return
|
||||
}
|
||||
yield { type: 'block-start', index: 0, blockType: 'tool-call' }
|
||||
yield {
|
||||
type: 'block-end',
|
||||
index: 0,
|
||||
block: { type: 'tool-call', id: CallId('crash-call'), name: 'crash_tool', arguments: '{}' },
|
||||
}
|
||||
yield { type: 'finish', reason: { kind: 'tool-calls' } }
|
||||
}
|
||||
}
|
||||
|
||||
const ctx = new Context()
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root: persistenceRoot, compression: 'none' })
|
||||
await ctx.plugin(checkpointPolicy)
|
||||
ctx.llm.registerAdapter(['crash'], new CrashAdapter())
|
||||
ctx.tools.register({
|
||||
name: 'crash_tool',
|
||||
description: 'records an external effect and never returns',
|
||||
parameters: {},
|
||||
async execute() {
|
||||
await writeFile(failpoint, 'tool-side-effect')
|
||||
return waitForCrash()
|
||||
},
|
||||
})
|
||||
|
||||
const handle = await ctx.agents.create({
|
||||
sessionId: SessionId('semantic-checkpoint-crash'),
|
||||
agentOptions: { provider: 'crash', model: 'crash' },
|
||||
})
|
||||
handle.agent.send([{ type: 'text', text: 'exercise the crash boundary' }])
|
||||
await waitForCrash()
|
||||
@@ -0,0 +1,250 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import { agentEvents, type Agent } from '@deepseek-ai/dsh-agent'
|
||||
import LlmService, { CallId, type GenerateOptions, LlmAdapter, type StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistence from '@deepseek-ai/dsh-session-persistence'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH } from '@deepseek-ai/dsh-tools'
|
||||
import * as checkpointPolicy from '../src/index.ts'
|
||||
|
||||
const contexts: Context[] = []
|
||||
|
||||
class TestPersistence extends SessionPersistence {
|
||||
locate(_meta: SessionHeader): undefined { return undefined }
|
||||
create(_meta: SessionHeader): Promise<void> { return Promise.resolve() }
|
||||
append(_id: SessionId, _events: readonly SessionEvent[]): Promise<void> { return Promise.resolve() }
|
||||
load(_id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return Promise.reject(new Error('not used'))
|
||||
}
|
||||
list(): Promise<SessionHeader[]> { return Promise.resolve([]) }
|
||||
listSnapshots(): Promise<never[]> { return Promise.resolve([]) }
|
||||
}
|
||||
|
||||
class RecordingAdapter extends LlmAdapter {
|
||||
constructor(private readonly order: string[]) { super() }
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
this.order.push('adapter')
|
||||
yield { type: 'finish', reason: { kind: 'stop' } }
|
||||
}
|
||||
}
|
||||
|
||||
async function setup(): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(TestPersistence)
|
||||
await ctx.plugin(checkpointPolicy)
|
||||
return ctx
|
||||
}
|
||||
|
||||
async function drain(stream: AsyncIterable<StreamChunk>): Promise<void> {
|
||||
for await (const _chunk of stream) { /* drain */ }
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose()))
|
||||
})
|
||||
|
||||
describe('session-checkpoint-policy request boundary', () => {
|
||||
it('awaits the live session checkpoint before constructing the downstream model stream', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('request-checkpoint'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
const gate = Promise.withResolvers<undefined>()
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', async () => {
|
||||
order.push('flush:start')
|
||||
await gate.promise
|
||||
order.push('flush:end')
|
||||
})
|
||||
ctx.llm.registerAdapter(['mock'], new RecordingAdapter(order))
|
||||
|
||||
const pending = drain(ctx.llm.stream({
|
||||
provider: 'mock', model: 'mock', messages: [], sessionId: session.id,
|
||||
}))
|
||||
await Promise.resolve()
|
||||
expect(order).toEqual(['flush:start'])
|
||||
gate.resolve(undefined)
|
||||
await pending
|
||||
expect(order).toEqual(['flush:start', 'flush:end', 'adapter'])
|
||||
})
|
||||
|
||||
it('delegates a request without a live session without checkpointing', async () => {
|
||||
const ctx = await setup()
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', () => { order.push('flush') })
|
||||
ctx.llm.registerAdapter(['mock'], new RecordingAdapter(order))
|
||||
await drain(ctx.llm.stream({ provider: 'mock', model: 'mock', messages: [] }))
|
||||
expect(order).toEqual(['adapter'])
|
||||
})
|
||||
|
||||
it('delegates an already-detached session id without checkpointing', async () => {
|
||||
const ctx = await setup()
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', () => { order.push('flush') })
|
||||
ctx.llm.registerAdapter(['mock'], new RecordingAdapter(order))
|
||||
await drain(ctx.llm.stream({
|
||||
provider: 'mock', model: 'mock', messages: [], sessionId: SessionId('detached'),
|
||||
}))
|
||||
expect(order).toEqual(['adapter'])
|
||||
})
|
||||
|
||||
it('does not dispatch the adapter when the checkpoint rejects', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('request-failure'))
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', () => Promise.reject(new Error('disk unavailable')))
|
||||
ctx.llm.registerAdapter(['mock'], new RecordingAdapter(order))
|
||||
await expect(drain(ctx.llm.stream({
|
||||
provider: 'mock', model: 'mock', messages: [], sessionId: session.id,
|
||||
}))).rejects.toThrow('disk unavailable')
|
||||
expect(order).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('session-checkpoint-policy tool and step boundaries', () => {
|
||||
it('awaits the checkpoint before a top-level tool body', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('tool-checkpoint'))
|
||||
const agent = { session } as Agent
|
||||
const gate = Promise.withResolvers<undefined>()
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', async () => {
|
||||
order.push('flush:start')
|
||||
await gate.promise
|
||||
order.push('flush:end')
|
||||
})
|
||||
ctx.tools.register({
|
||||
name: 'write', description: 'side effect', parameters: {},
|
||||
execute: async () => { order.push('tool'); return [] },
|
||||
})
|
||||
|
||||
const pending = ctx.tools.execute({
|
||||
callId: CallId('write-1'), name: 'write', arguments: {}, agent,
|
||||
signal: new AbortController().signal,
|
||||
})
|
||||
await Promise.resolve()
|
||||
expect(order).toEqual(['flush:start'])
|
||||
gate.resolve(undefined)
|
||||
await expect(pending).resolves.toMatchObject({ isError: false })
|
||||
expect(order).toEqual(['flush:start', 'flush:end', 'tool'])
|
||||
})
|
||||
|
||||
it('does not dispatch when cancellation lands during the tool checkpoint', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('tool-checkpoint-cancel'))
|
||||
const agent = { session } as Agent
|
||||
const controller = new AbortController()
|
||||
const gate = Promise.withResolvers<undefined>()
|
||||
const order: string[] = []
|
||||
ctx.on('session/flush', async () => {
|
||||
order.push('flush:start')
|
||||
await gate.promise
|
||||
order.push('flush:end')
|
||||
})
|
||||
ctx.tools.register({
|
||||
name: 'write', description: 'side effect', parameters: {},
|
||||
execute: async () => { order.push('tool'); return [] },
|
||||
})
|
||||
|
||||
const pending = ctx.tools.execute({
|
||||
callId: CallId('write-cancelled'), name: 'write', arguments: {}, agent,
|
||||
signal: controller.signal,
|
||||
})
|
||||
await Promise.resolve()
|
||||
expect(order).toEqual(['flush:start'])
|
||||
controller.abort('cancelled during checkpoint')
|
||||
gate.resolve(undefined)
|
||||
|
||||
await expect(pending).resolves.toEqual({
|
||||
content: [{ type: 'text', text: 'Error: tool call aborted before dispatch' }],
|
||||
isError: true,
|
||||
error: { name: 'AbortError', code: TOOL_ABORTED_BEFORE_DISPATCH },
|
||||
})
|
||||
expect(order).toEqual(['flush:start', 'flush:end'])
|
||||
})
|
||||
|
||||
it('turns a rejected checkpoint into an error result without running the tool body', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('tool-failure'))
|
||||
const agent = { session } as Agent
|
||||
let ran = false
|
||||
ctx.on('session/flush', () => Promise.reject(new Error('disk unavailable')))
|
||||
ctx.tools.register({
|
||||
name: 'write', description: 'side effect', parameters: {},
|
||||
execute: async () => { ran = true; return [] },
|
||||
})
|
||||
const result = await ctx.tools.execute({
|
||||
callId: CallId('write-2'), name: 'write', arguments: {}, agent,
|
||||
signal: new AbortController().signal,
|
||||
})
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'Error: disk unavailable' }])
|
||||
expect(ran).toBe(false)
|
||||
})
|
||||
|
||||
it('reuses the outer checkpoint for a nested tool dispatch', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('nested-tool'))
|
||||
const agent = { session } as Agent
|
||||
let flushes = 0
|
||||
ctx.on('session/flush', () => { flushes += 1 })
|
||||
ctx.tools.register({ name: 'nested', description: 'nested', parameters: {}, execute: async () => [] })
|
||||
await ctx.tools.execute({
|
||||
callId: CallId('nested-1'), name: 'nested', arguments: {}, agent,
|
||||
parent: Symbol('outer') as never,
|
||||
signal: new AbortController().signal,
|
||||
})
|
||||
expect(flushes).toBe(0)
|
||||
})
|
||||
|
||||
it('checkpoints the complete recorded step at agent/post-step', async () => {
|
||||
const ctx = await setup()
|
||||
const session = ctx.sessions.create(SessionId('post-step'))
|
||||
const agent = { session } as Agent
|
||||
const flushed: string[] = []
|
||||
ctx.on('session/flush', (current) => { flushed.push(current.id) })
|
||||
await agentEvents(ctx, agent).serial(
|
||||
'agent/post-step', 1, 1, new AbortController().signal,
|
||||
)
|
||||
expect(flushed).toEqual([session.id])
|
||||
})
|
||||
})
|
||||
|
||||
describe('session-checkpoint-policy lifecycle', () => {
|
||||
it('removes its wrappers when the owning fiber is disposed', async () => {
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(TestPersistence)
|
||||
const session = ctx.sessions.create(SessionId('disposed-policy'))
|
||||
let flushes = 0
|
||||
ctx.on('session/flush', () => { flushes += 1 })
|
||||
ctx.llm.registerAdapter(['mock'], new RecordingAdapter([]))
|
||||
const fiber = await ctx.plugin(checkpointPolicy)
|
||||
await drain(ctx.llm.stream({ provider: 'mock', model: 'mock', messages: [], sessionId: session.id }))
|
||||
expect(flushes).toBe(1)
|
||||
await fiber.dispose()
|
||||
await drain(ctx.llm.stream({ provider: 'mock', model: 'mock', messages: [], sessionId: session.id }))
|
||||
expect(flushes).toBe(1)
|
||||
})
|
||||
|
||||
it('keeps the Loader-safe namespace plugin shape', () => {
|
||||
expect('default' in checkpointPolicy).toBe(false)
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(checkpointPolicy) as Record<string, unknown>
|
||||
expect(unwrapped).toBe(checkpointPolicy)
|
||||
expect(unwrapped.name).toBe('session-checkpoint-policy')
|
||||
expect(unwrapped.inject).toEqual(['llm', 'sessionPersistence', 'sessions', 'tools'])
|
||||
expect(typeof unwrapped.apply).toBe('function')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../session-persistence/session-persistence"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,29 +1,41 @@
|
||||
# @deepseek-ai/dsh-session-persistence-jsonl
|
||||
|
||||
The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). One append-only `.jsonl` event log per session.
|
||||
The JSONL durable session-persistence backend — a concrete `SessionPersistence` (the `dsh-session-persistence` seam). Each session has one append-only logical JSONL log, stored as `.jsonl.zstd` by default or raw `.jsonl` when compression is disabled.
|
||||
|
||||
## On-disk layout
|
||||
|
||||
```
|
||||
<root>/
|
||||
cwd-<sha256(cwd)[:12]>/ # per-project bucket (or _no-cwd/ when no cwd)
|
||||
<encoded-id>.jsonl # header line + one SessionEvent per line (verbatim)
|
||||
<encoded-id>.jsonl.zstd # default: checksummed header frame + append frames
|
||||
<encoded-id>.jsonl # only with compression: 'none'
|
||||
```
|
||||
|
||||
- The first `.jsonl` line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength? }`; every subsequent line is one `SessionEvent` JSON, **verbatim including `assistant/chunk`** so `seq` stays contiguous (`events[i].seq === i`).
|
||||
- Session ids are unvalidated branded strings, so they are percent-encoded to a single safe path segment before use (no traversal, no collision).
|
||||
- The first logical line is the immutable `SessionHeader` tagged `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength?, delegationDepth }`. `delegationDepth` is required on disk and is `0` for a top-level session; a missing or invalid value rejects the log. Every subsequent logical line is one storage record; `assistant/chunk` events are never dropped, and `seq` stays contiguous across the decoded log (`events[i].seq === i`).
|
||||
- A storage record is a `SessionEvent` JSON verbatim, or — written only under `packChunks` — a **packed chunk row** (`text-chunks` / `reasoning-chunks` / `tool-call-chunks`; bare slash-less tags like the header's `session`, so row tags cannot be confused with event types): one line holding a run of ≥3 consecutive same-block `assistant/chunk` delta events, `seq0`/`time0` plus per-member `dt` gaps reconstructing every member's `seq`/`time` exactly. The lossless codec lives in `@deepseek-ai/dsh-session` (`packChunkRuns`/`decodeStorageRecord`) and whitelists exact shapes — anything unrecognized stores verbatim. Reading is layout-blind: `load` always decodes rows, so packed, unpacked, and mixed files load identically.
|
||||
- Session ids are unvalidated branded strings, so they are injectively escaped to a single safe path segment before use (no traversal, no collision).
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Type | Notes |
|
||||
|---|---|---|
|
||||
| `root` | `string` (required) | Root directory for all session files. **No default** — a `process.cwd()` default would scatter files as the process's cwd changes (bash calls, subprocesses). |
|
||||
| `packChunks` | `boolean` (default `false`) | Write delta-chunk runs as packed rows (~60% smaller logical logs measured on a real coding session). Off, the written logical layout is byte-identical to the pre-packing format; reading packed rows works regardless of this switch. Off by default while the snapshot goldens stay one-event-per-line — recording with packing on rewrites every fixture `session.jsonl`. |
|
||||
| `compression` | `'zstd' \| 'none'` | Defaults to `'zstd'`; `'none'` retains newline-delimited UTF-8 text. |
|
||||
|
||||
`locate(meta)` returns `{ kind: 'jsonl', path }` using the resolved absolute root and the same cwd-bucket/id encoding as materialization. It performs no filesystem I/O: the target can be returned before the file exists, and an existing file contains only the last flushed prefix.
|
||||
|
||||
## Physical encoding
|
||||
|
||||
The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, followed by one checksummed frame per durable append batch. The backend uses Node's built-in Zstandard API with its default compression level and exposes no level knob. Listing reads and validates only the header frame. `compression: 'none'` keeps the same logical lines in the original raw representation.
|
||||
|
||||
A root belongs to one encoding. Startup discovery and targeted lookup reject the opposite suffix with an error naming the incompatible artifact and instructing the caller to select the matching mode or a separate root. There is no migration, mixed-root fallback, or dual write.
|
||||
|
||||
## Durability and crash semantics
|
||||
|
||||
- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s a temporary file, publishes it without overwrite via a hard link, then `fsync`s the directory. A created-but-never-appended session leaves nothing on disk and is absent from `list`.
|
||||
- **Append-only.** Committed events (at or below a flushed `turn/end`) are never rewritten. Subsequent appends are line appends at EOF + `fsync`.
|
||||
- **Crash recovery — preserve valid tail work.** `load` keeps the contiguous valid prefix of an interrupted final turn. It truncates from the first unparsable or sequence-gapped uncommitted record, then appends the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md); the same defect at or before the last committed `turn/end` rejects.
|
||||
- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s the encoded header and first batch in a temporary file. POSIX publishes it without overwrite via a hard link and `fsync`s the parent directory. Windows publishes it without overwrite via `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` and creates missing directories through the same write-through pattern. A created-but-never-appended session leaves nothing on disk and is absent from `list`.
|
||||
- **Append-only.** Flushed events are never rewritten. Subsequent raw batches append lines; compressed batches append one frame. Both paths `fsync`, and a caught write or sync failure rolls the file back to its prior byte length.
|
||||
- **Crash recovery — preserve valid tail work.** `load` validates every complete compressed frame and scans their decompressed JSONL. If the last frame is structurally incomplete, the reader keeps its complete decoded records, truncates from that frame's start, and re-encodes those records with the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md). Raw mode truncates from its first incomplete line. A checksum/decompression failure in a complete frame, or a defect at or before the last committed `turn/end`, is corruption and rejects.
|
||||
- **Contiguous-seq.** `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type.
|
||||
- **Lightweight revisions.** `listSnapshots()` identifies a log by its device, inode, size, and nanosecond timestamps, avoiding a full-log parse while changing after append, repair, replacement, or store changes.
|
||||
|
||||
@@ -35,13 +47,22 @@ The plugin buffers frozen session events and drains them on flush or disposal. A
|
||||
|
||||
### Resumed conversation history
|
||||
|
||||
**What the model sees**: JSONL storage contributes no live prompt or schema. Loading restores stored surface history and preserves prior request headers for reconstruction; the new loop composes its current envelope. Each unanswered call in an interrupted tail is balanced with the exact error text `Tool call interrupted by a crash; no result was recorded.` Raw `assistant/chunk` records do not duplicate messages.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Zero live-request tokens. A resumed agent pays for retained history and its current envelope, plus the quoted repair result for each interrupted call.
|
||||
JSONL storage contributes no live prompt or schema. Loading restores stored surface history and preserves prior request headers for reconstruction; the new loop composes its current envelope. Recovery balances an assistant request without a durable call with `TOOL_NOT_STARTED`; a durable call without a result becomes `TOOL_OUTCOME_UNKNOWN`, which tells the model to retry only read-only or idempotent work and to verify possible side effects or ask the user. Raw `assistant/chunk` records do not duplicate messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Zero live-request tokens. A resumed agent pays for retained history and its current envelope, plus the quoted repair result for each interrupted call.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
JSONL storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Only the current `SESSION_FORMAT_VERSION` (v0) loads** — the on-disk format is pre-release/unstable: a breaking format change is absorbed at v0 and non-current logs are rejected; there is no migration.
|
||||
- **Only the configured encoding and current `SESSION_FORMAT_VERSION` (v0) load** — changing compression requires a separate/fresh root or selecting the legacy raw mode; the pre-release format has no migration.
|
||||
- **Compressed files are not directly line-readable** — use the backend to load them, or select `compression: 'none'` before writing a fresh root when text fixtures or external line readers are required.
|
||||
- **Nothing deletes session files** — logs accumulate under `root` until removed externally (the seam has no deletion surface).
|
||||
- **Single-process assumption** — per-session serialization and the write cursor live in this process; two processes appending to the same `root` are not coordinated.
|
||||
- **Initial materialization requires hard-link support** — first append uses `link()` so same-id races fail instead of overwriting a committed log; a filesystem that cannot create hard links cannot host this backend.
|
||||
- **POSIX materialization requires hard-link support** — first append uses `link()` so same-id races fail instead of overwriting a committed log; Windows uses write-through rename without replacement.
|
||||
|
||||
@@ -11,25 +11,33 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"koffi": "^3.1.0",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
|
||||
@@ -10,10 +10,23 @@
|
||||
|
||||
import { createHash } from 'node:crypto'
|
||||
import { join } from 'node:path'
|
||||
import type { SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import { decodeStorageRecord, packChunkRuns } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader, SessionId, StorageRecord } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/** Physical encoding selected for JSONL session artifacts. */
|
||||
export type JsonlCompression = 'zstd' | 'none'
|
||||
|
||||
/**
|
||||
* The first line of a session's `.jsonl` file: the immutable
|
||||
* Return the artifact suffix for one physical encoding.
|
||||
* @param compression - configured JSONL artifact encoding.
|
||||
* @returns `.jsonl.zstd` for Zstandard or `.jsonl` for plaintext.
|
||||
*/
|
||||
export function logSuffix(compression: JsonlCompression): '.jsonl.zstd' | '.jsonl' {
|
||||
return compression === 'zstd' ? '.jsonl.zstd' : '.jsonl'
|
||||
}
|
||||
|
||||
/**
|
||||
* The first JSONL record of a session artifact: the immutable
|
||||
* {@link SessionHeader} tagged as a `session` record so a reader can tell it
|
||||
* apart from an event line.
|
||||
*/
|
||||
@@ -25,6 +38,7 @@ export interface HeaderLine {
|
||||
cwd?: string
|
||||
parentSession?: SessionId
|
||||
seedLength?: number
|
||||
delegationDepth: number
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -41,6 +55,7 @@ export function toHeaderLine(header: SessionHeader): HeaderLine {
|
||||
...header.cwd !== undefined ? { cwd: header.cwd } : {},
|
||||
...header.parentSession !== undefined ? { parentSession: header.parentSession } : {},
|
||||
...header.seedLength !== undefined ? { seedLength: header.seedLength } : {},
|
||||
delegationDepth: header.delegationDepth ?? 0,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -57,6 +72,7 @@ export function fromHeaderLine(line: HeaderLine): SessionHeader {
|
||||
...line.cwd !== undefined ? { cwd: line.cwd } : {},
|
||||
...line.parentSession !== undefined ? { parentSession: line.parentSession } : {},
|
||||
...line.seedLength !== undefined ? { seedLength: line.seedLength } : {},
|
||||
delegationDepth: line.delegationDepth,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -68,6 +84,10 @@ function isHeaderLine(value: unknown): value is HeaderLine {
|
||||
&& typeof (value as { version?: unknown }).version === 'number'
|
||||
&& typeof (value as { id?: unknown }).id === 'string'
|
||||
&& typeof (value as { createdAt?: unknown }).createdAt === 'number'
|
||||
&& typeof (value as { delegationDepth?: unknown }).delegationDepth === 'number'
|
||||
&& Number.isSafeInteger((value as { delegationDepth: number }).delegationDepth)
|
||||
&& (value as { delegationDepth: number }).delegationDepth >= 0
|
||||
&& !Object.is((value as { delegationDepth: number }).delegationDepth, -0)
|
||||
)
|
||||
}
|
||||
|
||||
@@ -119,24 +139,39 @@ export function sessionDir(root: string, cwd: string | undefined): string {
|
||||
* @param root - the backend's session root directory.
|
||||
* @param cwd - the session's project directory (picks the per-cwd bucket; `undefined` → `_no-cwd`).
|
||||
* @param id - the session id, path-encoded via {@link encodeSegment} before filesystem use.
|
||||
* @returns the session's `.jsonl` log file path.
|
||||
* @param compression - physical artifact encoding and filename suffix.
|
||||
* @returns the session's configured JSONL artifact path.
|
||||
*/
|
||||
export function logPath(root: string, cwd: string | undefined, id: SessionId): string {
|
||||
return join(sessionDir(root, cwd), `${encodeSegment(id)}.jsonl`)
|
||||
export function logPath(
|
||||
root: string,
|
||||
cwd: string | undefined,
|
||||
id: SessionId,
|
||||
compression: JsonlCompression,
|
||||
): string {
|
||||
return join(sessionDir(root, cwd), `${encodeSegment(id)}${logSuffix(compression)}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize one event as a JSONL line (no trailing newline).
|
||||
* @param event - the event to serialize verbatim.
|
||||
* @returns the event's single-line JSON text; the writer adds the newline.
|
||||
* Serialize an event batch as JSONL lines (no trailing newline). With
|
||||
* `packChunks` on, delta-chunk runs pack into `text-chunks` /
|
||||
* `reasoning-chunks` / `tool-call-chunks` storage rows; off writes one event
|
||||
* per line, byte-identical to the pre-packing layout. Reading is layout-blind
|
||||
* either way ({@link scanLog} always decodes rows), so the switch only shapes
|
||||
* NEW bytes.
|
||||
* @param events - the batch to serialize, in log order.
|
||||
* @param packChunks - whether to pack delta runs into storage rows.
|
||||
* @returns the batch's JSONL text; the writer adds the final newline.
|
||||
*/
|
||||
export function eventLine(event: SessionEvent): string {
|
||||
return JSON.stringify(event)
|
||||
export function eventLines(events: readonly SessionEvent[], packChunks: boolean): string {
|
||||
const records: readonly StorageRecord[] = packChunks ? packChunkRuns(events) : events
|
||||
return records.map(record => JSON.stringify(record)).join('\n')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a JSONL log buffer into its preserved event prefix (the header is line
|
||||
* 0). Fully written events in an interrupted final turn remain part of the
|
||||
* 0). Event lines pass through verbatim; packed chunk rows expand back into
|
||||
* their events, so callers see one contiguous event list regardless of layout.
|
||||
* Fully written events in an interrupted final turn remain part of the
|
||||
* prefix. The first unparsable record or seq gap after the last `turn/end`
|
||||
* marks a tolerated torn tail; the same hole in the committed region rejects.
|
||||
*
|
||||
@@ -175,46 +210,60 @@ export function scanLog(buffer: Buffer): { meta: SessionHeader; events: SessionE
|
||||
}
|
||||
const headerLine = parsedHeader
|
||||
|
||||
// Parse every complete record first so the last valid `turn/end` determines
|
||||
// whether an earlier hole is committed corruption or an uncommitted tail.
|
||||
interface Parsed { ok: boolean; event?: SessionEvent; endByte: number }
|
||||
// Parse and decode every complete line first so the last valid `turn/end`
|
||||
// determines whether an earlier hole is committed corruption or an
|
||||
// uncommitted tail. One line yields one event, or a whole run for a packed
|
||||
// chunk row; a row-tagged line that fails row validation is a hole, exactly
|
||||
// like unparsable JSON.
|
||||
interface Parsed { ok: boolean; events?: SessionEvent[]; endByte: number }
|
||||
const parsed: Parsed[] = eventEntries.map((entry) => {
|
||||
try {
|
||||
return { ok: true, event: JSON.parse(entry.text) as SessionEvent, endByte: entry.endByte }
|
||||
return { ok: true, events: decodeStorageRecord(JSON.parse(entry.text)), endByte: entry.endByte }
|
||||
} catch {
|
||||
return { ok: false, endByte: entry.endByte }
|
||||
}
|
||||
})
|
||||
|
||||
// The last index (into eventEntries) that is a valid `turn/end` — the last
|
||||
// fully-committed boundary (the loop flushes only at turn/end).
|
||||
// The last index (into eventEntries) that ends in a valid `turn/end` — the
|
||||
// last fully-committed boundary (the loop flushes only at turn/end). A packed
|
||||
// row never stores a turn/end, so only single-event lines can match.
|
||||
let lastTurnEnd = -1
|
||||
for (let i = parsed.length - 1; i >= 0; i--) {
|
||||
const p = parsed[i]
|
||||
if (p?.ok && p.event?.type === 'turn/end') { lastTurnEnd = i; break }
|
||||
if (p?.ok && p.events?.some(e => e.type === 'turn/end')) { lastTurnEnd = i; break }
|
||||
}
|
||||
|
||||
// Preserve the contiguous prefix, including a complete interrupted turn;
|
||||
// holes through the last committed boundary throw, while later holes stop.
|
||||
// Contiguity is a cursor over seqs (not the line index): a packed row
|
||||
// advances the cursor by its whole run.
|
||||
const preserved: SessionEvent[] = []
|
||||
for (let i = 0; i < parsed.length; i++) {
|
||||
let lastPreservedLine = -1
|
||||
scan: for (let i = 0; i < parsed.length; i++) {
|
||||
const p = parsed[i]
|
||||
if (!p?.ok || p.event === undefined) {
|
||||
if (!p?.ok || p.events === undefined) {
|
||||
if (i <= lastTurnEnd) throw new Error(`corrupt session log: unparsable committed event at line ${i + 1}`)
|
||||
break // torn tail fragment after the last turn/end — stop, tolerate
|
||||
}
|
||||
if (p.event.seq !== i) {
|
||||
if (i <= lastTurnEnd) throw new Error(`corrupt session log: seq gap in committed region at line ${i + 1} (expected ${i}, got ${p.event.seq})`)
|
||||
break // gap after the last turn/end — torn tail, stop
|
||||
for (const event of p.events) {
|
||||
if (event.seq !== preserved.length) {
|
||||
if (i <= lastTurnEnd) {
|
||||
throw new Error(`corrupt session log: seq gap in committed region at line ${i + 1} (expected ${preserved.length}, got ${event.seq})`)
|
||||
}
|
||||
break scan // gap after the last turn/end — torn tail, stop
|
||||
}
|
||||
preserved.push(event)
|
||||
}
|
||||
preserved.push(p.event)
|
||||
lastPreservedLine = i
|
||||
}
|
||||
|
||||
// committedBytes = end of the last PRESERVED line (header if none): the next
|
||||
// append truncates any torn bytes past this point before writing the
|
||||
// synthetic closers + new events.
|
||||
const lastPreserved = parsed[preserved.length - 1]
|
||||
const committedBytes = preserved.length > 0 && lastPreserved ? lastPreserved.endByte : headerEntry.endByte
|
||||
// committedBytes = end of the last FULLY preserved line (header if none): the
|
||||
// next append truncates any torn bytes past this point before writing the
|
||||
// synthetic closers + new events. A line is preserved whole or not at all —
|
||||
// a mid-row seq gap discards the whole row, keeping the truncation offset on
|
||||
// a line boundary.
|
||||
const lastPreserved = parsed[lastPreservedLine]
|
||||
const committedBytes = lastPreserved !== undefined ? lastPreserved.endByte : headerEntry.endByte
|
||||
return { meta: fromHeaderLine(headerLine), events: preserved, committedBytes }
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
/**
|
||||
* JSONL durable session-persistence backend. It stores a header and contiguous
|
||||
* events in one append-only file per session, and delegates orchestration to
|
||||
* {@link PersistenceCoordinator}.
|
||||
* {@link PersistenceCoordinator}. Its side-effect-free locator returns the
|
||||
* absolute per-session log target before materialization.
|
||||
* @module @deepseek-ai/dsh-session-persistence-jsonl
|
||||
*/
|
||||
|
||||
@@ -12,14 +13,28 @@ import { dirname, resolve } from 'node:path'
|
||||
import { randomBytes } from 'node:crypto'
|
||||
import {
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator,
|
||||
type PersistenceBackend, type SessionPersistenceSnapshot, type StoredPrefix,
|
||||
type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot,
|
||||
type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
encodeSegment, eventLine, logPath, parseHeaderMeta, scanLog, sessionDir, toHeaderLine,
|
||||
encodeSegment, eventLines, logPath, logSuffix, parseHeaderMeta, scanLog, sessionDir, toHeaderLine,
|
||||
type JsonlCompression,
|
||||
} from './format.ts'
|
||||
import { compressZstdFrame, decompressZstdFrame, scanZstdFrames } from './zstd.ts'
|
||||
import { ensureDurableDirectoryWin32, publishNewFileWin32 } from './win32.ts'
|
||||
|
||||
/** Plugin config: where the JSONL backend keeps its session logs (`root` is required — no default). */
|
||||
export type { JsonlCompression } from './format.ts'
|
||||
|
||||
const DEFAULT_COMPRESSION: JsonlCompression = 'zstd'
|
||||
|
||||
/** Loader schema for the JSONL artifact's physical encoding. */
|
||||
export const JsonlCompressionSchema: z<JsonlCompression> = z.union([
|
||||
z.const('zstd'),
|
||||
z.const('none'),
|
||||
]).default(DEFAULT_COMPRESSION)
|
||||
|
||||
/** Plugin config: where the JSONL backend keeps its session logs, and the packed-row write switch. */
|
||||
export interface Config {
|
||||
/**
|
||||
* Root directory for all session files. Required (no default): a default of
|
||||
@@ -27,6 +42,23 @@ export interface Config {
|
||||
* (bash calls, subprocesses). Sessions group under per-cwd subdirectories.
|
||||
*/
|
||||
root: string
|
||||
/**
|
||||
* Write runs of consecutive `assistant/chunk` delta events as packed
|
||||
* `text-chunks`/`reasoning-chunks`/`tool-call-chunks` rows (lossless,
|
||||
* ~60% smaller logs measured on a real session). Off by default while
|
||||
* snapshot fixtures stay in the one-event-per-line layout: recording with
|
||||
* packing on rewrites every golden `session.jsonl`. READING packed rows is
|
||||
* unconditional — a log's layout never depends on this switch.
|
||||
*/
|
||||
packChunks?: boolean
|
||||
/** Physical encoding; defaults to checksummed Zstandard frames. */
|
||||
compression?: JsonlCompression
|
||||
}
|
||||
|
||||
/** Opaque coordinator token for replacing bytes recovered from a torn frame. */
|
||||
interface JsonlTornMarker {
|
||||
truncateTo: number
|
||||
recoveredEvents: SessionEvent[]
|
||||
}
|
||||
|
||||
/** Whether a filesystem error means absence; every non-ENOENT failure must surface. */
|
||||
@@ -37,13 +69,16 @@ function isENOENT(error: unknown): boolean {
|
||||
/**
|
||||
* The JSONL persistence backend. Load as a plugin; it registers as
|
||||
* `ctx.sessionPersistence` and (via the coordinator) installs the write-path
|
||||
* listeners. Its torn-tail marker is the byte offset to truncate the log to.
|
||||
* listeners. Its torn-tail marker carries the byte offset and any events
|
||||
* recovered from an incomplete final Zstandard frame.
|
||||
*/
|
||||
export class SessionPersistenceJsonl extends SessionPersistence implements PersistenceBackend<number> {
|
||||
export class SessionPersistenceJsonl extends SessionPersistence implements PersistenceBackend<JsonlTornMarker> {
|
||||
static inject = ['sessions']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
root: z.string().required(),
|
||||
packChunks: z.boolean().default(false),
|
||||
compression: JsonlCompressionSchema,
|
||||
})
|
||||
|
||||
/**
|
||||
@@ -54,13 +89,20 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
override readonly name = 'session-persistence-jsonl'
|
||||
|
||||
private root: string
|
||||
private coordinator: PersistenceCoordinator<number>
|
||||
private packChunks: boolean
|
||||
private compression: JsonlCompression
|
||||
private coordinator: PersistenceCoordinator<JsonlTornMarker>
|
||||
private rootEncodingCheck: Promise<void> | undefined
|
||||
|
||||
constructor(ctx: Context, public config: Config) {
|
||||
super(ctx)
|
||||
// Resolve once so later process.cwd() changes cannot split one backend across roots.
|
||||
this.root = resolve(config.root)
|
||||
this.coordinator = new PersistenceCoordinator<number>(this.ctx, this)
|
||||
// schemastery (static Config) applied the default before construction;
|
||||
// the cast records that runtime fact for exactOptionalPropertyTypes.
|
||||
this.packChunks = (config as Required<Config>).packChunks
|
||||
this.compression = config.compression ?? DEFAULT_COMPRESSION
|
||||
this.coordinator = new PersistenceCoordinator<JsonlTornMarker>(this.ctx, this)
|
||||
}
|
||||
|
||||
// Each backend keeps the typed service surface beside its storage hooks;
|
||||
@@ -68,6 +110,11 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
/* jscpd:ignore-start */
|
||||
// --- SessionPersistence service surface (delegated to the coordinator) ---
|
||||
|
||||
/** Resolve the absolute target path without touching the filesystem. */
|
||||
locate(meta: SessionHeader): SessionLocation {
|
||||
return { kind: 'jsonl', path: logPath(this.root, meta.cwd, meta.id, this.compression) }
|
||||
}
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(meta)
|
||||
}
|
||||
@@ -87,7 +134,8 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
// --- PersistenceBackend hooks (the file-bytes storage primitives) ---
|
||||
|
||||
/** Read a stored prefix by id across all cwd buckets when cwd is unknown. */
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<number> | undefined> {
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<JsonlTornMarker> | undefined> {
|
||||
await this.ensureRootEncoding()
|
||||
const file = await this.findLog(id)
|
||||
if (file === undefined) return undefined
|
||||
return this.readPrefix(file.path)
|
||||
@@ -97,28 +145,85 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
* Read a stored prefix within one cwd for HMR adoption. `undefined` names the
|
||||
* no-cwd bucket rather than an unknown cwd, so this never scans other buckets.
|
||||
*/
|
||||
async loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<number> | undefined> {
|
||||
const path = logPath(this.root, cwd, id)
|
||||
if (!await this.exists(path)) return undefined
|
||||
async loadLive(id: SessionId, cwd: string | undefined): Promise<StoredPrefix<JsonlTornMarker> | undefined> {
|
||||
await this.ensureRootEncoding()
|
||||
const path = logPath(this.root, cwd, id, this.compression)
|
||||
if (!await this.exists(path)) {
|
||||
await this.rejectOppositeArtifact(cwd, id)
|
||||
return undefined
|
||||
}
|
||||
return this.readPrefix(path)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a stored prefix and convert torn-tail state to the byte offset the
|
||||
* coordinator can round-trip without knowing the file format.
|
||||
* Read a stored prefix and convert torn-tail state to the opaque marker the
|
||||
* coordinator can round-trip without knowing the physical encoding.
|
||||
*/
|
||||
private async readPrefix(path: string): Promise<StoredPrefix<number>> {
|
||||
private async readPrefix(path: string): Promise<StoredPrefix<JsonlTornMarker>> {
|
||||
const buffer = await readFile(path)
|
||||
if (this.compression === 'zstd') return this.readZstdPrefix(buffer)
|
||||
const { meta, events, committedBytes } = scanLog(buffer)
|
||||
return {
|
||||
meta,
|
||||
events,
|
||||
...committedBytes < buffer.byteLength ? { tornMarker: committedBytes } : {},
|
||||
...committedBytes < buffer.byteLength
|
||||
? { tornMarker: { truncateTo: committedBytes, recoveredEvents: [] } }
|
||||
: {},
|
||||
}
|
||||
}
|
||||
|
||||
/** Decode complete frames and retain complete JSONL records from a torn final frame. */
|
||||
private async readZstdPrefix(buffer: Buffer): Promise<StoredPrefix<JsonlTornMarker>> {
|
||||
const { frames, tornStart } = scanZstdFrames(buffer)
|
||||
if (frames.length === 0) throw new Error('empty or header-less Zstandard session log')
|
||||
|
||||
const plaintextFrames: Buffer[] = []
|
||||
for (const frame of frames) {
|
||||
try {
|
||||
plaintextFrames.push(await decompressZstdFrame(buffer.subarray(frame.start, frame.end)))
|
||||
} catch (error) {
|
||||
throw new Error(`corrupt Zstandard session log: frame at byte ${frame.start} failed validation`, { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
const headerFrame = plaintextFrames[0]
|
||||
if (headerFrame === undefined || headerFrame.length === 0 || headerFrame.indexOf(0x0A) !== headerFrame.length - 1) {
|
||||
throw new Error('corrupt Zstandard session log: first frame is not exactly one header line')
|
||||
}
|
||||
const completePlaintext = Buffer.concat(plaintextFrames)
|
||||
const completePrefix = scanLog(completePlaintext)
|
||||
if (completePrefix.committedBytes !== completePlaintext.length) {
|
||||
throw new Error('corrupt Zstandard session log: complete frame contains a torn JSONL record')
|
||||
}
|
||||
if (tornStart === undefined) {
|
||||
return { meta: completePrefix.meta, events: completePrefix.events }
|
||||
}
|
||||
|
||||
let recoveredPlaintext: Buffer = Buffer.alloc(0)
|
||||
try {
|
||||
recoveredPlaintext = await decompressZstdFrame(buffer.subarray(tornStart))
|
||||
} catch {
|
||||
// A structurally incomplete final frame may end before Node's decoder can
|
||||
// emit any plaintext; the complete prior frames remain recoverable.
|
||||
}
|
||||
const recoveredPrefix = scanLog(Buffer.concat([completePlaintext, recoveredPlaintext]))
|
||||
/* v8 ignore next 3 -- appending plaintext cannot shorten the already-scanned complete prefix */
|
||||
if (recoveredPrefix.events.length < completePrefix.events.length) {
|
||||
throw new Error('corrupt Zstandard session log: recovered prefix does not extend complete frames')
|
||||
}
|
||||
return {
|
||||
meta: recoveredPrefix.meta,
|
||||
events: recoveredPrefix.events,
|
||||
tornMarker: {
|
||||
truncateTo: tornStart,
|
||||
recoveredEvents: recoveredPrefix.events.slice(completePrefix.events.length),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** Durably append a batch, lazily materializing the file when not yet present. */
|
||||
async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void> {
|
||||
await this.ensureRootEncoding()
|
||||
if (isMaterialized) {
|
||||
await this.appendLines(meta, events)
|
||||
} else {
|
||||
@@ -127,13 +232,18 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
}
|
||||
|
||||
/**
|
||||
* Make a crash repair durable: truncate the torn tail to `tornMarker` bytes (if
|
||||
* any), then append the synthetic `closers` (if any). Two fsync'd steps — the
|
||||
* seam does not require this to be atomic.
|
||||
* Make a crash repair durable: truncate a torn tail, restore complete events
|
||||
* decoded from it, then append synthetic closers. Two fsync'd steps — the seam
|
||||
* does not require this to be atomic.
|
||||
*/
|
||||
async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
if (tornMarker !== undefined) await this.repair(meta, tornMarker)
|
||||
if (closers.length > 0) await this.appendLines(meta, closers)
|
||||
async commitRepair(
|
||||
meta: SessionHeader,
|
||||
tornMarker: JsonlTornMarker | undefined,
|
||||
closers: readonly SessionEvent[],
|
||||
): Promise<void> {
|
||||
if (tornMarker !== undefined) await this.repair(meta, tornMarker.truncateTo)
|
||||
const repairedEvents = [...(tornMarker?.recoveredEvents ?? []), ...closers]
|
||||
if (repairedEvents.length > 0) await this.appendLines(meta, repairedEvents)
|
||||
}
|
||||
|
||||
/** List all stored sessions' metadata (header line only — no full-log parse). */
|
||||
@@ -165,12 +275,15 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
}
|
||||
|
||||
private async listArtifacts(): Promise<Array<{ header: SessionHeader; path: string }>> {
|
||||
await this.ensureRootEncoding()
|
||||
const artifacts: Array<{ header: SessionHeader; path: string }> = []
|
||||
for (const dir of await this.listCwdDirs()) {
|
||||
for (const name of await this.listJsonl(dir)) {
|
||||
for (const name of await this.listArtifactNames(dir)) {
|
||||
const path = `${dir}/${name}`
|
||||
// Read only headers so listing scales with session count, not log size.
|
||||
const first = await this.readFirstLine(path)
|
||||
const first = this.compression === 'zstd'
|
||||
? await this.readFirstZstdLine(path)
|
||||
: await this.readFirstLine(path)
|
||||
if (first === undefined) continue // empty/half-written file
|
||||
const meta = parseHeaderMeta(first)
|
||||
if (meta === undefined) continue // not a session header
|
||||
@@ -182,33 +295,36 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
|
||||
// --- materialization / append / repair (file mechanics) ---
|
||||
|
||||
/** Atomically write the header line + first batch (temp-write, fsync, collision-safe hard-link publish). */
|
||||
/** Atomically write the header line + first batch (temp-write, fsync, publish). */
|
||||
private async materialize(meta: SessionHeader, events: readonly SessionEvent[]): Promise<void> {
|
||||
const dir = sessionDir(this.root, meta.cwd)
|
||||
await mkdir(this.root, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(dirname(this.root))
|
||||
await mkdir(dir, { recursive: true, mode: 0o700 })
|
||||
await this.syncDir(this.root)
|
||||
const finalPath = logPath(this.root, meta.cwd, meta.id)
|
||||
// Materialization is the first write; an existing log is an id collision.
|
||||
/* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */
|
||||
if (await this.exists(finalPath)) {
|
||||
throw new Error(`refusing to materialize "${meta.id}": a log already exists on disk (load/resume it instead)`)
|
||||
const finalPath = logPath(this.root, meta.cwd, meta.id, this.compression)
|
||||
await this.rejectOppositeArtifact(meta.cwd, meta.id)
|
||||
const content = await this.encodeMaterialization(meta, events)
|
||||
/* v8 ignore next -- native Windows coverage exercises this platform dispatch; Linux covers the POSIX peer */
|
||||
if (process.platform === 'win32') {
|
||||
await this.materializeWin32(dir, finalPath, meta.id, content)
|
||||
} else {
|
||||
await this.materializePosix(dir, finalPath, meta.id, content)
|
||||
}
|
||||
const header = JSON.stringify(toHeaderLine(meta))
|
||||
const body = events.map(eventLine).join('\n')
|
||||
const content = header + '\n' + body + '\n'
|
||||
}
|
||||
|
||||
const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp`
|
||||
const handle = await open(tmp, 'wx', 0o600)
|
||||
try {
|
||||
await handle.writeFile(content)
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
// Publish with link()+unlink(): unlike rename(), link fails if another
|
||||
// process materialized the same id first.
|
||||
/* v8 ignore start -- Windows uses the Win32 durable-publish path; POSIX coverage exercises this peer. */
|
||||
private async materializePosix(
|
||||
dir: string,
|
||||
finalPath: string,
|
||||
id: SessionId,
|
||||
content: Buffer | string,
|
||||
): Promise<void> {
|
||||
await mkdir(this.root, { recursive: true, mode: 0o700 })
|
||||
await this.syncDirPosix(dirname(this.root))
|
||||
await mkdir(dir, { recursive: true, mode: 0o700 })
|
||||
await this.syncDirPosix(this.root)
|
||||
await this.rejectExistingLog(finalPath, id)
|
||||
const tmp = await this.writeSyncedTempFile(finalPath, content)
|
||||
// Publish via link()+unlink(), NOT rename(): link fails with EEXIST if the
|
||||
// final path already exists, so two processes materializing the same id
|
||||
// concurrently cannot clobber each other. rename() would silently overwrite.
|
||||
let linked = false
|
||||
try {
|
||||
await link(tmp, finalPath)
|
||||
@@ -219,19 +335,84 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
/* v8 ignore next -- link failure is the TOCTOU/IO race guarded above; not reachable in test */
|
||||
if (!linked) await rm(tmp, { force: true })
|
||||
}
|
||||
// The published link becomes crash-durable only after its directory fsync.
|
||||
await this.syncDir(dir)
|
||||
// Best-effort temp cleanup: the log is already published and durable, so a failure to
|
||||
// remove the (now-redundant) temp hard link must not reject the append.
|
||||
// link() succeeded — the log is published. fsync the directory so the new
|
||||
// entry survives a power loss: the new link is not crash-durable until the
|
||||
// parent directory's metadata is synced.
|
||||
await this.syncDirPosix(dir)
|
||||
// Best-effort temp cleanup: the log is already published and durable, so a
|
||||
// failure to remove the (now-redundant) temp hard link must NOT reject the
|
||||
// append. Swallow only the rm failure; nothing else of consequence runs here.
|
||||
try {
|
||||
await rm(tmp, { force: true })
|
||||
} catch {
|
||||
/* v8 ignore next -- redundant temp link; publish already durable, rm failure is an unreachable IO edge */
|
||||
}
|
||||
}
|
||||
/* v8 ignore stop */
|
||||
|
||||
/** fsync a directory so a just-created or published entry inside it is crash-durable. */
|
||||
private async syncDir(dir: string): Promise<void> {
|
||||
/* v8 ignore start -- native Windows coverage exercises this integration path */
|
||||
private async materializeWin32(
|
||||
dir: string,
|
||||
finalPath: string,
|
||||
id: SessionId,
|
||||
content: Buffer | string,
|
||||
): Promise<void> {
|
||||
await ensureDurableDirectoryWin32(this.root)
|
||||
await ensureDurableDirectoryWin32(dir)
|
||||
await this.rejectExistingLog(finalPath, id)
|
||||
const tmp = await this.writeSyncedTempFile(finalPath, content)
|
||||
try {
|
||||
await publishNewFileWin32(tmp, finalPath)
|
||||
} catch (error) {
|
||||
await rm(tmp, { force: true })
|
||||
throw error
|
||||
}
|
||||
}
|
||||
/* v8 ignore stop */
|
||||
|
||||
private async rejectExistingLog(finalPath: string, id: SessionId): Promise<void> {
|
||||
// Never publish over an existing committed log: materialize is the first
|
||||
// write of a session the backend believes is new. A file here means a
|
||||
// different session shares this id on disk — reject loudly. (createCore
|
||||
// already guards the create path, so this is unreachable-in-practice TOCTOU
|
||||
// defense.)
|
||||
/* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */
|
||||
if (await this.exists(finalPath)) {
|
||||
throw new Error(`refusing to materialize "${id}": a log already exists on disk (load/resume it instead)`)
|
||||
}
|
||||
}
|
||||
|
||||
private async writeSyncedTempFile(finalPath: string, content: Buffer | string): Promise<string> {
|
||||
const tmp = `${finalPath}.${randomBytes(6).toString('hex')}.tmp`
|
||||
const handle = await open(tmp, 'wx', 0o600)
|
||||
try {
|
||||
await handle.writeFile(content)
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
return tmp
|
||||
}
|
||||
|
||||
/** Encode the header and first batch without combining their frame boundaries. */
|
||||
private async encodeMaterialization(meta: SessionHeader, events: readonly SessionEvent[]): Promise<Buffer | string> {
|
||||
const header = JSON.stringify(toHeaderLine(meta)) + '\n'
|
||||
const body = eventLines(events, this.packChunks) + '\n'
|
||||
if (this.compression === 'none') return header + body
|
||||
const headerFrame = await compressZstdFrame(header)
|
||||
const eventFrame = await compressZstdFrame(body)
|
||||
return Buffer.concat([headerFrame, eventFrame])
|
||||
}
|
||||
|
||||
/** Encode one durable append batch in the configured physical representation. */
|
||||
private async encodeEventBatch(events: readonly SessionEvent[]): Promise<Buffer | string> {
|
||||
const body = eventLines(events, this.packChunks) + '\n'
|
||||
return this.compression === 'zstd' ? compressZstdFrame(body) : body
|
||||
}
|
||||
|
||||
/** fsync a POSIX directory so a just-created/renamed entry is crash-durable. */
|
||||
/* v8 ignore start -- Windows uses write-through namespace operations; POSIX coverage exercises directory fsync. */
|
||||
private async syncDirPosix(dir: string): Promise<void> {
|
||||
const handle = await open(dir, 'r')
|
||||
try {
|
||||
await handle.sync()
|
||||
@@ -239,6 +420,7 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
/* v8 ignore stop */
|
||||
|
||||
/**
|
||||
* Append and fsync event lines. On a partial write or sync failure, restore the
|
||||
@@ -246,19 +428,40 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
* batch; leaving partial bytes would create duplicate sequence numbers.
|
||||
*/
|
||||
private async appendLines(meta: SessionHeader, events: readonly SessionEvent[]): Promise<void> {
|
||||
const path = logPath(this.root, meta.cwd, meta.id)
|
||||
const content = await this.encodeEventBatch(events)
|
||||
const path = logPath(this.root, meta.cwd, meta.id, this.compression)
|
||||
const handle = await open(path, 'a')
|
||||
let closed = false
|
||||
const closeAppendHandle = async (): Promise<void> => {
|
||||
if (closed) return
|
||||
closed = true
|
||||
await handle.close()
|
||||
}
|
||||
|
||||
try {
|
||||
const { size: before } = await handle.stat()
|
||||
try {
|
||||
await handle.writeFile(events.map(eventLine).join('\n') + '\n')
|
||||
await handle.writeFile(content)
|
||||
await handle.sync()
|
||||
} catch (error) {
|
||||
// Roll back whatever bytes landed so a retry starts from a clean EOF.
|
||||
await handle.truncate(before)
|
||||
await handle.sync()
|
||||
try {
|
||||
await closeAppendHandle()
|
||||
await this.rollbackAppend(path, before)
|
||||
} catch (rollbackError) {
|
||||
throw new AggregateError([error, rollbackError], `failed to roll back append to "${path}"`)
|
||||
}
|
||||
throw error
|
||||
}
|
||||
} finally {
|
||||
await closeAppendHandle()
|
||||
}
|
||||
}
|
||||
|
||||
private async rollbackAppend(path: string, size: number): Promise<void> {
|
||||
const handle = await open(path, 'r+')
|
||||
try {
|
||||
await handle.truncate(size)
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
@@ -266,7 +469,7 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
|
||||
/** Truncate the log file to `offset` bytes and fsync (discard the crash tail). */
|
||||
private async repair(meta: SessionHeader, offset: number): Promise<void> {
|
||||
const path = logPath(this.root, meta.cwd, meta.id)
|
||||
const path = logPath(this.root, meta.cwd, meta.id, this.compression)
|
||||
await truncate(path, offset)
|
||||
const handle = await open(path, 'r+')
|
||||
try {
|
||||
@@ -304,17 +507,47 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
}
|
||||
}
|
||||
|
||||
/** Read and validate only the independently compressed header frame. */
|
||||
private async readFirstZstdLine(path: string): Promise<string | undefined> {
|
||||
const handle = await open(path, 'r')
|
||||
try {
|
||||
let content = Buffer.alloc(0)
|
||||
const chunk = Buffer.alloc(8192)
|
||||
for (;;) {
|
||||
const { bytesRead } = await handle.read(chunk, 0, chunk.length, null)
|
||||
if (bytesRead === 0) return undefined
|
||||
content = Buffer.concat([content, chunk.subarray(0, bytesRead)])
|
||||
const first = scanZstdFrames(content, 1).frames[0]
|
||||
if (first === undefined) continue
|
||||
let plaintext: Buffer
|
||||
try {
|
||||
plaintext = await decompressZstdFrame(content.subarray(first.start, first.end))
|
||||
} catch (error) {
|
||||
throw new Error('corrupt Zstandard session log: header frame failed validation', { cause: error })
|
||||
}
|
||||
if (plaintext.length === 0 || plaintext.indexOf(0x0A) !== plaintext.length - 1) {
|
||||
throw new Error('corrupt Zstandard session log: first frame is not exactly one header line')
|
||||
}
|
||||
return plaintext.subarray(0, -1).toString('utf8')
|
||||
}
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Find a session by id across cwd buckets for resume. Cwd-scoped HMR adoption
|
||||
* bypasses this scan so a no-cwd session cannot claim another bucket.
|
||||
*/
|
||||
private async findLog(id: SessionId): Promise<{ path: string; cwd: string | undefined } | undefined> {
|
||||
const target = encodeSegment(id) + '.jsonl'
|
||||
const target = encodeSegment(id) + logSuffix(this.compression)
|
||||
for (const dir of await this.listCwdDirs()) {
|
||||
const path = `${dir}/${target}`
|
||||
const opposite = `${dir}/${encodeSegment(id)}${logSuffix(this.oppositeCompression())}`
|
||||
if (await this.exists(opposite)) throw this.encodingMismatch(opposite)
|
||||
if (await this.exists(path)) {
|
||||
// Recover the cwd from the header so the caller has the session's bucket.
|
||||
const { meta } = scanLog(await readFile(path))
|
||||
const { meta } = await this.readPrefix(path)
|
||||
return { path, cwd: meta.cwd }
|
||||
}
|
||||
}
|
||||
@@ -333,9 +566,45 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
}
|
||||
}
|
||||
|
||||
private async listJsonl(dir: string): Promise<string[]> {
|
||||
private async listArtifactNames(dir: string): Promise<string[]> {
|
||||
const entries = await readdir(dir)
|
||||
return entries.filter(n => n.endsWith('.jsonl'))
|
||||
const oppositeSuffix = logSuffix(this.oppositeCompression())
|
||||
const incompatible = entries.find(name => name.endsWith(oppositeSuffix))
|
||||
if (incompatible !== undefined) throw this.encodingMismatch(`${dir}/${incompatible}`)
|
||||
const suffix = logSuffix(this.compression)
|
||||
return entries.filter(name => name.endsWith(suffix))
|
||||
}
|
||||
|
||||
/** Reject a root that already belongs to the other physical encoding. */
|
||||
private ensureRootEncoding(): Promise<void> {
|
||||
this.rootEncodingCheck ??= this.checkRootEncoding()
|
||||
return this.rootEncodingCheck
|
||||
}
|
||||
|
||||
private async checkRootEncoding(): Promise<void> {
|
||||
const oppositeSuffix = logSuffix(this.oppositeCompression())
|
||||
for (const dir of await this.listCwdDirs()) {
|
||||
const entries = await readdir(dir)
|
||||
const incompatible = entries.find(name => name.endsWith(oppositeSuffix))
|
||||
if (incompatible !== undefined) throw this.encodingMismatch(`${dir}/${incompatible}`)
|
||||
}
|
||||
}
|
||||
|
||||
private async rejectOppositeArtifact(cwd: string | undefined, id: SessionId): Promise<void> {
|
||||
const path = logPath(this.root, cwd, id, this.oppositeCompression())
|
||||
if (await this.exists(path)) throw this.encodingMismatch(path)
|
||||
}
|
||||
|
||||
private oppositeCompression(): JsonlCompression {
|
||||
return this.compression === 'zstd' ? 'none' : 'zstd'
|
||||
}
|
||||
|
||||
private encodingMismatch(path: string): Error {
|
||||
return new Error(
|
||||
`session artifact ${JSON.stringify(path)} uses ${logSuffix(this.oppositeCompression())}, `
|
||||
+ `but this backend is configured for compression ${JSON.stringify(this.compression)}; `
|
||||
+ 'use a separate root or select the matching compression mode',
|
||||
)
|
||||
}
|
||||
|
||||
private async exists(path: string): Promise<boolean> {
|
||||
@@ -344,13 +613,36 @@ export class SessionPersistenceJsonl extends SessionPersistence implements Persi
|
||||
await handle.close()
|
||||
return true
|
||||
} catch (error) {
|
||||
// Only ENOENT means absent. A permission/I/O error must surface, not be
|
||||
// collapsed to `false` — otherwise load() reports "not found" and collision
|
||||
// checks proceed under a false absence assumption.
|
||||
if (isENOENT(error)) return false
|
||||
// Only ENOENT means absent. A permission/I/O error must surface rather
|
||||
// than letting load or collision checks proceed under false absence.
|
||||
// Windows reports ENOENT, not ENOTDIR, for `regular-file/child`; verify
|
||||
// the immediate parent so a blocked cwd bucket remains a storage fault.
|
||||
/* v8 ignore else -- Windows reports file-valued parents as ENOENT; POSIX covers direct ENOTDIR. */
|
||||
if (isENOENT(error)) {
|
||||
await this.assertLogParentAllowsAbsence(path)
|
||||
return false
|
||||
}
|
||||
/* v8 ignore next -- Windows repairs ENOTDIR from ENOENT above; POSIX covers direct ENOTDIR. */
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
/* v8 ignore start -- native Windows coverage exercises this repair; POSIX open reports ENOTDIR before this point. */
|
||||
private async assertLogParentAllowsAbsence(path: string): Promise<void> {
|
||||
try {
|
||||
const parent = dirname(path)
|
||||
const info = await stat(parent)
|
||||
if (info.isDirectory()) return
|
||||
const error = new Error(`ENOTDIR: parent path exists but is not a directory: ${parent}`) as NodeJS.ErrnoException
|
||||
error.code = 'ENOTDIR'
|
||||
error.path = parent
|
||||
throw error
|
||||
} catch (error) {
|
||||
if (isENOENT(error)) return
|
||||
throw error
|
||||
}
|
||||
}
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
|
||||
export default SessionPersistenceJsonl
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-persistence-jsonl`.
|
||||
* @module @deepseek-ai/dsh-session-persistence-jsonl/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-persistence-jsonl-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: persistence correctness requires backend round-trip and crash-tail tests;
|
||||
* this package exposes no continuously observable in-process relation.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Windows durable namespace helpers for the JSONL backend.
|
||||
*
|
||||
* POSIX publishes a newly-created log by creating a directory entry and then
|
||||
* fsyncing the parent directory. Windows does not expose that parent-directory
|
||||
* fsync contract through Node, so the Windows path uses the native durable
|
||||
* namespace primitive instead: create a staging object in the target directory
|
||||
* and publish it with `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` without
|
||||
* replacement or cross-volume copy fallback.
|
||||
*
|
||||
* @module dsh-session-persistence-jsonl/win32
|
||||
*/
|
||||
|
||||
import { mkdtemp, rm, stat } from 'node:fs/promises'
|
||||
import { basename, join, parse, resolve, toNamespacedPath } from 'node:path'
|
||||
|
||||
type MoveFileExW = (existing: string, replacement: string, flags: number) => number
|
||||
type GetLastError = () => number
|
||||
|
||||
interface Win32Bindings {
|
||||
moveFileExW: MoveFileExW
|
||||
getLastError: GetLastError
|
||||
}
|
||||
|
||||
interface Win32ErrnoException extends NodeJS.ErrnoException {
|
||||
win32Code: number
|
||||
dest: string
|
||||
}
|
||||
|
||||
const MOVEFILE_WRITE_THROUGH = 0x00000008
|
||||
const ERROR_FILE_NOT_FOUND = 2
|
||||
const ERROR_PATH_NOT_FOUND = 3
|
||||
const ERROR_ACCESS_DENIED = 5
|
||||
const ERROR_NOT_SAME_DEVICE = 17
|
||||
const ERROR_FILE_EXISTS = 80
|
||||
const ERROR_INVALID_NAME = 123
|
||||
const ERROR_ALREADY_EXISTS = 183
|
||||
|
||||
let bindings: Win32Bindings | undefined
|
||||
|
||||
/** Load the small Win32 surface lazily so non-Windows processes never load Koffi. */
|
||||
async function win32(): Promise<Win32Bindings> {
|
||||
if (bindings !== undefined) return bindings
|
||||
const koffi = (await import('koffi')).default
|
||||
const kernel32 = koffi.load('kernel32.dll')
|
||||
bindings = {
|
||||
moveFileExW: kernel32.func('__stdcall', 'MoveFileExW', 'int', ['str16', 'str16', 'uint']) as MoveFileExW,
|
||||
getLastError: kernel32.func('__stdcall', 'GetLastError', 'uint', []) as GetLastError,
|
||||
}
|
||||
return bindings
|
||||
}
|
||||
|
||||
function errnoCode(win32Code: number): string {
|
||||
switch (win32Code) {
|
||||
case ERROR_FILE_NOT_FOUND:
|
||||
case ERROR_PATH_NOT_FOUND:
|
||||
return 'ENOENT'
|
||||
case ERROR_ACCESS_DENIED:
|
||||
return 'EACCES'
|
||||
case ERROR_NOT_SAME_DEVICE:
|
||||
return 'EXDEV'
|
||||
case ERROR_FILE_EXISTS:
|
||||
case ERROR_ALREADY_EXISTS:
|
||||
return 'EEXIST'
|
||||
case ERROR_INVALID_NAME:
|
||||
return 'EINVAL'
|
||||
default:
|
||||
return 'EIO'
|
||||
}
|
||||
}
|
||||
|
||||
function win32Error(syscall: string, win32Code: number, path: string, dest: string): Win32ErrnoException {
|
||||
const code = errnoCode(win32Code)
|
||||
const error = new Error(`${syscall} ${code} (Win32 ${win32Code}): ${path} -> ${dest}`) as Win32ErrnoException
|
||||
error.code = code
|
||||
error.errno = win32Code
|
||||
error.syscall = syscall
|
||||
error.path = path
|
||||
error.dest = dest
|
||||
error.win32Code = win32Code
|
||||
return error
|
||||
}
|
||||
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'ENOENT'
|
||||
}
|
||||
|
||||
function isEEXIST(error: unknown): boolean {
|
||||
return (error as NodeJS.ErrnoException | null)?.code === 'EEXIST'
|
||||
}
|
||||
|
||||
async function assertDirectory(path: string): Promise<boolean> {
|
||||
try {
|
||||
const info = await stat(path)
|
||||
if (info.isDirectory()) return true
|
||||
const error = new Error(`path exists but is not a directory: ${path}`) as NodeJS.ErrnoException
|
||||
error.code = 'ENOTDIR'
|
||||
error.path = path
|
||||
throw error
|
||||
} catch (error) {
|
||||
if (isENOENT(error)) return false
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish `existing` at `replacement` with Windows write-through rename
|
||||
* semantics. The destination must not already exist; the move must stay within
|
||||
* the volume (no copy fallback flag is set).
|
||||
* @param existing - the synced staging path to move.
|
||||
* @param replacement - the final path, which must not already exist.
|
||||
*/
|
||||
export async function publishNewFileWin32(existing: string, replacement: string): Promise<void> {
|
||||
const api = await win32()
|
||||
const ok = api.moveFileExW(toNamespacedPath(existing), toNamespacedPath(replacement), MOVEFILE_WRITE_THROUGH)
|
||||
if (ok === 0) throw win32Error('MoveFileExW', api.getLastError(), existing, replacement)
|
||||
}
|
||||
|
||||
/**
|
||||
* Create `target` and its missing ancestors with durable Windows namespace
|
||||
* publication. Each missing directory is first created as a random staging
|
||||
* sibling, then moved to its final name with `MOVEFILE_WRITE_THROUGH`; races
|
||||
* with another creator are accepted only after verifying the winner is a
|
||||
* directory.
|
||||
* @param target - the absolute directory path to create durably when absent.
|
||||
*/
|
||||
export async function ensureDurableDirectoryWin32(target: string): Promise<void> {
|
||||
const absolute = resolve(target)
|
||||
const root = parse(absolute).root
|
||||
await assertDirectory(root)
|
||||
|
||||
const segments = absolute.slice(root.length).split(/[\\/]+/).filter(part => part.length > 0)
|
||||
let current = root
|
||||
for (const segment of segments) {
|
||||
const next = join(current, segment)
|
||||
if (!await assertDirectory(next)) await createLeafDirectoryWin32(current, next)
|
||||
current = next
|
||||
}
|
||||
}
|
||||
|
||||
async function createLeafDirectoryWin32(parent: string, target: string): Promise<void> {
|
||||
const staging = await mkdtemp(join(parent, `.dsh-mkdir-${basename(target)}-`))
|
||||
try {
|
||||
await publishNewFileWin32(staging, target)
|
||||
} catch (error) {
|
||||
await rm(staging, { recursive: true, force: true })
|
||||
if (isEEXIST(error) && await assertDirectory(target)) return
|
||||
throw error
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,116 @@
|
||||
/**
|
||||
* Zstandard frame primitives for the JSONL persistence backend. The backend
|
||||
* owns a concatenated-frame container so it can append and recover batches
|
||||
* without exposing compression mechanics through the persistence seam.
|
||||
* @module dsh-session-persistence-jsonl/zstd
|
||||
*/
|
||||
|
||||
import { constants, zstdCompress, zstdDecompress, type ZstdOptions } from 'node:zlib'
|
||||
import { promisify } from 'node:util'
|
||||
|
||||
const ZSTD_MAGIC = 0xFD2FB528
|
||||
const zstdCompressAsync = promisify(zstdCompress)
|
||||
const zstdDecompressAsync = promisify(zstdDecompress)
|
||||
const CHECKSUM_OPTIONS: ZstdOptions = {
|
||||
params: { [constants.ZSTD_c_checksumFlag]: 1 },
|
||||
}
|
||||
|
||||
/** Byte range occupied by one structurally complete Zstandard frame. */
|
||||
export interface ZstdFrameRange {
|
||||
/** Inclusive frame start. */
|
||||
start: number
|
||||
/** Exclusive frame end. */
|
||||
end: number
|
||||
}
|
||||
|
||||
/** Structural scan result for a concatenated Zstandard stream. */
|
||||
export interface ZstdFrameScan {
|
||||
/** Complete frames in file order. */
|
||||
frames: ZstdFrameRange[]
|
||||
/** Start of an incomplete final frame, when EOF interrupts one. */
|
||||
tornStart?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Locate complete frames without decompressing their blocks. Invalid complete
|
||||
* structure rejects; EOF inside the final frame returns its start for repair.
|
||||
* @param buffer - complete bytes currently present in the session artifact.
|
||||
* @param maxFrames - optional complete-frame limit for metadata-only readers.
|
||||
* @returns complete frame ranges and an optional incomplete-final-frame start.
|
||||
*/
|
||||
export function scanZstdFrames(buffer: Buffer, maxFrames = Number.POSITIVE_INFINITY): ZstdFrameScan {
|
||||
const frames: ZstdFrameRange[] = []
|
||||
let offset = 0
|
||||
|
||||
while (offset < buffer.length) {
|
||||
const start = offset
|
||||
if (buffer.length - offset < 4) return { frames, tornStart: start }
|
||||
if (buffer.readUInt32LE(offset) !== ZSTD_MAGIC) {
|
||||
throw new Error(`corrupt Zstandard session log: invalid frame magic at byte ${offset}`)
|
||||
}
|
||||
offset += 4
|
||||
|
||||
if (offset === buffer.length) return { frames, tornStart: start }
|
||||
const descriptor = buffer.readUInt8(offset)
|
||||
offset += 1
|
||||
if ((descriptor & 0x18) !== 0) {
|
||||
throw new Error(`corrupt Zstandard session log: reserved frame-header bit at byte ${offset - 1}`)
|
||||
}
|
||||
|
||||
const contentSizeFlag = descriptor >>> 6
|
||||
const singleSegment = (descriptor & 0x20) !== 0
|
||||
const checksum = (descriptor & 0x04) !== 0
|
||||
const dictionaryFlag = descriptor & 0x03
|
||||
const dictionaryBytes = dictionaryFlag === 3 ? 4 : dictionaryFlag
|
||||
const contentSizeBytes = contentSizeFlag === 0
|
||||
? (singleSegment ? 1 : 0)
|
||||
: 1 << contentSizeFlag
|
||||
const remainingHeaderBytes = (singleSegment ? 0 : 1) + dictionaryBytes + contentSizeBytes
|
||||
if (buffer.length - offset < remainingHeaderBytes) return { frames, tornStart: start }
|
||||
offset += remainingHeaderBytes
|
||||
|
||||
for (;;) {
|
||||
if (buffer.length - offset < 3) return { frames, tornStart: start }
|
||||
const blockHeader = buffer.readUIntLE(offset, 3)
|
||||
offset += 3
|
||||
const lastBlock = (blockHeader & 1) !== 0
|
||||
const blockType = (blockHeader >>> 1) & 0x03
|
||||
const blockSize = blockHeader >>> 3
|
||||
if (blockType === 0x03) {
|
||||
throw new Error(`corrupt Zstandard session log: reserved block type at byte ${offset - 3}`)
|
||||
}
|
||||
const payloadBytes = blockType === 0x01 ? 1 : blockSize
|
||||
if (buffer.length - offset < payloadBytes) return { frames, tornStart: start }
|
||||
offset += payloadBytes
|
||||
if (lastBlock) break
|
||||
}
|
||||
|
||||
if (checksum) {
|
||||
if (buffer.length - offset < 4) return { frames, tornStart: start }
|
||||
offset += 4
|
||||
}
|
||||
frames.push({ start, end: offset })
|
||||
if (frames.length === maxFrames) return { frames }
|
||||
}
|
||||
|
||||
return { frames }
|
||||
}
|
||||
|
||||
/**
|
||||
* Compress one independently decodable, checksummed Zstandard frame.
|
||||
* @param input - JSONL bytes for a header or durable event batch.
|
||||
* @returns the complete encoded frame.
|
||||
*/
|
||||
export async function compressZstdFrame(input: Buffer | string): Promise<Buffer> {
|
||||
return zstdCompressAsync(input, CHECKSUM_OPTIONS)
|
||||
}
|
||||
|
||||
/**
|
||||
* Decompress one complete frame or the available prefix of a torn final frame.
|
||||
* Complete-frame checksums are validated by Node's decoder.
|
||||
* @param input - bytes beginning at a Zstandard frame boundary.
|
||||
* @returns plaintext produced from the available input.
|
||||
*/
|
||||
export async function decompressZstdFrame(input: Buffer): Promise<Buffer> {
|
||||
return zstdDecompressAsync(input)
|
||||
}
|
||||
@@ -2,11 +2,11 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { isAbsolute, join, relative, resolve } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import { encodeSegment, logPath, scanLog, sessionDir } from '../src/format.ts'
|
||||
import { encodeSegment, eventLines, logPath, scanLog, sessionDir, toHeaderLine } from '../src/format.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
@@ -20,17 +20,15 @@ function mutableHeader(header: SessionHeader): MutableSessionHeader {
|
||||
return header
|
||||
}
|
||||
|
||||
async function expectParallelFlushError(promise: Promise<unknown>, message: RegExp): Promise<void> {
|
||||
async function expectFlushError(promise: Promise<unknown>, message: RegExp): Promise<void> {
|
||||
try {
|
||||
await promise
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(AggregateError)
|
||||
const [cause] = (error as AggregateError).errors as unknown[]
|
||||
expect(cause).toBeInstanceOf(Error)
|
||||
expect((cause as Error).message).toMatch(message)
|
||||
expect(error).toBeInstanceOf(Error)
|
||||
expect((error as Error).message).toMatch(message)
|
||||
return
|
||||
}
|
||||
throw new Error('expected parallel flush to reject')
|
||||
throw new Error('expected flush to reject')
|
||||
}
|
||||
|
||||
async function freshRoot(): Promise<string> {
|
||||
@@ -39,7 +37,12 @@ async function freshRoot(): Promise<string> {
|
||||
return dir
|
||||
}
|
||||
|
||||
function rawLogPath(root: string, cwd: string | undefined, id: SessionId): string {
|
||||
return logPath(root, cwd, id, 'none')
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks()
|
||||
for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
@@ -53,11 +56,11 @@ function appendClosedTurn(session: Session): void {
|
||||
}
|
||||
|
||||
// Run the shared backend contract against the real JSONL backend.
|
||||
runPersistenceContract('jsonl', async () => {
|
||||
runPersistenceContract('jsonl-none', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-'))
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir })
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir, compression: 'none' })
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
dispose: async () => {
|
||||
@@ -69,18 +72,18 @@ runPersistenceContract('jsonl', async () => {
|
||||
|
||||
// Two mounts share this temp root to exercise reload. `corruptTail` appends a partial,
|
||||
// newline-less fragment past the committed region so coordinator repair runs on real file bytes.
|
||||
runCoordinatorContract('jsonl', async (): Promise<CoordinatorFixture> => {
|
||||
runCoordinatorContract('jsonl-none', async (): Promise<CoordinatorFixture> => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-coord-'))
|
||||
return {
|
||||
mount: async (ctx) => {
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir })
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root: dir, compression: 'none' })
|
||||
return fiber
|
||||
},
|
||||
corruptTail: async (id, cwd) => {
|
||||
// A half-written record with no trailing newline: scanLog treats it as an
|
||||
// uncommitted crash fragment and reports committedBytes < byteLength, so
|
||||
// the coordinator sees a tornMarker to truncate.
|
||||
await appendFile(logPath(dir, cwd, id), '{"type":"assistant/chunk","seq":8,"ti')
|
||||
await appendFile(rawLogPath(dir, cwd, id), '{"type":"assistant/chunk","seq":8,"ti')
|
||||
},
|
||||
cleanup: async () => { await rm(dir, { recursive: true, force: true }) },
|
||||
}
|
||||
@@ -112,6 +115,22 @@ describe('SessionPersistenceJsonl: format helpers', () => {
|
||||
it('encodeSegment rejects an empty id', () => {
|
||||
expect(() => encodeSegment('')).toThrow(/empty/)
|
||||
})
|
||||
|
||||
it('resolves a relative custom root before locating a session', async () => {
|
||||
const absoluteRoot = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, {
|
||||
root: relative(process.cwd(), absoluteRoot),
|
||||
compression: 'none',
|
||||
})
|
||||
const m = meta('relative-location', '/work')
|
||||
expect(ctx.sessionPersistence.locate(m)).toEqual({
|
||||
kind: 'jsonl',
|
||||
path: rawLogPath(resolve(absoluteRoot), '/work', m.id),
|
||||
})
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
@@ -120,25 +139,50 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
root = await freshRoot()
|
||||
ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
})
|
||||
afterEach(async () => { await ctx.fiber.dispose() })
|
||||
|
||||
it('lazy materialization: create() writes no file until the first append', async () => {
|
||||
const m = meta('lazy', '/work')
|
||||
const location = ctx.sessionPersistence.locate(m)
|
||||
expect(location).toEqual({ kind: 'jsonl', path: rawLogPath(root, '/work', m.id) })
|
||||
expect(isAbsolute(location!.path)).toBe(true)
|
||||
|
||||
await ctx.sessionPersistence.create(m)
|
||||
// nothing on disk yet
|
||||
// locate() is a pure target-path calculation: neither it nor create()
|
||||
// materializes a file before the first append.
|
||||
const dir = sessionDir(root, '/work')
|
||||
await expect(stat(logPath(root, '/work', m.id))).rejects.toThrow()
|
||||
await expect(stat(rawLogPath(root, '/work', m.id))).rejects.toThrow()
|
||||
expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id)
|
||||
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
// now materialized
|
||||
expect((await stat(logPath(root, '/work', m.id))).isFile()).toBe(true)
|
||||
expect((await stat(rawLogPath(root, '/work', m.id))).isFile()).toBe(true)
|
||||
expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id)
|
||||
void dir
|
||||
})
|
||||
|
||||
it('keeps the same location on resume and gives a fork its own location', async () => {
|
||||
const parent = meta('location-parent', '/work')
|
||||
const parentLocation = ctx.sessionPersistence.locate(parent)
|
||||
await ctx.sessionPersistence.create(parent)
|
||||
await ctx.sessionPersistence.append(parent.id, oneTurnLog())
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(parent.id)
|
||||
expect(ctx.sessionPersistence.locate(loaded.meta)).toEqual(parentLocation)
|
||||
|
||||
const child = {
|
||||
...loaded.meta,
|
||||
id: SessionId('location-child'),
|
||||
parentSession: parent.id,
|
||||
seedLength: loaded.events.length,
|
||||
}
|
||||
const childLocation = ctx.sessionPersistence.locate(child)
|
||||
expect(childLocation?.path).not.toBe(parentLocation?.path)
|
||||
expect(childLocation).toEqual({ kind: 'jsonl', path: rawLogPath(root, '/work', child.id) })
|
||||
})
|
||||
|
||||
it('round-trip is byte-identical (incl. assistant/chunk verbatim)', async () => {
|
||||
const m = meta('chunks')
|
||||
const log: SessionEvent[] = [
|
||||
@@ -146,7 +190,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'he' } } },
|
||||
{ type: 'assistant/chunk', seq: 3, time: 4, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'llo' } } },
|
||||
{ type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] }, surfaceOp: 'append', sourceEventSeqs: [2, 3] },
|
||||
{ type: 'assistant/message', seq: 4, time: 5, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }], provenance: { provider: 'mock', model: 'mock' } }, surfaceOp: 'append', sourceEventSeqs: [2, 3] },
|
||||
{ type: 'step/end', seq: 5, time: 6, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 6, time: 7, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
@@ -164,13 +208,13 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
|
||||
const reopenedCtx = new Context()
|
||||
await reopenedCtx.plugin(SessionStore)
|
||||
await reopenedCtx.plugin(SessionPersistenceJsonl, { root })
|
||||
await reopenedCtx.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
expect((await reopenedCtx.sessionPersistence.listSnapshots())[0]?.revision).toBe(revision)
|
||||
|
||||
const otherRoot = await freshRoot()
|
||||
const otherCtx = new Context()
|
||||
await otherCtx.plugin(SessionStore)
|
||||
await otherCtx.plugin(SessionPersistenceJsonl, { root: otherRoot })
|
||||
await otherCtx.plugin(SessionPersistenceJsonl, { root: otherRoot, compression: 'none' })
|
||||
await otherCtx.sessionPersistence.create(m)
|
||||
await otherCtx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
expect((await otherCtx.sessionPersistence.listSnapshots())[0]?.revision).not.toBe(revision)
|
||||
@@ -212,12 +256,46 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
discovery.mockRestore()
|
||||
})
|
||||
|
||||
it('rejects a stored v0 log containing a legacy request/header-delta event', async () => {
|
||||
const m = meta('legacy-header-delta', '/legacy')
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
await mkdir(sessionDir(root, m.cwd), { recursive: true })
|
||||
await writeFile(path, [
|
||||
JSON.stringify(toHeaderLine(m)),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'request/header-delta', seq: 1, time: 2, data: { config: { model: 'legacy' } } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
'',
|
||||
].join('\n'))
|
||||
|
||||
await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1/)
|
||||
})
|
||||
|
||||
it('rejects a stored v0 full header carrying the legacy fallback reason', async () => {
|
||||
const m = meta('legacy-header-fallback', '/legacy')
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
await mkdir(sessionDir(root, m.cwd), { recursive: true })
|
||||
await writeFile(path, [
|
||||
JSON.stringify(toHeaderLine(m)),
|
||||
JSON.stringify({
|
||||
type: 'request/header',
|
||||
seq: 0,
|
||||
time: 1,
|
||||
data: { header: { config: { model: 'legacy' } }, reason: 'fallback' },
|
||||
}),
|
||||
'',
|
||||
].join('\n'))
|
||||
|
||||
await expect(ctx.sessionPersistence.load(m.id))
|
||||
.rejects.toThrow(/unsupported legacy request\/header reason "fallback" at seq 0/)
|
||||
})
|
||||
|
||||
it('persists a forked child seed through the existing session write path', async () => {
|
||||
const source = ctx.sessions.create(SessionId('persist-parent'), { meta: { cwd: '/workspace' } })
|
||||
appendClosedTurn(source)
|
||||
|
||||
const child = ctx.sessions.fork(source, undefined, SessionId('persist-child'))
|
||||
await ctx.parallel('session/flush', child)
|
||||
await ctx.sessions.flush(child)
|
||||
const loaded = await ctx.sessionPersistence.load(child.id)
|
||||
|
||||
expect(loaded.events).toEqual(source.events)
|
||||
@@ -237,7 +315,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
// Simulate a crash mid-second-turn: append raw lines that are NOT closed by
|
||||
// a turn/end (turn/start + step/start are fully written), plus a final
|
||||
// partial line with no newline (a torn fragment never fully flushed).
|
||||
const path = logPath(root, '/proj', m.id)
|
||||
const path = rawLogPath(root, '/proj', m.id)
|
||||
await writeFile(path, [
|
||||
JSON.stringify({ type: 'turn/start', seq: 6, time: 8, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'step/start', seq: 7, time: 9, data: { turn: 2, step: 1 } }),
|
||||
@@ -270,17 +348,17 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
const m = meta('append-only')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const before = await readFile(logPath(root, undefined, m.id), 'utf8')
|
||||
const before = await readFile(rawLogPath(root, undefined, m.id), 'utf8')
|
||||
const committedPrefix = before // the whole committed log
|
||||
|
||||
// A crash tail then a repair-append.
|
||||
await writeFile(logPath(root, undefined, m.id), '\n{"partial', { flag: 'a' })
|
||||
await writeFile(rawLogPath(root, undefined, m.id), '\n{"partial', { flag: 'a' })
|
||||
await ctx.sessionPersistence.load(m.id)
|
||||
await ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[])
|
||||
const after = await readFile(logPath(root, undefined, m.id), 'utf8')
|
||||
const after = await readFile(rawLogPath(root, undefined, m.id), 'utf8')
|
||||
// the committed prefix is byte-for-byte intact at the head of the file
|
||||
expect(after.startsWith(committedPrefix)).toBe(true)
|
||||
})
|
||||
@@ -289,12 +367,12 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
const m = meta('truncate-retry')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog()) // materialized, seqs 0..5
|
||||
const sizeBefore = (await stat(logPath(root, undefined, m.id))).size
|
||||
const sizeBefore = (await stat(rawLogPath(root, undefined, m.id))).size
|
||||
|
||||
// Force the NEXT fsync (inside appendLines) to fail once, AFTER writeFile
|
||||
// has already put bytes on disk — simulating an ENOSPC/fsync error
|
||||
// mid-append. The recovery truncate() also fsyncs, so allow that one.
|
||||
const handle = await (await import('node:fs/promises')).open(logPath(root, undefined, m.id), 'r')
|
||||
const handle = await (await import('node:fs/promises')).open(rawLogPath(root, undefined, m.id), 'r')
|
||||
const proto = Object.getPrototypeOf(handle) as { sync: () => Promise<void> }
|
||||
await handle.close()
|
||||
const realSync = proto.sync
|
||||
@@ -311,7 +389,7 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
// The append rejects, but the partial bytes are truncated back: the file is
|
||||
// its pre-append size and the cursor is unchanged.
|
||||
await expect(ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/ENOSPC/)
|
||||
expect((await stat(logPath(root, undefined, m.id))).size).toBe(sizeBefore)
|
||||
expect((await stat(rawLogPath(root, undefined, m.id))).size).toBe(sizeBefore)
|
||||
spy.mockRestore()
|
||||
|
||||
// The retry now succeeds with NO seq gap — the log is contiguous 0..7.
|
||||
@@ -320,6 +398,45 @@ describe('SessionPersistenceJsonl: durability and crash semantics', () => {
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
})
|
||||
|
||||
it('reports both the append failure and a failed rollback', async () => {
|
||||
const m = meta('rollback-failure')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
|
||||
const path = rawLogPath(root, undefined, m.id)
|
||||
const handle = await (await import('node:fs/promises')).open(path, 'r')
|
||||
const proto = Object.getPrototypeOf(handle) as { sync: () => Promise<void> }
|
||||
await handle.close()
|
||||
const realSync = proto.sync
|
||||
let failed = false
|
||||
const syncSpy = vi.spyOn(proto, 'sync').mockImplementation(async function (this: unknown) {
|
||||
if (!failed) { failed = true; throw new Error('simulated append fsync failure') }
|
||||
return realSync.call(this)
|
||||
})
|
||||
const backend = ctx.sessionPersistence as unknown as {
|
||||
rollbackAppend: (path: string, size: number) => Promise<void>
|
||||
}
|
||||
const realRollback = backend.rollbackAppend.bind(backend)
|
||||
backend.rollbackAppend = () => Promise.reject(new Error('simulated rollback failure'))
|
||||
|
||||
try {
|
||||
await ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
] as SessionEvent[])
|
||||
throw new Error('expected append to reject')
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(AggregateError)
|
||||
const aggregate = error as AggregateError
|
||||
expect(aggregate.message).toContain(`failed to roll back append to "${path}"`)
|
||||
expect(aggregate.errors).toHaveLength(2)
|
||||
expect(aggregate.errors[0]).toMatchObject({ message: 'simulated append fsync failure' })
|
||||
expect(aggregate.errors[1]).toMatchObject({ message: 'simulated rollback failure' })
|
||||
} finally {
|
||||
backend.rollbackAppend = realRollback
|
||||
syncSpy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('load returns a meta copy: mutating it does not corrupt backend pathing', async () => {
|
||||
const m = meta('meta-copy', '/proj')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -370,16 +487,18 @@ describe('SessionPersistenceJsonl: write path (session/event → flush)', () =>
|
||||
root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
|
||||
const a = ctx.sessions.create(SessionId('sa'))
|
||||
const b = ctx.sessions.create(SessionId('sb'))
|
||||
a.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
b.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
a.append('user/message', { content: [{ type: 'text', text: 'A' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
b.append('user/message', { content: [{ type: 'text', text: 'B' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
a.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
b.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', a)
|
||||
await ctx.parallel('session/flush', b)
|
||||
await ctx.sessions.flush(a)
|
||||
await ctx.sessions.flush(b)
|
||||
|
||||
const la = await ctx.sessionPersistence.load(SessionId('sa'))
|
||||
const lb = await ctx.sessionPersistence.load(SessionId('sb'))
|
||||
@@ -406,9 +525,30 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
expect(() => scanLog(Buffer.from('{"type":"event"}\n'))).toThrow(/session header/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['missing', undefined],
|
||||
['a string', '1'],
|
||||
['fractional', 1.5],
|
||||
['negative', -1],
|
||||
])('rejects a session header with %s delegationDepth', (_label, delegationDepth) => {
|
||||
const log = JSON.stringify({
|
||||
type: 'session',
|
||||
version: 0,
|
||||
id: 'invalid-depth',
|
||||
createdAt: 1,
|
||||
...delegationDepth === undefined ? {} : { delegationDepth },
|
||||
}) + '\n'
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(/session header/)
|
||||
})
|
||||
|
||||
it('rejects a session header with negative-zero delegationDepth', () => {
|
||||
const log = '{"type":"session","version":0,"id":"invalid-depth","createdAt":1,"delegationDepth":-0}\n'
|
||||
expect(() => scanLog(Buffer.from(log))).toThrow(/session header/)
|
||||
})
|
||||
|
||||
it('a seq gap after the last turn/end bounds the preserved tail (torn fragment tolerated)', () => {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'g', createdAt: 1 }),
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'g', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1
|
||||
].join('\n') + '\n'
|
||||
@@ -420,7 +560,7 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
|
||||
it('rejects a seq gap BEFORE a later committed turn/end (committed data damaged)', () => {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'g2', createdAt: 1 }),
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'g2', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1
|
||||
JSON.stringify({ type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
@@ -432,7 +572,7 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
|
||||
it('rejects a corrupt line BEFORE a later committed turn/end (committed data damaged)', () => {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'c', createdAt: 1 }),
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'c', createdAt: 1, delegationDepth: 0 }),
|
||||
'{not json', // corrupt, sits in the committed region (a turn/end follows)
|
||||
JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
@@ -440,7 +580,7 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
})
|
||||
|
||||
it('a header-only log (no event lines at all) preserves nothing — committedBytes is the header', () => {
|
||||
const log = JSON.stringify({ type: 'session', version: 0, id: 'h0', createdAt: 1 }) + '\n'
|
||||
const log = JSON.stringify({ type: 'session', version: 0, id: 'h0', createdAt: 1, delegationDepth: 0 }) + '\n'
|
||||
const scanned = scanLog(Buffer.from(log))
|
||||
expect(scanned.events).toEqual([])
|
||||
// committedBytes falls back to the header line's end (no preserved events).
|
||||
@@ -449,7 +589,7 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
|
||||
it('a corrupt line after the last turn/end bounds the preserved tail', () => {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'c2', createdAt: 1 }),
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'c2', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
'{not json', // corrupt crash fragment, no turn/end committed
|
||||
].join('\n') + '\n'
|
||||
@@ -460,7 +600,7 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
|
||||
it('tolerates a seq gap AFTER a turn/end (uncommitted tail)', () => {
|
||||
const log = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 't', createdAt: 1 }),
|
||||
JSON.stringify({ type: 'session', version: 0, id: 't', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
JSON.stringify({ type: 'step/start', seq: 9, time: 3, data: { turn: 2, step: 1 } }), // gap in uncommitted tail
|
||||
@@ -470,13 +610,128 @@ describe('SessionPersistenceJsonl: scanLog unit', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: packed chunk rows (packChunks: true)', () => {
|
||||
let ctx: Context
|
||||
beforeEach(async () => {
|
||||
root = await freshRoot()
|
||||
ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// compression: 'none' — these tests assert the textual storage-record layout
|
||||
// (row tags per line); packing is orthogonal to the physical encoding.
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root, packChunks: true, compression: 'none' })
|
||||
})
|
||||
afterEach(async () => { await ctx.fiber.dispose() })
|
||||
|
||||
/** A one-turn log whose step streams a five-member text-delta run. */
|
||||
function chunkRunLog(): SessionEvent[] {
|
||||
const deltas: SessionEvent[] = Array.from({ length: 5 }, (_, k) => ({
|
||||
type: 'assistant/chunk',
|
||||
seq: 2 + k,
|
||||
time: 3 + k,
|
||||
data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: `t${k}` } },
|
||||
}))
|
||||
return [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
...deltas,
|
||||
{ type: 'assistant/message', seq: 7, time: 8, data: { turn: 1, step: 1, content: [{ type: 'text', text: 't0t1t2t3t4' }], provenance: { provider: 'mock', model: 'mock' } }, surfaceOp: 'append', sourceEventSeqs: [2, 3, 4, 5, 6] },
|
||||
{ type: 'step/end', seq: 8, time: 9, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 9, time: 10, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
}
|
||||
|
||||
it('writes a delta run as one text-chunks row and loads back identical events', async () => {
|
||||
const m = meta('packed', '/work')
|
||||
const log = chunkRunLog()
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, log)
|
||||
|
||||
const raw = (await readFile(rawLogPath(root, '/work', m.id), 'utf8')).split('\n').filter(Boolean)
|
||||
const tags = raw.slice(1).map(line => (JSON.parse(line) as { type: string }).type)
|
||||
expect(tags).toEqual(['turn/start', 'step/start', 'text-chunks', 'assistant/message', 'step/end', 'turn/end'])
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toEqual(log)
|
||||
})
|
||||
|
||||
it('loads a mixed file: verbatim lines from an unpacked writer, then packed appends', async () => {
|
||||
const m = meta('mixed', '/work')
|
||||
const log = chunkRunLog()
|
||||
// First turn written line-per-event by an unpacked-config writer (an old
|
||||
// file, hand-planted so this packed-config backend adopts it on load).
|
||||
await mkdir(sessionDir(root, '/work'), { recursive: true })
|
||||
await writeFile(rawLogPath(root, '/work', m.id), [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'mixed', createdAt: 1000, cwd: '/work', delegationDepth: 0 }),
|
||||
...log.map(e => JSON.stringify(e)),
|
||||
].join('\n') + '\n')
|
||||
// Adopt the stored log (cursor = stored length), then append a second turn
|
||||
// through THIS packed-config backend.
|
||||
expect((await ctx.sessionPersistence.load(m.id)).events).toEqual(log)
|
||||
const secondTurn: SessionEvent[] = JSON.parse(JSON.stringify(log)) as SessionEvent[]
|
||||
for (const [k, e] of secondTurn.entries()) {
|
||||
;(e as { seq: number }).seq = 10 + k
|
||||
;(e.data as { turn: number }).turn = 2
|
||||
}
|
||||
await ctx.sessionPersistence.append(m.id, secondTurn)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toEqual([...log, ...secondTurn])
|
||||
// The packed append really packed: the file's tail carries a text-chunks row.
|
||||
const tags = (await readFile(rawLogPath(root, '/work', m.id), 'utf8')).split('\n').filter(Boolean)
|
||||
.map(line => (JSON.parse(line) as { type: string }).type)
|
||||
expect(tags.filter(t => t === 'text-chunks')).toHaveLength(1)
|
||||
expect(tags.filter(t => t === 'assistant/chunk')).toHaveLength(5)
|
||||
})
|
||||
|
||||
it('scanLog: a packed row advances the seq cursor by its whole run', () => {
|
||||
const logText = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'rows', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 1, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: 4, time: 5, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
const { events } = scanLog(Buffer.from(logText))
|
||||
expect(events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4])
|
||||
expect(events[2]).toEqual({ type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'b' } } })
|
||||
})
|
||||
|
||||
it('scanLog: a malformed packed row in the committed region rejects like corrupt JSON', () => {
|
||||
const logText = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'bad-row', createdAt: 1, delegationDepth: 0 }),
|
||||
// dt arity mismatch — row validation throws, so the line is a committed hole.
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [], texts: ['a', 'b'] } }),
|
||||
JSON.stringify({ type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
expect(() => scanLog(Buffer.from(logText))).toThrow(/unparsable committed event/)
|
||||
})
|
||||
|
||||
it('scanLog: a packed row with a mid-run seq gap after the last turn/end drops the whole row', () => {
|
||||
const logText = [
|
||||
JSON.stringify({ type: 'session', version: 0, id: 'row-gap', createdAt: 1, delegationDepth: 0 }),
|
||||
JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } }),
|
||||
// seq0 skips 1 — the run's first member is already a gap; no turn/end follows.
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 2, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }),
|
||||
].join('\n') + '\n'
|
||||
const scanned = scanLog(Buffer.from(logText))
|
||||
expect(scanned.events.map(e => e.seq)).toEqual([0])
|
||||
// committedBytes stays on the line boundary BEFORE the dropped row.
|
||||
const headerAndTurn = logText.split('\n').slice(0, 2).join('\n') + '\n'
|
||||
expect(scanned.committedBytes).toBe(Buffer.byteLength(headerAndTurn, 'utf8'))
|
||||
})
|
||||
|
||||
it('eventLines(packChunks: false) is byte-identical to the pre-packing layout', () => {
|
||||
const log = chunkRunLog()
|
||||
expect(eventLines(log, false)).toBe(log.map(e => JSON.stringify(e)).join('\n'))
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
let ctx: Context
|
||||
beforeEach(async () => {
|
||||
root = await freshRoot()
|
||||
ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
})
|
||||
afterEach(async () => { await ctx.fiber.dispose() })
|
||||
|
||||
@@ -496,8 +751,8 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
await p
|
||||
await ctx.sessionPersistence.append(SessionId('create-snap'), oneTurnLog())
|
||||
// The log materialized under the ORIGINAL cwd, not the mutated one.
|
||||
expect((await stat(logPath(root, '/orig', SessionId('create-snap')))).isFile()).toBe(true)
|
||||
await expect(stat(logPath(root, '/mutated', SessionId('create-snap')))).rejects.toThrow()
|
||||
expect((await stat(rawLogPath(root, '/orig', SessionId('create-snap')))).isFile()).toBe(true)
|
||||
await expect(stat(rawLogPath(root, '/mutated', SessionId('create-snap')))).rejects.toThrow()
|
||||
})
|
||||
|
||||
it('list discovers sessions across multiple cwd buckets', async () => {
|
||||
@@ -538,7 +793,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
// `readFirstLine` accumulates chunks before `list()` parses it.
|
||||
const bucket = join(root, '_no-cwd')
|
||||
await mkdir(bucket, { recursive: true })
|
||||
const bigHeader = JSON.stringify({ type: 'session', version: 0, id: 'big', createdAt: 1, pad: 'x'.repeat(9000) })
|
||||
const bigHeader = JSON.stringify({ type: 'session', version: 0, id: 'big', createdAt: 1, delegationDepth: 0, pad: 'x'.repeat(9000) })
|
||||
await writeFile(join(bucket, 'big.jsonl'), bigHeader + '\n')
|
||||
const ids = (await ctx.sessionPersistence.list()).map(x => x.id)
|
||||
expect(ids).toContain('big')
|
||||
@@ -552,7 +807,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
}, { inject: ['sessions'] }))
|
||||
// Drain A, then dispose ITS fiber (the live session A is gone) while the
|
||||
// backend stays loaded.
|
||||
for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s)
|
||||
for (const s of ctx.sessions.list()) await ctx.sessions.flush(s)
|
||||
await sessFiberA.dispose()
|
||||
|
||||
// A new Session object reuses the id. Object-keyed initialization must run independently,
|
||||
@@ -578,7 +833,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
// of grafting no-cwd events onto a log with mismatched cwd.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
let b!: Session
|
||||
await ctx2.plugin(Object.assign((inner: Context) => {
|
||||
b = inner.sessions.create(SessionId('x')) // no cwd
|
||||
@@ -587,10 +842,10 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
|
||||
// The "/w" log is untouched — no no-cwd events were grafted onto it, and no
|
||||
// `_no-cwd` log for "x" was created.
|
||||
const inW = scanLog(await readFile(logPath(root, '/w', SessionId('x'))))
|
||||
const inW = scanLog(await readFile(rawLogPath(root, '/w', SessionId('x'))))
|
||||
expect(inW.meta.cwd).toBe('/w')
|
||||
expect(inW.events).toHaveLength(6)
|
||||
await expect(stat(logPath(root, undefined, SessionId('x')))).rejects.toThrow()
|
||||
await expect(stat(rawLogPath(root, undefined, SessionId('x')))).rejects.toThrow()
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
@@ -620,7 +875,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
a.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
a.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
}, { inject: ['sessions'] }))
|
||||
for (const s of ctx.sessions.list()) await ctx.parallel('session/flush', s)
|
||||
for (const s of ctx.sessions.list()) await ctx.sessions.flush(s)
|
||||
await firstFiber.dispose()
|
||||
|
||||
let second!: Session
|
||||
@@ -634,7 +889,10 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
it('list returns nothing when the root directory does not exist', async () => {
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root: join(root, 'does-not-exist-yet') })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, {
|
||||
root: join(root, 'does-not-exist-yet'),
|
||||
compression: 'none',
|
||||
})
|
||||
expect(await ctx2.sessionPersistence.list()).toEqual([])
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
@@ -646,7 +904,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
await writeFile(filePath, 'x')
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root: filePath })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root: filePath, compression: 'none' })
|
||||
await expect(ctx2.sessionPersistence.list()).rejects.toThrow(/ENOTDIR/)
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
@@ -657,7 +915,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
const cwd = '/x'
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
await writeFile(sessionDir(root, cwd), 'x') // bucket path is now a FILE
|
||||
let s!: Session
|
||||
await ctx2.plugin(Object.assign((inner: Context) => {
|
||||
@@ -672,14 +930,14 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
const m = meta('disk-append', '/d')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await writeFile(logPath(root, '/d', m.id), '\n{"partial crash', { flag: 'a' })
|
||||
await writeFile(rawLogPath(root, '/d', m.id), '\n{"partial crash', { flag: 'a' })
|
||||
|
||||
// A FRESH backend with no in-memory state: append directly (no prior load)
|
||||
// → append must adopt from disk, and the adopt's load schedules a repair
|
||||
// that the same append then performs before writing.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
await ctx2.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 9, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 10, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
@@ -715,7 +973,7 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
// nondeterministic. create scans every bucket, not just meta.cwd's.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
await expect(ctx2.sessionPersistence.create(meta('dup-id', '/projB')))
|
||||
.rejects.toThrow(/already has a persisted log on disk/)
|
||||
await ctx2.fiber.dispose()
|
||||
@@ -725,21 +983,22 @@ describe('SessionPersistenceJsonl: edge cases', () => {
|
||||
root = await freshRoot()
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root })
|
||||
await ctx2.plugin(SessionPersistenceJsonl, { root, compression: 'none' })
|
||||
const session = ctx2.sessions.create(SessionId('flush-fail'))
|
||||
// A full turn lands in the write-behind buffer.
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// Make the durable materialize fail on the next flush.
|
||||
const backend = ctx2.sessionPersistence as unknown as { materialize: (...args: unknown[]) => Promise<void> }
|
||||
const origMat = backend.materialize.bind(backend)
|
||||
backend.materialize = () => Promise.reject(new Error('disk full'))
|
||||
await expectParallelFlushError(ctx2.parallel('session/flush', session), /disk full/)
|
||||
await expectFlushError(ctx2.sessions.flush(session), /disk full/)
|
||||
// The events are STILL buffered (not silently dropped): a retry persists them.
|
||||
backend.materialize = origMat
|
||||
await ctx2.parallel('session/flush', session)
|
||||
await ctx2.sessions.flush(session)
|
||||
const loaded = await ctx2.sessionPersistence.load(SessionId('flush-fail'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2])
|
||||
await ctx2.fiber.dispose()
|
||||
})
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
/**
|
||||
* Unit tests for the Windows durable namespace helper with a mocked kernel32
|
||||
* binding. The real JSONL suite exercises the helper on native Windows; these
|
||||
* tests keep the Win32 error mapping and race handling covered on every host.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs'
|
||||
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
const MOVEFILE_WRITE_THROUGH = 0x00000008
|
||||
const ERROR_FILE_NOT_FOUND = 2
|
||||
const ERROR_PATH_NOT_FOUND = 3
|
||||
const ERROR_ACCESS_DENIED = 5
|
||||
const ERROR_NOT_SAME_DEVICE = 17
|
||||
const ERROR_FILE_EXISTS = 80
|
||||
const ERROR_INVALID_NAME = 123
|
||||
const ERROR_ALREADY_EXISTS = 183
|
||||
|
||||
type MoveFileExW = (existing: string, replacement: string, flags: number, setLastError: (code: number) => void) => number
|
||||
|
||||
const roots: string[] = []
|
||||
|
||||
function stripNamespace(path: string): string {
|
||||
if (path.startsWith('\\\\?\\UNC\\')) return `\\\\${path.slice('\\\\?\\UNC\\'.length)}`
|
||||
if (path.startsWith('\\\\?\\')) return path.slice('\\\\?\\'.length)
|
||||
return path
|
||||
}
|
||||
|
||||
async function tempRoot(): Promise<string> {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-jsonl-win32-'))
|
||||
roots.push(dir)
|
||||
return dir
|
||||
}
|
||||
|
||||
async function importWithMove(moveFileExW: MoveFileExW): Promise<typeof import('../src/win32.ts')> {
|
||||
vi.resetModules()
|
||||
vi.doMock('koffi', () => {
|
||||
let lastError = 0
|
||||
const setLastError = (code: number): void => { lastError = code }
|
||||
const move: MoveFileExW = (existing, replacement, flags, setError) => {
|
||||
const ok = moveFileExW(existing, replacement, flags, setError)
|
||||
lastError = ok === 0 ? lastError : 0
|
||||
return ok
|
||||
}
|
||||
return {
|
||||
default: {
|
||||
load: () => ({
|
||||
func: (_convention: string, name: string, result: string) => {
|
||||
if (name === 'MoveFileExW') return (existing: string, replacement: string, flags: number) => {
|
||||
expect(result).toBe('int')
|
||||
const ok = move(existing, replacement, flags, setLastError)
|
||||
return ok
|
||||
}
|
||||
return () => lastError
|
||||
},
|
||||
}),
|
||||
},
|
||||
}
|
||||
})
|
||||
return import('../src/win32.ts')
|
||||
}
|
||||
|
||||
async function importWithError(code: number): Promise<typeof import('../src/win32.ts')> {
|
||||
vi.resetModules()
|
||||
vi.doMock('koffi', () => ({
|
||||
default: {
|
||||
load: () => ({
|
||||
func: (_convention: string, name: string) => {
|
||||
if (name === 'MoveFileExW') return () => 0
|
||||
return () => code
|
||||
},
|
||||
}),
|
||||
},
|
||||
}))
|
||||
return import('../src/win32.ts')
|
||||
}
|
||||
|
||||
async function importWithFilesystemMove(): Promise<typeof import('../src/win32.ts')> {
|
||||
return importWithMove((existing, replacement, flags, setLastError) => {
|
||||
expect(flags).toBe(MOVEFILE_WRITE_THROUGH)
|
||||
const from = stripNamespace(existing)
|
||||
const to = stripNamespace(replacement)
|
||||
if (!existsSync(from)) { setLastError(ERROR_FILE_NOT_FOUND); return 0 }
|
||||
if (existsSync(to)) { setLastError(ERROR_ALREADY_EXISTS); return 0 }
|
||||
renameSync(from, to)
|
||||
return 1
|
||||
})
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
vi.doUnmock('koffi')
|
||||
vi.resetModules()
|
||||
for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
describe('Windows durable namespace helpers', () => {
|
||||
it('publishes a new file with write-through MoveFileExW semantics', async () => {
|
||||
const { publishNewFileWin32 } = await importWithFilesystemMove()
|
||||
const root = await tempRoot()
|
||||
const tmp = join(root, 'log.tmp')
|
||||
const final = join(root, 'log.jsonl')
|
||||
await writeFile(tmp, 'content')
|
||||
|
||||
await publishNewFileWin32(tmp, final)
|
||||
expect(existsSync(tmp)).toBe(false)
|
||||
expect(readFileSync(final, 'utf8')).toBe('content')
|
||||
})
|
||||
|
||||
it('maps Win32 publish failures to Node-style errno codes', async () => {
|
||||
const cases = [
|
||||
[ERROR_FILE_NOT_FOUND, 'ENOENT'],
|
||||
[ERROR_PATH_NOT_FOUND, 'ENOENT'],
|
||||
[ERROR_ACCESS_DENIED, 'EACCES'],
|
||||
[ERROR_NOT_SAME_DEVICE, 'EXDEV'],
|
||||
[ERROR_FILE_EXISTS, 'EEXIST'],
|
||||
[ERROR_ALREADY_EXISTS, 'EEXIST'],
|
||||
[ERROR_INVALID_NAME, 'EINVAL'],
|
||||
[9999, 'EIO'],
|
||||
] as const
|
||||
for (const [win32Code, code] of cases) {
|
||||
const { publishNewFileWin32 } = await importWithError(win32Code)
|
||||
await expect(publishNewFileWin32('from', 'to')).rejects.toMatchObject({ code, win32Code, path: 'from', dest: 'to' })
|
||||
}
|
||||
})
|
||||
|
||||
it('creates missing directories through staging siblings and tolerates an already-created race', async () => {
|
||||
const root = await tempRoot()
|
||||
const raced = join(root, 'raced')
|
||||
const { ensureDurableDirectoryWin32 } = await importWithMove((existing, replacement, flags, setLastError) => {
|
||||
expect(flags).toBe(MOVEFILE_WRITE_THROUGH)
|
||||
const from = stripNamespace(existing)
|
||||
const to = stripNamespace(replacement)
|
||||
if (to === raced) {
|
||||
mkdirSync(to)
|
||||
setLastError(ERROR_ALREADY_EXISTS)
|
||||
return 0
|
||||
}
|
||||
if (!existsSync(from)) { setLastError(ERROR_FILE_NOT_FOUND); return 0 }
|
||||
if (existsSync(to)) { setLastError(ERROR_ALREADY_EXISTS); return 0 }
|
||||
renameSync(from, to)
|
||||
return 1
|
||||
})
|
||||
|
||||
await ensureDurableDirectoryWin32(join(root, 'a', 'b'))
|
||||
expect(existsSync(join(root, 'a', 'b'))).toBe(true)
|
||||
await ensureDurableDirectoryWin32(join(root, 'a', 'b'))
|
||||
await ensureDurableDirectoryWin32(raced)
|
||||
expect(existsSync(raced)).toBe(true)
|
||||
})
|
||||
|
||||
it('surfaces directory publication failures other than an existing-target race', async () => {
|
||||
const { ensureDurableDirectoryWin32 } = await importWithError(ERROR_ACCESS_DENIED)
|
||||
const root = await tempRoot()
|
||||
|
||||
await expect(ensureDurableDirectoryWin32(join(root, 'denied'))).rejects.toMatchObject({ code: 'EACCES' })
|
||||
})
|
||||
|
||||
it('rejects a non-directory component instead of treating it as missing', async () => {
|
||||
const { ensureDurableDirectoryWin32 } = await importWithFilesystemMove()
|
||||
const root = await tempRoot()
|
||||
const blocked = join(root, 'blocked')
|
||||
writeFileSync(blocked, 'x')
|
||||
|
||||
await expect(ensureDurableDirectoryWin32(join(blocked, 'child'))).rejects.toMatchObject({ code: 'ENOTDIR' })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,24 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { compressZstdFrame, decompressZstdFrame, scanZstdFrames } from '../src/zstd.ts'
|
||||
|
||||
describe('JSONL Zstandard compatibility', () => {
|
||||
it('round-trips concatenated checksummed frames through the built-in Node API', async () => {
|
||||
const encoded = Buffer.concat([
|
||||
await compressZstdFrame('{"type":"session","version":0,"id":"compat","createdAt":1}\n'),
|
||||
await compressZstdFrame('{"type":"turn/start","seq":0,"turn":1}\n'),
|
||||
])
|
||||
const { frames, tornStart } = scanZstdFrames(encoded)
|
||||
|
||||
expect(tornStart).toBeUndefined()
|
||||
expect(frames).toHaveLength(2)
|
||||
expect(frames.map(frame => encoded.subarray(frame.start, frame.start + 4).toString('hex')))
|
||||
.toEqual(['28b52ffd', '28b52ffd'])
|
||||
const decoded = await Promise.all(frames.map(frame => decompressZstdFrame(encoded.subarray(frame.start, frame.end))))
|
||||
expect(Buffer.concat(decoded).toString()).toContain('"type":"turn/start"')
|
||||
|
||||
const eventFrame = encoded.subarray(frames[1]!.start, frames[1]!.end)
|
||||
const missingChecksumByte = eventFrame.subarray(0, -1)
|
||||
expect(scanZstdFrames(missingChecksumByte)).toEqual({ frames: [], tornStart: 0 })
|
||||
expect((await decompressZstdFrame(missingChecksumByte)).toString()).toContain('"type":"turn/start"')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,483 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { appendFile, mkdir, mkdtemp, open, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import type { FileHandle } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import { logPath, scanLog, sessionDir, toHeaderLine, type JsonlCompression } from '../src/format.ts'
|
||||
import { compressZstdFrame, decompressZstdFrame, scanZstdFrames } from '../src/zstd.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
const MAGIC = Buffer.from([0x28, 0xB5, 0x2F, 0xFD])
|
||||
const roots: string[] = []
|
||||
const contexts: Context[] = []
|
||||
|
||||
async function freshRoot(prefix = 'dsh-jsonl-zstd-'): Promise<string> {
|
||||
const root = await mkdtemp(join(tmpdir(), prefix))
|
||||
roots.push(root)
|
||||
return root
|
||||
}
|
||||
|
||||
async function mount(root: string, compression?: JsonlCompression): Promise<Context> {
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SessionPersistenceJsonl, {
|
||||
root,
|
||||
...(compression === undefined ? {} : { compression }),
|
||||
})
|
||||
return ctx
|
||||
}
|
||||
|
||||
async function decodeCompleteFrames(buffer: Buffer): Promise<Buffer> {
|
||||
const { frames, tornStart } = scanZstdFrames(buffer)
|
||||
expect(tornStart).toBeUndefined()
|
||||
const plaintext: Buffer[] = []
|
||||
for (const frame of frames) {
|
||||
plaintext.push(await decompressZstdFrame(buffer.subarray(frame.start, frame.end)))
|
||||
}
|
||||
return Buffer.concat(plaintext)
|
||||
}
|
||||
|
||||
async function tornFrame(
|
||||
plaintext: string,
|
||||
accepts: (decoded: string) => boolean,
|
||||
): Promise<Buffer> {
|
||||
const frame = await compressZstdFrame(plaintext)
|
||||
const candidateEnds = [
|
||||
frame.length - 1,
|
||||
frame.length - 4,
|
||||
...[0.9, 0.75, 0.6, 0.5, 0.4, 0.25].map(ratio => Math.floor(frame.length * ratio)),
|
||||
]
|
||||
for (const end of candidateEnds) {
|
||||
const candidate = frame.subarray(0, end)
|
||||
if (scanZstdFrames(candidate).tornStart !== 0) continue
|
||||
try {
|
||||
const decoded = (await decompressZstdFrame(candidate)).toString('utf8')
|
||||
if (accepts(decoded)) return candidate
|
||||
} catch {
|
||||
// Some early cuts precede the first decodable block; keep searching for
|
||||
// a cut that exercises partial-plaintext recovery.
|
||||
}
|
||||
}
|
||||
throw new Error('test fixture could not produce the requested torn Zstandard frame')
|
||||
}
|
||||
|
||||
function deterministicNoise(length: number): string {
|
||||
let state = 0x12345678
|
||||
let output = ''
|
||||
for (let index = 0; index < length; index++) {
|
||||
state = (Math.imul(state, 1_664_525) + 1_013_904_223) >>> 0
|
||||
output += String.fromCharCode(33 + (state % 90))
|
||||
}
|
||||
return output
|
||||
}
|
||||
|
||||
function emptyStructuralFrame(descriptor: number): Buffer {
|
||||
const contentSizeFlag = descriptor >>> 6
|
||||
const singleSegment = (descriptor & 0x20) !== 0
|
||||
const dictionaryBytes = [0, 1, 2, 4][descriptor & 0x03]!
|
||||
const contentSizeBytes = contentSizeFlag === 0 ? (singleSegment ? 1 : 0) : 1 << contentSizeFlag
|
||||
const variableHeader = Buffer.alloc((singleSegment ? 0 : 1) + dictionaryBytes + contentSizeBytes)
|
||||
const lastEmptyRawBlock = Buffer.from([1, 0, 0])
|
||||
const checksum = (descriptor & 0x04) === 0 ? Buffer.alloc(0) : Buffer.alloc(4)
|
||||
return Buffer.concat([MAGIC, Buffer.from([descriptor]), variableHeader, lastEmptyRawBlock, checksum])
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks()
|
||||
for (const ctx of contexts.splice(0).reverse()) await ctx.fiber.dispose()
|
||||
for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
runPersistenceContract('jsonl-zstd', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-jsonl-zstd-contract-'))
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceJsonl, { root })
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
dispose: async () => {
|
||||
await fiber.dispose()
|
||||
await rm(root, { recursive: true, force: true })
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
runCoordinatorContract('jsonl-zstd', async (): Promise<CoordinatorFixture> => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'dsh-jsonl-zstd-coordinator-'))
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(SessionPersistenceJsonl, { root }),
|
||||
corruptTail: async (id, cwd) => {
|
||||
const line = JSON.stringify({
|
||||
type: 'assistant/chunk',
|
||||
seq: 8,
|
||||
time: 9,
|
||||
data: { turn: 2, step: 1, chunk: { type: 'text-delta', index: 0, text: deterministicNoise(300_000) } },
|
||||
}) + '\n'
|
||||
const partial = await tornFrame(line, decoded => decoded.length > 0 && !decoded.endsWith('\n'))
|
||||
await appendFile(logPath(root, cwd, id, 'zstd'), partial)
|
||||
},
|
||||
cleanup: async () => { await rm(root, { recursive: true, force: true }) },
|
||||
}
|
||||
})
|
||||
|
||||
describe('Zstandard frame structure', () => {
|
||||
it('scans concatenated checksummed frames and honors a frame limit', async () => {
|
||||
const first = await compressZstdFrame('header\n')
|
||||
const second = await compressZstdFrame('event\n')
|
||||
const stream = Buffer.concat([first, second])
|
||||
expect(scanZstdFrames(Buffer.alloc(0))).toEqual({ frames: [] })
|
||||
expect(scanZstdFrames(stream)).toEqual({
|
||||
frames: [{ start: 0, end: first.length }, { start: first.length, end: stream.length }],
|
||||
})
|
||||
expect(scanZstdFrames(stream, 1)).toEqual({ frames: [{ start: 0, end: first.length }] })
|
||||
expect(first[4]! & 0x04).toBe(0x04)
|
||||
expect(second[4]! & 0x04).toBe(0x04)
|
||||
expect((await decompressZstdFrame(first)).toString()).toBe('header\n')
|
||||
})
|
||||
|
||||
it('distinguishes incomplete frame regions from invalid complete structure', () => {
|
||||
expect(scanZstdFrames(MAGIC.subarray(0, 2))).toEqual({ frames: [], tornStart: 0 })
|
||||
expect(scanZstdFrames(MAGIC)).toEqual({ frames: [], tornStart: 0 })
|
||||
expect(() => scanZstdFrames(Buffer.alloc(4))).toThrow(/invalid frame magic/)
|
||||
expect(() => scanZstdFrames(Buffer.concat([MAGIC, Buffer.from([0x08])]))).toThrow(/reserved frame-header bit/)
|
||||
|
||||
// Non-single-segment descriptor with no window descriptor.
|
||||
expect(scanZstdFrames(Buffer.concat([MAGIC, Buffer.from([0x00])]))).toEqual({ frames: [], tornStart: 0 })
|
||||
// Single-segment header followed by only two bytes of the three-byte block header.
|
||||
expect(scanZstdFrames(Buffer.concat([MAGIC, Buffer.from([0x20, 0x00, 0x01, 0x00])]))).toEqual({
|
||||
frames: [],
|
||||
tornStart: 0,
|
||||
})
|
||||
|
||||
const rawFiveBytes = Buffer.from([(5 << 3) | 1, 0, 0])
|
||||
expect(scanZstdFrames(Buffer.concat([
|
||||
MAGIC,
|
||||
Buffer.from([0x20, 0x00]),
|
||||
rawFiveBytes,
|
||||
Buffer.from([0x01, 0x02]),
|
||||
]))).toEqual({ frames: [], tornStart: 0 })
|
||||
|
||||
const reservedBlock = Buffer.concat([
|
||||
MAGIC,
|
||||
Buffer.from([0x20, 0x00, 0x07, 0x00, 0x00]),
|
||||
])
|
||||
expect(() => scanZstdFrames(reservedBlock)).toThrow(/reserved block type/)
|
||||
})
|
||||
|
||||
it('covers standard header variants, RLE blocks, multiple blocks, and checksums', () => {
|
||||
for (const descriptor of [0x00, 0x21, 0x42, 0x83, 0xE3]) {
|
||||
const frame = emptyStructuralFrame(descriptor)
|
||||
expect(scanZstdFrames(frame)).toEqual({ frames: [{ start: 0, end: frame.length }] })
|
||||
}
|
||||
|
||||
const rle = Buffer.concat([
|
||||
MAGIC,
|
||||
Buffer.from([0x20, 0x01]),
|
||||
Buffer.from([(1 << 3) | (1 << 1) | 1, 0, 0]),
|
||||
Buffer.from([0x41]),
|
||||
])
|
||||
expect(scanZstdFrames(rle)).toEqual({ frames: [{ start: 0, end: rle.length }] })
|
||||
|
||||
const twoBlocks = Buffer.concat([
|
||||
MAGIC,
|
||||
Buffer.from([0x20, 0x00]),
|
||||
Buffer.from([0, 0, 0]),
|
||||
Buffer.from([1, 0, 0]),
|
||||
])
|
||||
expect(scanZstdFrames(twoBlocks)).toEqual({ frames: [{ start: 0, end: twoBlocks.length }] })
|
||||
|
||||
const checksummed = emptyStructuralFrame(0x24)
|
||||
expect(scanZstdFrames(checksummed.subarray(0, -1))).toEqual({ frames: [], tornStart: 0 })
|
||||
expect(scanZstdFrames(checksummed)).toEqual({ frames: [{ start: 0, end: checksummed.length }] })
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: default Zstandard encoding', () => {
|
||||
it('writes .jsonl.zstd by default with one header frame and one first-batch frame', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('default-zstd', '/work')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const buffer = await readFile(path)
|
||||
expect(buffer.subarray(0, 4)).toEqual(MAGIC)
|
||||
await expect(stat(logPath(root, header.cwd, header.id, 'none'))).rejects.toThrow()
|
||||
expect(ctx.sessionPersistence.locate(header)).toEqual({ kind: 'jsonl', path })
|
||||
|
||||
const scan = scanZstdFrames(buffer)
|
||||
expect(scan.frames).toHaveLength(2)
|
||||
const plaintext = await decodeCompleteFrames(buffer)
|
||||
expect(plaintext.toString()).toBe([
|
||||
JSON.stringify(toHeaderLine(header)),
|
||||
...oneTurnLog().map(e => JSON.stringify(e)),
|
||||
'',
|
||||
].join('\n'))
|
||||
expect((await ctx.sessionPersistence.load(header.id)).events).toEqual(oneTurnLog())
|
||||
})
|
||||
|
||||
it('resolves the default when a programmatic wrapper bypasses Loader schema normalization', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(SessionStore)
|
||||
let backend!: SessionPersistenceJsonl
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
backend = new SessionPersistenceJsonl(inner, { root })
|
||||
}, { inject: ['sessions'] }))
|
||||
const header = meta('direct-default')
|
||||
expect(backend.locate(header)).toEqual({
|
||||
kind: 'jsonl',
|
||||
path: logPath(root, header.cwd, header.id, 'zstd'),
|
||||
})
|
||||
})
|
||||
|
||||
it('appends one frame per durable batch without rewriting prior bytes', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('append-frame')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const before = await readFile(path)
|
||||
const secondTurn = [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
await ctx.sessionPersistence.append(header.id, secondTurn)
|
||||
|
||||
const after = await readFile(path)
|
||||
expect(after.subarray(0, before.length)).toEqual(before)
|
||||
expect(scanZstdFrames(after).frames).toHaveLength(3)
|
||||
expect((await ctx.sessionPersistence.load(header.id)).events).toEqual([...oneTurnLog(), ...secondTurn])
|
||||
})
|
||||
|
||||
it('lists from a multi-chunk header frame without decoding a corrupt event frame', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('large-header', `/work/${'x'.repeat(24_000)}`)
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const buffer = Buffer.from(await readFile(path))
|
||||
const eventFrame = scanZstdFrames(buffer).frames[1]!
|
||||
buffer[eventFrame.end - 1] = buffer[eventFrame.end - 1]! ^ 0xFF
|
||||
await writeFile(path, buffer)
|
||||
|
||||
expect((await ctx.sessionPersistence.list()).map(item => item.id)).toEqual([header.id])
|
||||
await expect(ctx.sessionPersistence.load(header.id)).rejects.toThrow(/frame at byte .* failed validation/)
|
||||
})
|
||||
|
||||
it('preserves complete records from a torn frame and re-encodes them with crash closers', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('recover-torn', '/proj')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const committed = await readFile(path)
|
||||
const openTurn = [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
{ type: 'assistant/chunk', seq: 8, time: 9, data: { turn: 2, step: 1, chunk: { type: 'text-delta', index: 0, text: deterministicNoise(300_000) } } },
|
||||
] as SessionEvent[]
|
||||
const plaintext = openTurn.map(e => JSON.stringify(e)).join('\n') + '\n'
|
||||
const partial = await tornFrame(plaintext, (decoded) => {
|
||||
const newlines = decoded.match(/\n/g)?.length ?? 0
|
||||
return newlines >= 2 && !decoded.endsWith('\n')
|
||||
})
|
||||
await appendFile(path, partial)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(header.id)
|
||||
expect(loaded.events.map(event => event.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
expect(loaded.events[6]).toEqual(openTurn[0])
|
||||
expect(loaded.events[7]).toEqual(openTurn[1])
|
||||
expect(loaded.events.some(event => event.type === 'assistant/chunk' && event.seq === 8)).toBe(false)
|
||||
expect(loaded.events[8]?.type).toBe('step/end')
|
||||
expect(loaded.events[9]?.type).toBe('turn/end')
|
||||
|
||||
const repaired = await readFile(path)
|
||||
expect(repaired.subarray(0, committed.length)).toEqual(committed)
|
||||
expect(scanZstdFrames(repaired).tornStart).toBeUndefined()
|
||||
expect(scanLog(await decodeCompleteFrames(repaired)).events).toEqual(loaded.events)
|
||||
})
|
||||
|
||||
it('drops a frame torn in its header before it has produced plaintext', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('partial-magic')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const committed = await readFile(path)
|
||||
await appendFile(path, MAGIC.subarray(0, 2))
|
||||
|
||||
expect((await ctx.sessionPersistence.load(header.id)).events).toEqual(oneTurnLog())
|
||||
expect(await readFile(path)).toEqual(committed)
|
||||
})
|
||||
|
||||
it('recovers complete events when EOF tears only the final frame checksum', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('partial-checksum')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const secondTurn = [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
const frame = await compressZstdFrame(secondTurn.map(e => JSON.stringify(e)).join('\n') + '\n')
|
||||
await appendFile(path, frame.subarray(0, -1))
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(header.id)
|
||||
expect(loaded.events).toEqual([...oneTurnLog(), ...secondTurn])
|
||||
const repaired = await readFile(path)
|
||||
expect(scanZstdFrames(repaired).tornStart).toBeUndefined()
|
||||
expect(scanLog(await decodeCompleteFrames(repaired)).events).toEqual(loaded.events)
|
||||
})
|
||||
|
||||
it('rejects a complete frame containing a torn JSONL record', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('complete-bad-jsonl')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
await appendFile(
|
||||
logPath(root, header.cwd, header.id, 'zstd'),
|
||||
await compressZstdFrame('{"type":"turn/start"'),
|
||||
)
|
||||
await expect(ctx.sessionPersistence.load(header.id)).rejects.toThrow(/complete frame contains a torn JSONL record/)
|
||||
})
|
||||
|
||||
it('rolls back a checksummed append frame when fsync fails', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('zstd-fsync-rollback')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const path = logPath(root, header.cwd, header.id, 'zstd')
|
||||
const before = await readFile(path)
|
||||
|
||||
const handle = await open(path, 'r')
|
||||
const prototype = Object.getPrototypeOf(handle) as { sync: () => Promise<void> }
|
||||
await handle.close()
|
||||
const realSync = prototype.sync
|
||||
let failed = false
|
||||
const spy = vi.spyOn(prototype, 'sync').mockImplementation(async function (this: FileHandle) {
|
||||
if (!failed) {
|
||||
failed = true
|
||||
throw new Error('simulated Zstandard fsync failure')
|
||||
}
|
||||
return realSync.call(this)
|
||||
})
|
||||
const secondTurn = [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
await expect(ctx.sessionPersistence.append(header.id, secondTurn)).rejects.toThrow(/simulated Zstandard fsync failure/)
|
||||
expect(await readFile(path)).toEqual(before)
|
||||
spy.mockRestore()
|
||||
await ctx.sessionPersistence.append(header.id, secondTurn)
|
||||
expect((await ctx.sessionPersistence.load(header.id)).events).toEqual([...oneTurnLog(), ...secondTurn])
|
||||
})
|
||||
|
||||
it('skips empty, incomplete, and non-header compressed artifacts while rejecting malformed header frames', async () => {
|
||||
const root = await freshRoot()
|
||||
const bucket = sessionDir(root, undefined)
|
||||
await mkdir(bucket, { recursive: true })
|
||||
await writeFile(join(bucket, 'empty.jsonl.zstd'), '')
|
||||
await writeFile(join(bucket, 'partial.jsonl.zstd'), MAGIC)
|
||||
await writeFile(join(bucket, 'not-header.jsonl.zstd'), await compressZstdFrame('{"type":"turn/start"}\n'))
|
||||
const ctx = await mount(root)
|
||||
expect(await ctx.sessionPersistence.list()).toEqual([])
|
||||
|
||||
await writeFile(join(bucket, 'two-lines.jsonl.zstd'), await compressZstdFrame([
|
||||
JSON.stringify(toHeaderLine(meta('two-lines'))),
|
||||
JSON.stringify({ type: 'turn/start' }),
|
||||
'',
|
||||
].join('\n')))
|
||||
await expect(ctx.sessionPersistence.list()).rejects.toThrow(/first frame is not exactly one header line/)
|
||||
await expect(ctx.sessionPersistence.load(SessionId('two-lines')))
|
||||
.rejects.toThrow(/first frame is not exactly one header line/)
|
||||
})
|
||||
|
||||
it('rejects missing, empty, and checksum-corrupt header frames on targeted reads', async () => {
|
||||
const root = await freshRoot()
|
||||
const bucket = sessionDir(root, undefined)
|
||||
await mkdir(bucket, { recursive: true })
|
||||
await writeFile(logPath(root, undefined, SessionId('partial-only'), 'zstd'), MAGIC)
|
||||
await writeFile(logPath(root, undefined, SessionId('empty-header'), 'zstd'), await compressZstdFrame(''))
|
||||
const corruptHeader = Buffer.from(await compressZstdFrame(`${JSON.stringify(toHeaderLine(meta('bad-checksum')))}\n`))
|
||||
corruptHeader[corruptHeader.length - 1] = corruptHeader[corruptHeader.length - 1]! ^ 0xFF
|
||||
await writeFile(logPath(root, undefined, SessionId('bad-checksum'), 'zstd'), corruptHeader)
|
||||
const ctx = await mount(root)
|
||||
|
||||
await expect(ctx.sessionPersistence.load(SessionId('partial-only')))
|
||||
.rejects.toThrow(/empty or header-less Zstandard session log/)
|
||||
await expect(ctx.sessionPersistence.load(SessionId('empty-header')))
|
||||
.rejects.toThrow(/first frame is not exactly one header line/)
|
||||
await expect(ctx.sessionPersistence.list()).rejects.toThrow(/header frame failed validation/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceJsonl: encoding selection', () => {
|
||||
it('rejects roots owned by the opposite encoding in both directions', async () => {
|
||||
const rawRoot = await freshRoot('dsh-jsonl-raw-mismatch-')
|
||||
const raw = await mount(rawRoot, 'none')
|
||||
const rawHeader = meta('raw-log')
|
||||
await raw.sessionPersistence.create(rawHeader)
|
||||
await raw.sessionPersistence.append(rawHeader.id, oneTurnLog())
|
||||
const defaultBackend = await mount(rawRoot)
|
||||
await expect(defaultBackend.sessionPersistence.list()).rejects.toThrow(/configured for compression "zstd"/)
|
||||
|
||||
const zstdRoot = await freshRoot('dsh-jsonl-zstd-mismatch-')
|
||||
const zstd = await mount(zstdRoot)
|
||||
const zstdHeader = meta('zstd-log')
|
||||
await zstd.sessionPersistence.create(zstdHeader)
|
||||
await zstd.sessionPersistence.append(zstdHeader.id, oneTurnLog())
|
||||
const rawBackend = await mount(zstdRoot, 'none')
|
||||
await expect(rawBackend.sessionPersistence.list()).rejects.toThrow(/configured for compression "none"/)
|
||||
})
|
||||
|
||||
it('rechecks targeted artifacts and listing after an initially empty root', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
expect(await ctx.sessionPersistence.list()).toEqual([])
|
||||
|
||||
const loadHeader = meta('late-raw-load', '/late')
|
||||
await mkdir(sessionDir(root, loadHeader.cwd), { recursive: true })
|
||||
await writeFile(logPath(root, loadHeader.cwd, loadHeader.id, 'none'), [
|
||||
JSON.stringify(toHeaderLine(loadHeader)),
|
||||
...oneTurnLog().map(e => JSON.stringify(e)),
|
||||
'',
|
||||
].join('\n'))
|
||||
await expect(ctx.sessionPersistence.load(loadHeader.id)).rejects.toThrow(/uses \.jsonl/)
|
||||
await expect((ctx.sessionPersistence as SessionPersistenceJsonl).loadLive(loadHeader.id, loadHeader.cwd))
|
||||
.rejects.toThrow(/uses \.jsonl/)
|
||||
await expect(ctx.sessionPersistence.list()).rejects.toThrow(/uses \.jsonl/)
|
||||
})
|
||||
|
||||
it('refuses materialization when an opposite artifact appears after create', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
await ctx.sessionPersistence.list()
|
||||
const header = meta('late-raw-materialize', '/late')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await mkdir(sessionDir(root, header.cwd), { recursive: true })
|
||||
await writeFile(logPath(root, header.cwd, header.id, 'none'), [
|
||||
JSON.stringify(toHeaderLine(header)),
|
||||
...oneTurnLog().map(e => JSON.stringify(e)),
|
||||
'',
|
||||
].join('\n'))
|
||||
await expect(ctx.sessionPersistence.append(header.id, oneTurnLog())).rejects.toThrow(/uses \.jsonl/)
|
||||
expect((await readdir(sessionDir(root, header.cwd))).some(name => name.endsWith('.jsonl.zstd'))).toBe(false)
|
||||
})
|
||||
})
|
||||
@@ -22,6 +22,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../session-persistence/session-persistence"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,20 +1,24 @@
|
||||
# @deepseek-ai/dsh-session-persistence-sqlite
|
||||
|
||||
A SQLite durable session-persistence backend — a second `SessionPersistence` implementation ([session persistence](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md)), built to validate that the abstract seam and the shared `runPersistenceContract` suite are genuinely backend-agnostic. It satisfies the SAME contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes.
|
||||
A SQLite durable session-persistence backend — a second `SessionPersistence` implementation ([session persistence](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)), built to validate that the abstract seam and the shared `runPersistenceContract` suite are genuinely backend-agnostic. It satisfies the SAME contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes.
|
||||
|
||||
`locate(meta)` returns `undefined`: all sessions share one database, so there is no honest independent per-session transcript path.
|
||||
|
||||
> **TODO:** this backend talks to `node:sqlite` directly. If a cordis database service (`cordis/db` / a `@cordisjs` SQL driver plugin) is adopted, route through that instead of holding a raw `DatabaseSync` here — the contract surface (`SessionPersistence`) would not change, only the storage driver.
|
||||
|
||||
## Storage model
|
||||
|
||||
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../docs/rfc/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`), a per-materialization incarnation id, and a monotonic per-log revision live in a `sessions` row; a singleton state row carries the immutable store id. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row).
|
||||
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../.agents/notes/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`), a per-materialization incarnation id, and a monotonic per-log revision live in a `sessions` row; a singleton state row carries the immutable store id. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row).
|
||||
|
||||
The repository's Node range supports unflagged `node:sqlite`. The database enables foreign keys and uses the configured journal mode (`wal` by default; use a rollback mode where WAL shared-memory files are unsuitable). `PRAGMA user_version` stores the table-layout version; databases with any other version are rejected because this unreleased format has no migrations.
|
||||
|
||||
On filesystems with POSIX modes, the backend requests mode `0700` for missing directories and exclusively creates a missing database with mode `0600` before SQLite opens it; the process umask may further restrict both. New WAL, shared-memory, and persistent rollback-journal sidecars receive the database's resulting owner-only mode. Existing directories, database files, and sidecars keep their modes; filesystem setup errors other than an existing database fail initialization. These defaults prevent incidental exposure through a permissive process umask, but do not protect database confidentiality or integrity when another principal can replace the database entry in its parent directory.
|
||||
|
||||
## Contract semantics over rows
|
||||
|
||||
- **Append = a transaction.** `append` runs `BEGIN`/`COMMIT` around the batch: it materializes the `sessions` row (if still lazy) and INSERTs every event, asserting the contiguous-seq contract first (the first event's `seq` must equal the stored next-seq). A mid-batch failure (a UNIQUE violation on a duplicated seq) rolls back entirely, so the stored log and the in-memory cursor stay consistent. (`load()` already balanced the stored log, so `append` never has to repair a crash tail.)
|
||||
- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `list()` (which reports exactly the sessions that have a row).
|
||||
- **Interrupted-turn close on load.** `load()` implements the shared [crash-recovery contract](../../../docs/rfc/implemented/architecture/2026-06-14-session-persistence.md): preserve the valid interrupted turn, append its synthetic closing events in one transaction, and remove only a torn tail row. Committed parse errors or sequence gaps make the session unloadable. Because recovery mutates stored rows, the next append starts from a balanced log and accurate cursor.
|
||||
- **Interrupted-turn close on load.** `load()` implements the shared [crash-recovery contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md): preserve the valid interrupted turn, append its synthetic closing events in one transaction, and remove only a torn tail row. Committed parse errors or sequence gaps make the session unloadable. Because recovery mutates stored rows, the next append starts from a balanced log and accurate cursor.
|
||||
- **Lightweight revisions.** `listSnapshots()` combines the immutable store and database-file identity, a per-materialization incarnation id, and a per-session counter incremented in each mutating transaction. This keeps unchanged observations stable without parsing event rows and distinguishes independent stores and recreated same-id logs.
|
||||
|
||||
## Configuration (schemastery)
|
||||
@@ -34,9 +38,17 @@ Like the JSONL backend, the plugin also installs the `session/event` → buffer
|
||||
|
||||
### Resumed conversation history
|
||||
|
||||
**What the model sees**: SQLite storage contributes no live prompt or schema. Loading restores the same surface history as JSONL and preserves prior headers for reconstruction; the new loop composes its current envelope. Each unanswered call in interrupted rows is balanced with the exact error text `Tool call interrupted by a crash; no result was recorded.` Row metadata and raw chunks are not messages.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Zero live-request tokens. Resume restores retained history and pays the current envelope, plus the quoted repair result for each interrupted call.
|
||||
SQLite storage contributes no live prompt or schema. Loading restores the same surface history as JSONL and preserves prior headers for reconstruction; the new loop composes its current envelope. Recovery balances an assistant request without a durable call with `TOOL_NOT_STARTED`; a durable call without a result becomes `TOOL_OUTCOME_UNKNOWN`, which tells the model to retry only read-only or idempotent work and to verify possible side effects or ask the user. Row metadata and raw chunks are not messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Zero live-request tokens. Resume restores retained history and pays the current envelope, plus the quoted repair result for each interrupted call.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
SQLite storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
@@ -11,17 +11,23 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
@@ -30,6 +36,7 @@
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
/**
|
||||
* SQLite durable session-persistence backend. It maps each session header and
|
||||
* event to rows, and delegates write-path orchestration to
|
||||
* {@link PersistenceCoordinator}.
|
||||
* {@link PersistenceCoordinator}. It has no independent per-session artifact,
|
||||
* so its locator returns `undefined`.
|
||||
* @module @deepseek-ai/dsh-session-persistence-sqlite
|
||||
*/
|
||||
|
||||
@@ -10,11 +11,12 @@ import z from 'schemastery'
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { statSync } from 'node:fs'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { mkdir } from 'node:fs/promises'
|
||||
import { mkdir, open } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import {
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator,
|
||||
type PersistenceBackend, type SessionPersistenceSnapshot, type StoredPrefix,
|
||||
type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot,
|
||||
type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { SessionEvent, SurfaceEventType, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
@@ -36,12 +38,32 @@ function surfaceBindings(event: SessionEvent): [string | null, string | null] {
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* Exclusively create a missing database file with owner-only permissions.
|
||||
* Existing files retain their modes, and errors other than `EEXIST` propagate.
|
||||
* `DatabaseSync` reopens by path, so this does not protect confidentiality or
|
||||
* integrity when another principal can replace the database entry in its parent
|
||||
* directory.
|
||||
*/
|
||||
async function createDatabaseFile(path: string): Promise<void> {
|
||||
try {
|
||||
const handle = await open(path, 'wx', 0o600)
|
||||
await handle.close()
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error
|
||||
}
|
||||
}
|
||||
|
||||
/** Plugin configuration. */
|
||||
export interface Config {
|
||||
/**
|
||||
* Filesystem path to the SQLite database file. The special value `:memory:`
|
||||
* opens an in-process database (tests); a file path is created (with parent
|
||||
* dirs) on construction.
|
||||
* opens an in-process database (tests). On filesystems with POSIX modes,
|
||||
* missing directories and databases are created owner-only; existing path
|
||||
* modes are preserved. Filesystem setup errors other than an existing database
|
||||
* fail initialization. The backend does not protect confidentiality or
|
||||
* integrity when another principal can replace the database entry in its
|
||||
* parent directory.
|
||||
*/
|
||||
path: string
|
||||
/**
|
||||
@@ -88,7 +110,10 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
|
||||
private async openDb(path: string, journalMode: JournalMode): Promise<void> {
|
||||
const actual = path === ':memory:' ? path : resolve(path)
|
||||
if (actual !== ':memory:') await mkdir(dirname(actual), { recursive: true, mode: 0o700 })
|
||||
if (actual !== ':memory:') {
|
||||
await mkdir(dirname(actual), { recursive: true, mode: 0o700 })
|
||||
await createDatabaseFile(actual)
|
||||
}
|
||||
this.db = openDatabase(actual, journalMode)
|
||||
try {
|
||||
const row = this.db.prepare(
|
||||
@@ -115,6 +140,11 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
|
||||
// --- SessionPersistence service surface (delegated to the coordinator) ---
|
||||
|
||||
/** SQLite has one database, not an independent local artifact per session. */
|
||||
locate(_meta: SessionHeader): SessionLocation | undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(meta)
|
||||
}
|
||||
@@ -263,14 +293,15 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
private writeRow(meta: SessionHeader): void {
|
||||
this.db.prepare(`
|
||||
INSERT INTO sessions
|
||||
(id, version, created_at, cwd, parent_session, seed_length, incarnation, revision)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, 0)
|
||||
(id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, 0)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
version = excluded.version,
|
||||
created_at = excluded.created_at,
|
||||
cwd = excluded.cwd,
|
||||
parent_session = excluded.parent_session,
|
||||
seed_length = excluded.seed_length
|
||||
seed_length = excluded.seed_length,
|
||||
delegation_depth = excluded.delegation_depth
|
||||
`).run(
|
||||
meta.id,
|
||||
meta.version,
|
||||
@@ -278,6 +309,7 @@ export class SessionPersistenceSqlite extends SessionPersistence implements Pers
|
||||
meta.cwd ?? null,
|
||||
meta.parentSession ?? null,
|
||||
meta.seedLength ?? null,
|
||||
meta.delegationDepth ?? null,
|
||||
randomUUID(),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-persistence-sqlite`.
|
||||
* @module @deepseek-ai/dsh-session-persistence-sqlite/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-persistence-sqlite-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: persistence correctness requires backend round-trip and crash-tail tests;
|
||||
* this package exposes no continuously observable in-process relation.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -17,7 +17,7 @@ import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepsee
|
||||
* layout; orthogonal to a session's own `version` (which versions the EVENT
|
||||
* vocabulary, stored per session in the `sessions` row).
|
||||
*/
|
||||
export const SCHEMA_VERSION = 7
|
||||
export const SCHEMA_VERSION = 8
|
||||
|
||||
/**
|
||||
* A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}).
|
||||
@@ -37,6 +37,7 @@ export interface SessionRow {
|
||||
incarnation: string
|
||||
/** Monotonic log-change token incremented in each mutating transaction. */
|
||||
revision: number
|
||||
delegation_depth: number | null
|
||||
}
|
||||
|
||||
/** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */
|
||||
@@ -107,11 +108,12 @@ function configureDatabase(db: DatabaseSync, path: string, journalMode: JournalM
|
||||
id TEXT PRIMARY KEY,
|
||||
version INTEGER NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
cwd TEXT,
|
||||
parent_session TEXT,
|
||||
seed_length INTEGER,
|
||||
incarnation TEXT NOT NULL,
|
||||
revision INTEGER NOT NULL
|
||||
cwd TEXT,
|
||||
parent_session TEXT,
|
||||
seed_length INTEGER,
|
||||
delegation_depth INTEGER,
|
||||
incarnation TEXT NOT NULL,
|
||||
revision INTEGER NOT NULL
|
||||
) STRICT
|
||||
`)
|
||||
db.exec(`
|
||||
@@ -141,6 +143,7 @@ export function rowToMeta(row: SessionRow): SessionHeader {
|
||||
...row.cwd !== null ? { cwd: row.cwd } : {},
|
||||
...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {},
|
||||
...row.seed_length !== null ? { seedLength: row.seed_length } : {},
|
||||
...row.delegation_depth !== null ? { delegationDepth: row.delegation_depth } : {},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -188,8 +191,8 @@ export function scanRows(rows: readonly EventRow[]): { preserved: SessionEvent[]
|
||||
}
|
||||
})
|
||||
|
||||
// The last index that is a valid `turn/end` — the last fully-committed
|
||||
// boundary (the loop flushes only at turn/end).
|
||||
// The last index that is a valid `turn/end` — holes through a closed turn
|
||||
// are always committed corruption.
|
||||
let lastTurnEnd = -1
|
||||
for (let i = parsed.length - 1; i >= 0; i--) {
|
||||
if (parsed[i]?.ok && rows[i]?.type === 'turn/end') { lastTurnEnd = i; break }
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { mkdtemp, rm, symlink } from 'node:fs/promises'
|
||||
import { chmod, mkdtemp, rm, stat, symlink, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { dirname, join } from 'node:path'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SurfaceEvent, SurfaceEventType } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
@@ -14,17 +14,15 @@ import { runCoordinatorContract, type CoordinatorFixture } from '../../session-p
|
||||
const dirs: string[] = []
|
||||
afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) })
|
||||
|
||||
async function expectParallelFlushError(promise: Promise<unknown>, message: RegExp): Promise<void> {
|
||||
async function expectFlushError(promise: Promise<unknown>, message: RegExp): Promise<void> {
|
||||
try {
|
||||
await promise
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(AggregateError)
|
||||
const [cause] = (error as AggregateError).errors as unknown[]
|
||||
expect(cause).toBeInstanceOf(Error)
|
||||
expect((cause as Error).message).toMatch(message)
|
||||
expect(error).toBeInstanceOf(Error)
|
||||
expect((error as Error).message).toMatch(message)
|
||||
return
|
||||
}
|
||||
throw new Error('expected parallel flush to reject')
|
||||
throw new Error('expected flush to reject')
|
||||
}
|
||||
|
||||
async function freshDbPath(): Promise<string> {
|
||||
@@ -153,6 +151,48 @@ describe('scanRows', () => {
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
it('rejects a stored v0 log containing a legacy request/header-delta event', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('legacy-header-delta', '/legacy')
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)')
|
||||
.run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-delta')
|
||||
const insert = db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
insert.run(m.id, 0, 'turn/start', 1, JSON.stringify({ turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }))
|
||||
insert.run(m.id, 1, 'request/header-delta', 2, JSON.stringify({ config: { model: 'legacy' } }))
|
||||
insert.run(m.id, 2, 'turn/end', 3, JSON.stringify({ turn: 1, reason: { kind: 'completed' } }))
|
||||
db.close()
|
||||
|
||||
const mounted = await backend(path)
|
||||
await expect(mounted.ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1/)
|
||||
await mounted.dispose()
|
||||
})
|
||||
|
||||
it('rejects a stored v0 full header carrying the legacy fallback reason', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('legacy-header-fallback', '/legacy')
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)')
|
||||
.run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-fallback')
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
.run(m.id, 0, 'request/header', 1, JSON.stringify({
|
||||
header: { config: { model: 'legacy' } },
|
||||
reason: 'fallback',
|
||||
}))
|
||||
db.close()
|
||||
|
||||
const mounted = await backend(path)
|
||||
await expect(mounted.ctx.sessionPersistence.load(m.id))
|
||||
.rejects.toThrow(/unsupported legacy request\/header reason "fallback" at seq 0/)
|
||||
await mounted.dispose()
|
||||
})
|
||||
|
||||
it('has no independent per-session log location', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
expect(ctx.sessionPersistence.locate(meta('sqlite-location'))).toBeUndefined()
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('crash')
|
||||
@@ -402,7 +442,7 @@ describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
})
|
||||
|
||||
it('exposes the schema version constant', () => {
|
||||
expect(SCHEMA_VERSION).toBe(7)
|
||||
expect(SCHEMA_VERSION).toBe(8)
|
||||
})
|
||||
|
||||
it('keeps the revision stable for an empty repair hook', async () => {
|
||||
@@ -429,6 +469,61 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
await expect(b.dispose()).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('creates a new database and WAL sidecars with owner-only modes without changing its parent mode', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
const dir = dirname(path)
|
||||
await chmod(dir, 0o755)
|
||||
|
||||
const b = await backend(path)
|
||||
await b.ctx.sessionPersistence.list()
|
||||
|
||||
expect((await stat(dir)).mode & 0o777).toBe(0o755)
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-wal`)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-shm`)).mode & 0o777).toBe(0o600)
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('creates a persistent rollback journal with owner-only mode', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path, journalMode: 'persist' })
|
||||
const m = meta('persist-permissions')
|
||||
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-journal`)).mode & 0o777).toBe(0o600)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('preserves the mode of an existing database file', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
await writeFile(path, '', { mode: 0o644 })
|
||||
await chmod(path, 0o644)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path, journalMode: 'delete' })
|
||||
await ctx.sessionPersistence.list()
|
||||
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o644)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('surfaces an invalid database path during pre-creation', async () => {
|
||||
const path = await freshDbPath()
|
||||
const b = await backend(`${path}\0`)
|
||||
|
||||
await expect(b.ctx.sessionPersistence.list()).rejects.toMatchObject({ code: 'ERR_INVALID_ARG_VALUE' })
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('rollback-insert')
|
||||
@@ -462,7 +557,9 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
const walPath = await freshDbPath()
|
||||
const bWal = await backend(walPath)
|
||||
await bWal.ctx.sessionPersistence.create(meta('jm-wal'))
|
||||
expect((openDatabase(walPath, 'wal').prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('wal')
|
||||
const probe = openDatabase(walPath, 'wal')
|
||||
expect((probe.prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('wal')
|
||||
probe.close()
|
||||
await bWal.dispose()
|
||||
|
||||
const deletePath = await freshDbPath()
|
||||
@@ -486,7 +583,7 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
const b1 = await backend(path)
|
||||
const s1 = b1.ctx.sessions.create(SessionId('hmr-collide'))
|
||||
appendLog(s1, oneTurnLog())
|
||||
await b1.ctx.parallel('session/flush', s1)
|
||||
await b1.ctx.sessions.flush(s1)
|
||||
await b1.dispose()
|
||||
|
||||
// A fresh context with an UNRELATED live session reusing the id meets a
|
||||
@@ -497,9 +594,9 @@ describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId('hmr-collide'))
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('turn/start', { turn: 9, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
await expectParallelFlushError(ctx.parallel('session/flush', session), /id collision/)
|
||||
await expectFlushError(ctx.sessions.flush(session), /id collision/)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
@@ -551,18 +648,20 @@ describe('surface field round-trip', () => {
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const session = ctx.sessions.create(SessionId('roundtrip-surface'))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('assistant/message', { turn: 1, step: 1, content: [] }, { surfaceOp: 'append', sourceEventSeqs: [0] })
|
||||
session.append('assistant/message', { provenance: { provider: 'mock', model: 'mock' }, turn: 1, step: 1, content: [] }, { surfaceOp: 'append', sourceEventSeqs: [2] })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('roundtrip-surface'))
|
||||
expect(loaded.events).toHaveLength(4)
|
||||
const um = loaded.events[1]!
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
const um = loaded.events[2]!
|
||||
expect((um as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((um as SurfaceEvent).sourceEventSeqs).toBeUndefined()
|
||||
const am = loaded.events[2]!
|
||||
const am = loaded.events[3]!
|
||||
expect((am as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((am as SurfaceEvent).sourceEventSeqs).toEqual([0])
|
||||
expect((am as SurfaceEvent).sourceEventSeqs).toEqual([2])
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
@@ -574,7 +673,7 @@ describe('surface field round-trip', () => {
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('steering/message', { turn: 1, content: [], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('surface-noseq'))
|
||||
expect((loaded.events[1]! as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((loaded.events[1]! as SurfaceEvent).sourceEventSeqs).toBeUndefined()
|
||||
|
||||
@@ -22,6 +22,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../session-persistence/session-persistence"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
# @deepseek-ai/dsh-session-persistence
|
||||
|
||||
The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
|
||||
The abstract durable session-persistence seam (`ctx.sessionPersistence`). Defines WHAT a persistence backend does — durably store, reload, and list sessions — without saying HOW. Mirrors the `dsh-bash` capability-seam template ([capability seams](../../../.agents/notes/implemented/architecture/2026-06-13-capability-seams.md)): an abstract service here, a concrete implementation in a sibling package, consumers that inject the interface.
|
||||
|
||||
The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage, seed boundary) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here.
|
||||
The persisted unit IS the existing `SessionEvent` (event-sourced model — the log is the single source of truth), so there is no parallel "persisted message" type. Metadata that is NOT replayable conversation state (format version, cwd, lineage, seed boundary, delegation depth) travels separately as `SessionHeader`, owned by `dsh-session` and re-exported here.
|
||||
|
||||
## Service API (`ctx.sessionPersistence`)
|
||||
|
||||
| Method | Contract |
|
||||
|---|---|
|
||||
| `locate(meta): SessionLocation \| undefined` | Resolve an absolute per-session artifact target without I/O or materialization. Backends without an independent local artifact return `undefined`. |
|
||||
| `create(meta): Promise<void>` | Register a new session's metadata. MAY defer the physical write until the first `append` (lazy materialization). |
|
||||
| `append(id, events): Promise<void>` | Durably persist a batch (from the `session/flush` drain). Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
|
||||
| `load(id): Promise<{ meta; events }>` | Reload meta + log. Preserves an interrupted (unclosed) final turn and closes it with synthetic closers — an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}` (a turn can be huge — never truncated); only a torn tail fragment is dropped. Events contiguous (`events[i].seq === i`); rejects a committed-region gap/parse error or unknown `version`. |
|
||||
@@ -16,14 +17,20 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
|
||||
## Invariants every backend must honor
|
||||
|
||||
- **Append-only; a crashed turn is closed, not truncated.** Committed events (at or below a flushed `turn/end`) are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (an error `tool/result` per unanswered `tool-call`, then `step/end?`+`turn/end {interrupted}`) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded.
|
||||
- **Append-only; a crashed turn is closed, not truncated.** Flushed events are never rewritten. A crash can leave an unclosed final turn whose events are real and possibly large; `load` preserves them and durably appends synthetic closers (a risk-classified error `tool/result` per unanswered assistant call, then `step/end?`+`turn/end {interrupted}`) to balance the log and keep the rehydrated history a valid provider transcript. Only a never-fully-written torn tail fragment is discarded.
|
||||
- **Contiguous seq.** `load` rejects a `seq` gap/parse error in the MIDDLE of the log; `append`'s first `seq` must equal the stored next-seq.
|
||||
- **JSON-serializable data.** `append` materializes each direct/replay batch through the shared one-pass lossless-JSON boundary. Live `Session` events are already deep-frozen, but the write coordinator still copies each event into a persistence-owned buffer.
|
||||
- **Durability.** `append` returns only once the batch is durable.
|
||||
|
||||
## The write coordinator
|
||||
|
||||
`PersistenceCoordinator` owns per-id state, write-behind buffers and serialization, the `session/event` → `session/flush` drain, lazy materialization, crash-tail repair, session adoption, and quiescent disposal. A first-party backend composes one, implements the small `PersistenceBackend` storage hook interface, and delegates its stateful methods. Lightweight snapshot listing remains backend-owned because revisions identify the underlying store; see the [coordinator RFC](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
`PersistenceCoordinator` owns per-id state, write-behind buffers and serialization, the `session/event` → `session/flush` drain, lazy materialization, crash-tail repair, session adoption, and quiescent disposal. A first-party backend composes one, implements the small `PersistenceBackend` storage hook interface, and delegates its stateful methods. JSONL and SQLite therefore share lifecycle correctness while retaining different storage primitives. Side-effect-free location queries and lightweight snapshot listing remain backend-owned because they describe storage topology and revision identity; see the [coordinator Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
The coordinator implements durability at checkpoints but does not select their schedule. Persisted deployments explicitly compose [`dsh-session-checkpoint-policy`](../session-checkpoint-policy) when they want request-, tool-dispatch-, and completed-step recovery boundaries; omitting it leaves the loop's coarser checkpoints intact.
|
||||
|
||||
When a live session emits `session/disposed`, the coordinator waits for its initialization, serializes a final buffer drain, then releases every map entry owned by that exact `Session` object. A failed final drain keeps the pending buffer for backend teardown to retry. Backend teardown stops event admission first, awaits all in-flight session retirements and remaining per-id operations, drains any retained buffers, and only then closes the storage handle.
|
||||
|
||||
The side-effect-free `locate` query remains backend-owned because it describes storage topology rather than write orchestration.
|
||||
|
||||
The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinator and storage):
|
||||
|
||||
@@ -37,7 +44,7 @@ The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinato
|
||||
| `list()` | List all stored metadata. |
|
||||
| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. |
|
||||
|
||||
The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). A third-party backend MAY implement the abstract service directly without the coordinator, but it must also provide trustworthy lightweight snapshot revisions. See [the write-coordinator RFC](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
The `tornMarker` is fully OPAQUE: the coordinator only tests `!== undefined` and round-trips it to `commitRepair`, never inspecting its value (the JSONL backend uses the byte offset to truncate to, the SQLite backend the seq to delete from). A third-party backend MAY implement the abstract service directly without the coordinator, but it must also provide trustworthy lightweight snapshot revisions. See [the write-coordinator Agent Note](../../../.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
## Testing backends
|
||||
|
||||
@@ -45,17 +52,25 @@ Import `runPersistenceContract` from `tests/contract.ts` (the public API, includ
|
||||
|
||||
Three backends run these suites: an in-memory reference (in `tests/`), `dsh-session-persistence-jsonl` (append-only file log) and `dsh-session-persistence-sqlite` (`node:sqlite`, each `SessionEvent` one row `(session_id, seq, type, time, data, source_event_seqs, surface_op)`). All passing the same contract + coordinator suite is the proof that the seam is genuinely backend-agnostic — lazy materialization, crash-tail-on-load, and contiguous-seq hold identically over file bytes and over a transactional store.
|
||||
|
||||
## Metadata types
|
||||
## Metadata and location types
|
||||
|
||||
Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`, `seedLength?`).
|
||||
Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`, `seedLength?`, `delegationDepth?`). `SessionLocation` is `{ readonly kind: string; readonly path: string }`; its path is an absolute backend target, not proof that the artifact exists or contains an unflushed turn.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Resumed conversation history
|
||||
|
||||
**What the model sees**: This seam adds no prompt or schema. Resume restores stored surface events as message history; stored request headers reconstruct earlier calls, while the new loop composes the current system prompt, tools, and session prefix for its next request. Crash repair inserts exactly `Tool call interrupted by a crash; no result was recorded.` as the error result for each unanswered tool call.
|
||||
#### What the model sees
|
||||
|
||||
**Token effect**: Zero tokens during ordinary persistence. Resume restores retained history cost and pays the current request envelope normally; each repaired call adds the quoted retained error text.
|
||||
This seam adds no prompt or schema. Resume restores stored surface events as message history; stored request headers reconstruct earlier calls, while the new loop composes the current system prompt, tools, and session prefix for its next request. Crash repair marks an assistant request without a durable call as `TOOL_NOT_STARTED`; a durable call without a result becomes `TOOL_OUTCOME_UNKNOWN`, whose text lets the model retry read-only or idempotent work but directs it to verify side effects or ask the user instead of retrying blindly.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Zero tokens during ordinary persistence. Resume restores retained history cost and pays the current request envelope normally; each repaired call adds the quoted retained error text.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Persistence does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append without rewriting earlier history.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
|
||||
@@ -11,11 +11,16 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
@@ -23,11 +28,14 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-scope": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
|
||||
@@ -118,6 +118,25 @@ function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly Sessio
|
||||
})
|
||||
}
|
||||
|
||||
/** Reject events from an obsolete v0 vocabulary that this build cannot replay. */
|
||||
function assertSupportedEvents(events: readonly SessionEvent[], id: SessionId): void {
|
||||
const legacyType: string = 'request/header-delta'
|
||||
const legacy = events.find(event => event.type === legacyType)
|
||||
if (legacy !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header-delta event at seq ${legacy.seq}`)
|
||||
}
|
||||
const legacyModeType: string = 'mode/set'
|
||||
const legacyMode = events.find(event => event.type === legacyModeType)
|
||||
if (legacyMode !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy mode/set event at seq ${legacyMode.seq}`)
|
||||
}
|
||||
const fallback = events.find(event => event.type === 'request/header'
|
||||
&& (event.data as { reason?: string }).reason === 'fallback')
|
||||
if (fallback !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header reason "fallback" at seq ${fallback.seq}`)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the backend-agnostic session write-path orchestration. A backend
|
||||
* constructs one (`new PersistenceCoordinator(ctx, this)`), implements
|
||||
@@ -126,7 +145,8 @@ function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly Sessio
|
||||
*
|
||||
* All per-id operations are serialized (a per-id promise chain) so concurrent
|
||||
* flushes / a flush racing a load never interleave storage writes. The
|
||||
* constructor installs the write-path listeners and the dispose effect.
|
||||
* constructor installs the write-path listeners, per-session retirement, and
|
||||
* the backend dispose effect.
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token.
|
||||
*/
|
||||
@@ -146,6 +166,8 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
* observation boundary; callers do not inspect this bookkeeping directly.
|
||||
*/
|
||||
private inits = new Map<Session, Promise<void>>()
|
||||
/** Final drains started by fire-and-forget session disposal notifications. */
|
||||
private retirements = new Set<Promise<void>>()
|
||||
|
||||
constructor(private ctx: Context, private backend: PersistenceBackend<TornMarker>) {
|
||||
this.installWritePath()
|
||||
@@ -204,6 +226,11 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
}
|
||||
|
||||
private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
// Every append route converges here: the public service, live write-behind
|
||||
// drains, and HMR seed/suffix adoption. Keep vocabulary rejection at that
|
||||
// shared boundary so a stale JavaScript plugin cannot persist an event that
|
||||
// this same backend will refuse to load.
|
||||
assertSupportedEvents(events, id)
|
||||
if (events.length === 0) return
|
||||
let state = this.states.get(id)
|
||||
if (state === undefined) state = await this.adopt(id) // calls loadCore, not load
|
||||
@@ -238,6 +265,7 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
if (stored === undefined) throw new Error(`session "${id}" not found`)
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
assertSupportedEvents(events, id)
|
||||
|
||||
// Preserve complete interrupted events and synthesize only missing closers.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
@@ -267,7 +295,13 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
const next = prior.then(op, op)
|
||||
// Keep the chain alive but swallow this op's rejection for the NEXT waiter
|
||||
// (the caller still sees the real rejection via `next`).
|
||||
this.chains.set(id, next.then(() => undefined, () => undefined))
|
||||
const tail = next.then(() => undefined, () => undefined)
|
||||
this.chains.set(id, tail)
|
||||
// Settled tails carry no serialization value. Delete only the exact tail
|
||||
// installed above: a later operation may already have replaced it.
|
||||
void tail.then(() => {
|
||||
if (this.chains.get(id) === tail) this.chains.delete(id)
|
||||
})
|
||||
return next
|
||||
}
|
||||
|
||||
@@ -293,27 +327,12 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
private installWritePath(): void {
|
||||
const ctx = this.ctx
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the init
|
||||
// promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Session emits an owned frozen event. Keep a persistence-owned copy anyway
|
||||
// so the write-behind queue owns exactly the record it will flush rather than
|
||||
// retaining a product-layer record by identity. Serializability is guaranteed
|
||||
// at the source, so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Dispose must reach quiescence: await every init + final drain BEFORE
|
||||
// returning, then close the backend's own resources (AFTER the drain), so no
|
||||
// write lands after teardown and a close failure never MASKS a drain error.
|
||||
// Register the disposer BEFORE the listeners. Cordis tears effects down in
|
||||
// reverse registration order, so event admission closes before this final
|
||||
// drain reaches quiescence and closes the backend.
|
||||
ctx.effect(() => async () => {
|
||||
await this.awaitRetirements()
|
||||
|
||||
let disposeError: unknown
|
||||
try {
|
||||
const errors = [
|
||||
@@ -341,11 +360,63 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
}
|
||||
}, `${this.backend.name} write path`)
|
||||
|
||||
// Capture the header on creation; persist a fork's seed once. Record the init
|
||||
// promise so flush/dispose can await it (onCreated is async).
|
||||
ctx.on('session/created', (session) => { void this.initFor(session) })
|
||||
|
||||
// Session emits an owned frozen event. Keep a persistence-owned copy anyway
|
||||
// so the write-behind queue owns exactly the record it will flush rather than
|
||||
// retaining a product-layer record by identity. Serializability is guaranteed
|
||||
// at the source, so structuredClone is safe.
|
||||
ctx.on('session/event', (session, event) => {
|
||||
let buffer = this.buffers.get(session)
|
||||
if (!buffer) this.buffers.set(session, buffer = [])
|
||||
buffer.push(structuredClone(event))
|
||||
})
|
||||
|
||||
// Drain to the backend at the durability checkpoint.
|
||||
ctx.on('session/flush', session => this.flush(session))
|
||||
|
||||
// Session disposal is observe-only, so the coordinator observes the
|
||||
// detached task itself and backend teardown awaits quiescence.
|
||||
ctx.on('session/disposed', (session) => { this.retire(session) })
|
||||
|
||||
// HMR: a hot reload does not replay session/created, so seed existing live
|
||||
// sessions (mirrors dsh-invariants).
|
||||
for (const session of ctx.sessions.list()) void this.initFor(session)
|
||||
}
|
||||
|
||||
/** Start, observe, and track one disposed session's final drain. */
|
||||
private retire(session: Session): void {
|
||||
const task = this.retireCore(session)
|
||||
this.retirements.add(task)
|
||||
const settled = (): void => { this.retirements.delete(task) }
|
||||
void task.then(settled, (error: unknown) => {
|
||||
settled()
|
||||
this.ctx.logger.warn(`${this.backend.name}: session "${session.id}" retirement failed: ${String(error)}`)
|
||||
})
|
||||
}
|
||||
|
||||
/** Drain and release state owned by one exact disposed Session lifecycle. */
|
||||
private async retireCore(session: Session): Promise<void> {
|
||||
await this.inits.get(session)
|
||||
|
||||
const id = session.header.id
|
||||
await this.serialize(id, async () => {
|
||||
await this.drain(session)
|
||||
this.buffers.delete(session)
|
||||
this.inits.delete(session)
|
||||
if (this.states.get(id)?.owner === session) this.states.delete(id)
|
||||
})
|
||||
}
|
||||
|
||||
/** Await every retirement admitted before listener teardown. */
|
||||
private async awaitRetirements(): Promise<void> {
|
||||
while (this.retirements.size > 0) {
|
||||
await Promise.allSettled([...this.retirements])
|
||||
}
|
||||
}
|
||||
|
||||
/** Start (once) the async init for a session and remember its promise. */
|
||||
private initFor(session: Session): Promise<void> {
|
||||
const existing = this.inits.get(session)
|
||||
@@ -463,6 +534,7 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix<TornMarker>): Promise<void> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
assertSupportedEvents(events, session.header.id)
|
||||
if (!seedCoversPrefix(seed, events)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
}
|
||||
|
||||
@@ -31,6 +31,18 @@ declare module 'cordis' {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A backend-resolved, per-session local artifact location. The path is an
|
||||
* absolute target path and can name an artifact that has not materialized yet.
|
||||
* Consumers must treat it as a location hint, never as an authorization token.
|
||||
*/
|
||||
export interface SessionLocation {
|
||||
/** Backend-specific artifact kind, for example `jsonl`. */
|
||||
readonly kind: string
|
||||
/** Absolute path to this session's backend-owned artifact. */
|
||||
readonly path: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Durable append-only session storage. Implementations preserve contiguous,
|
||||
* losslessly JSON-serializable events; {@link append} resolves only after
|
||||
@@ -42,6 +54,15 @@ export abstract class SessionPersistence extends Service {
|
||||
super(ctx, 'sessionPersistence')
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve this backend's independent local artifact for a session without
|
||||
* reading, creating, flushing, or otherwise materializing it. Backends such
|
||||
* as SQLite that do not own one artifact per session return `undefined`.
|
||||
* @param meta - the immutable session header whose artifact is requested.
|
||||
* @returns the backend-specific absolute location, when one exists.
|
||||
*/
|
||||
abstract locate(meta: SessionHeader): SessionLocation | undefined
|
||||
|
||||
/**
|
||||
* Register a new session's metadata. A backend MAY defer the physical write
|
||||
* until the first {@link append} (lazy materialization), in which case a
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-persistence`.
|
||||
* @module @deepseek-ai/dsh-session-persistence/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-persistence'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-persistence-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: persistence correctness requires backend round-trip and crash-tail tests;
|
||||
* this package exposes no continuously observable in-process relation.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -9,8 +9,8 @@
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader, SurfaceEventType, SurfaceIntent } from '@deepseek-ai/dsh-session'
|
||||
import { SESSION_FORMAT_VERSION, Session, SessionId, TOOL_NOT_STARTED, TOOL_OUTCOME_UNKNOWN } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader, SurfaceEventType, SurfaceIntent } from '@deepseek-ai/dsh-session'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { SessionPersistence } from '../src/index.ts'
|
||||
|
||||
@@ -36,7 +36,7 @@ export function oneTurnLog(): SessionEvent[] {
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, surfaceOp: 'append' },
|
||||
{ type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }] }, surfaceOp: 'append' },
|
||||
{ type: 'assistant/message', seq: 3, time: 4, data: { turn: 1, step: 1, content: [{ type: 'text', text: 'hello' }], provenance: { provider: 'mock', model: 'mock' } }, surfaceOp: 'append' },
|
||||
{ type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } },
|
||||
{ type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
@@ -127,7 +127,7 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
|
||||
}
|
||||
})
|
||||
|
||||
it('crash recovery: an interrupted tool call gets a synthetic error result so resume is a valid transcript', async () => {
|
||||
it('crash recovery: an unstarted assistant tool request gets a retryable synthetic result', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('interrupted-toolcall')
|
||||
@@ -141,7 +141,7 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 8, time: 9, data: { turn: 2, step: 1, content: [
|
||||
{ type: 'tool-call', id: CallId('call-x'), name: 'bash', arguments: '{}' },
|
||||
] } },
|
||||
], provenance: { provider: 'mock', model: 'mock' } } },
|
||||
])
|
||||
|
||||
const loaded = await persistence.load(m.id)
|
||||
@@ -154,7 +154,7 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
|
||||
])
|
||||
const synthetic = loaded.events.find(e => e.type === 'tool/result')
|
||||
expect(synthetic?.type === 'tool/result' && synthetic.data).toMatchObject({
|
||||
callId: CallId('call-x'), isError: true, error: { code: 'interrupted' },
|
||||
callId: CallId('call-x'), isError: true, error: { code: TOOL_NOT_STARTED },
|
||||
})
|
||||
// The synthetic result carries the SAME callId as the orphaned tool-call,
|
||||
// so deriveMessages() pairs them — no provider-invalid dangling call.
|
||||
@@ -167,6 +167,40 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
|
||||
}
|
||||
})
|
||||
|
||||
it('crash recovery: a recorded tool call with no result tells the model to assess retry risk', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
const m = meta('unknown-tool-outcome')
|
||||
await persistence.create(m)
|
||||
await persistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } } },
|
||||
{ type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } },
|
||||
{ type: 'assistant/message', seq: 2, time: 3, data: { turn: 1, step: 1, content: [
|
||||
{ type: 'tool-call', id: CallId('call-risk'), name: 'write', arguments: '{}' },
|
||||
], provenance: { provider: 'mock', model: 'mock' } }, surfaceOp: 'append' },
|
||||
{ type: 'tool/call', seq: 3, time: 4, data: { turn: 1, step: 1, callId: CallId('call-risk'), name: 'write', arguments: '{}' } },
|
||||
])
|
||||
|
||||
const loaded = await persistence.load(m.id)
|
||||
const synthetic = loaded.events.find(e => e.type === 'tool/result')
|
||||
expect(synthetic?.type === 'tool/result' && synthetic.data.error).toEqual({
|
||||
name: 'ToolOutcomeUnknownError', code: TOOL_OUTCOME_UNKNOWN,
|
||||
})
|
||||
if (synthetic?.type !== 'tool/result' || synthetic.data.content[0]?.type !== 'text') {
|
||||
throw new Error('expected a text tool result')
|
||||
}
|
||||
expect(synthetic.data.content[0].text).toContain('retry only if the operation is read-only or idempotent')
|
||||
expect(synthetic.data.content[0].text).toContain('if it may have side effects, first verify external state or ask the user')
|
||||
const resumed = new Session(m.id, loaded.events, loaded.meta)
|
||||
const resumedResult = resumed.deriveMessages().find(message => message.content.some(block => block.type === 'tool-result'))
|
||||
expect(resumedResult?.content[0]).toMatchObject({
|
||||
type: 'tool-result', toolCallId: CallId('call-risk'), isError: true,
|
||||
})
|
||||
} finally {
|
||||
await dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('list() excludes a created-but-never-appended (zero-event) session', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
|
||||
@@ -9,8 +9,9 @@
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import { scopeTarget } from '@deepseek-ai/dsh-scope'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { meta, oneTurnLog, appendLog } from './contract.ts'
|
||||
@@ -76,7 +77,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('live'), { meta: { cwd: WORK } })
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('live'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
@@ -96,7 +97,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('forked-child'), { meta: { cwd: WORK, seedLength: 3 } })
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked-child'))
|
||||
expect(loaded.meta.seedLength).toBe(3)
|
||||
@@ -106,22 +107,43 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
}
|
||||
})
|
||||
|
||||
it('round-trips the delegation depth through persistence', async () => {
|
||||
// A subagent child's recursion budget lives in its header; a reload that
|
||||
// dropped it would reset the child to top-level and un-bound maxDepth
|
||||
// (JSONL stores it in the header line; SQLite uses `delegation_depth`).
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('delegated-child'), {
|
||||
meta: { cwd: WORK, parentSession: SessionId('root'), delegationDepth: 2 },
|
||||
})
|
||||
send(session, oneTurnLog())
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('delegated-child'))
|
||||
expect(loaded.meta.delegationDepth).toBe(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
}
|
||||
})
|
||||
|
||||
it('source-frozen events cannot be mutated after buffering and persist unchanged', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('mutate'), { meta: { cwd: WORK } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
const ev = session.append('user/message', { content: [{ type: 'text', text: 'original' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
expect(() => {
|
||||
;(ev.data as { content: { type: 'text'; text: string }[] }).content[0]!.text = 'HACKED'
|
||||
}).toThrow(TypeError)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('mutate'))
|
||||
const first = loaded.events[0]
|
||||
expect(first?.type === 'user/message' && (first.data.content[0] as { text: string }).text).toBe('original')
|
||||
const message = loaded.events.find(event => event.type === 'user/message')
|
||||
expect(message?.type === 'user/message' && (message.data.content[0] as { text: string }).text).toBe('original')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
@@ -166,7 +188,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(loaded.events).toEqual(seed)
|
||||
// A flush with no NEW events must not double-write.
|
||||
await ctx.parallel('session/flush', forked)
|
||||
await ctx.sessions.flush(forked)
|
||||
const reloaded = await ctx.sessionPersistence.load(SessionId('forked'))
|
||||
expect(reloaded.events).toEqual(seed)
|
||||
} finally {
|
||||
@@ -182,7 +204,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
try {
|
||||
const s1 = first.ctx.sessions.create(SessionId('resumed'), { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
await first.ctx.sessions.flush(s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
@@ -194,7 +216,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await second.ctx.sessions.flush(s2) // let onCreated adopt
|
||||
s2.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
s2.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await second.ctx.parallel('session/flush', s2)
|
||||
await second.ctx.sessions.flush(s2)
|
||||
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
@@ -212,13 +234,14 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await ctx.plugin(SessionStore)
|
||||
// A session exists BEFORE the persistence plugin is applied.
|
||||
const session = ctx.sessions.create(SessionId('pre-existing'), { meta: { cwd: WORK } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
const fiber = await fix.mount(ctx)
|
||||
try {
|
||||
// The plugin seeded it on apply; a subsequent flush persists its events.
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('pre-existing'))
|
||||
expect(loaded.events.length).toBeGreaterThanOrEqual(2)
|
||||
} finally {
|
||||
@@ -233,6 +256,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await fix.mount(ctx)
|
||||
const session = await liveSessionInFiber(ctx, 'drain', WORK)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'buffered' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
// No explicit flush — dispose must drain.
|
||||
@@ -261,7 +285,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the same storage while the
|
||||
// session stays live. The new instance has no coordinator state but must adopt the
|
||||
@@ -271,7 +295,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'again' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 2, reason: { kind: 'completed' } })
|
||||
await expect(ctx.parallel('session/flush', session)).resolves.not.toThrow()
|
||||
await expect(ctx.sessions.flush(session)).resolves.not.toThrow()
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-adopt'))
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
@@ -291,7 +315,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
const backend1 = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
// Append turn 2 to the LIVE session, then dispose instance 1 WITHOUT
|
||||
// flushing turn 2: it is now ONLY in the live session's events; the new
|
||||
@@ -303,7 +327,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
// Instance 2 adopts the stored prefix (turn 1) and MUST also persist the
|
||||
// live suffix (turn 2) carried in the session's events.
|
||||
await fix.mount(ctx)
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-suffix'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3])
|
||||
expect(loaded.events.filter(e => e.type === 'turn/start')).toHaveLength(2)
|
||||
@@ -322,7 +346,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
const first = await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
// Crash-tail a torn fragment past the (open) committed turn, then reload.
|
||||
await first.dispose()
|
||||
@@ -332,7 +356,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
// end. Adoption must truncate the torn tail but NOT synthesize closers.
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('hmr-open'))
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'step/start', 'step/end', 'turn/end'])
|
||||
@@ -352,7 +376,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
try {
|
||||
const s1 = first.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
await first.ctx.sessions.flush(s1)
|
||||
} finally {
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
@@ -392,7 +416,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await expect(ctx.sessions.flush(reuse)).resolves.toBeUndefined()
|
||||
reuse.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
reuse.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', reuse)
|
||||
await ctx.sessions.flush(reuse)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('abandoned'))
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1])
|
||||
} finally {
|
||||
@@ -401,7 +425,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
}
|
||||
})
|
||||
|
||||
it('does NOT reclaim an id whose abandoned owner still has buffered (unflushed) events', async () => {
|
||||
it('session disposal drains buffered events before retiring ownership', async () => {
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
@@ -413,13 +437,20 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
// Append a turn but do NOT flush — events sit in the write-behind buffer.
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await firstFiber.dispose() // disposed before flush; not materialized, buffer pending
|
||||
await firstFiber.dispose()
|
||||
|
||||
// Disposal is an observe-only notification. Poll storage rather than
|
||||
// assuming the owning fiber awaits the coordinator's detached drain.
|
||||
await vi.waitFor(async () => {
|
||||
expect((await ctx.sessionPersistence.list()).map(meta => meta.id)).toContain(SessionId('buffered'))
|
||||
})
|
||||
expect((await ctx.sessionPersistence.load(SessionId('buffered'))).events.map(event => event.seq)).toEqual([0, 1])
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create(SessionId('buffered'), { meta: { cwd: WORK } })
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(ctx.sessions.flush(reuse)).rejects.toThrow(/already bound to a different live session/)
|
||||
await expect(ctx.sessions.flush(reuse)).rejects.toThrow(/persisted log|id collision/)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
@@ -431,14 +462,15 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
const session = ctx.sessions.create(SessionId('idem'), { meta: { cwd: WORK } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
// Re-emit session/created for the SAME live session (idempotent initFor).
|
||||
ctx.emit('session/created', session)
|
||||
await ctx.parallel('session/flush', session)
|
||||
ctx.emit(scopeTarget(session, undefined), 'session/created', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('idem'))
|
||||
expect(loaded.events).toHaveLength(2) // not doubled
|
||||
expect(loaded.events).toHaveLength(3) // not doubled
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
@@ -683,11 +715,12 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
// async onCreated init has necessarily set state (exercises the
|
||||
// state-undefined cursor path).
|
||||
const session = ctx.sessions.create(SessionId('flush-nostate'), { meta: { cwd: WORK } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'q' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('flush-nostate'))
|
||||
expect(loaded.events).toHaveLength(2)
|
||||
expect(loaded.events).toHaveLength(3)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await fix.cleanup()
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import SessionStore, { SessionId, isJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator,
|
||||
type PersistenceBackend, type SessionPersistenceSnapshot, type StoredPrefix,
|
||||
@@ -12,9 +12,48 @@ import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-c
|
||||
/** The durable store shape: materialized sessions only (no lazy entries). */
|
||||
type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
/** An obsolete event fixture that emulates an untyped pre-change producer. */
|
||||
function legacyHeaderDelta(seq = 0): SessionEvent {
|
||||
return {
|
||||
type: 'request/header-delta',
|
||||
seq,
|
||||
time: 1,
|
||||
data: { config: { model: 'legacy' } },
|
||||
} as unknown as SessionEvent
|
||||
}
|
||||
|
||||
/** An unsupported named-mode fixture emulating an untyped producer. */
|
||||
function legacyModeSet(seq = 0): SessionEvent {
|
||||
return {
|
||||
type: 'mode/set',
|
||||
seq,
|
||||
time: 1,
|
||||
data: { mode: 'plan' },
|
||||
} as unknown as SessionEvent
|
||||
}
|
||||
|
||||
/** An obsolete full-header reason fixture from the removed delta codec. */
|
||||
function legacyFallbackHeader(seq = 0): SessionEvent {
|
||||
return {
|
||||
type: 'request/header',
|
||||
seq,
|
||||
time: 1,
|
||||
data: { header: { config: { model: 'legacy' } }, reason: 'fallback' },
|
||||
} as unknown as SessionEvent
|
||||
}
|
||||
|
||||
/** Optional plugin config: an EXTERNAL store shared across backend instances. */
|
||||
interface MemoryConfig { store?: MemoryStore }
|
||||
|
||||
/** Test-only view of the coordinator containers whose retirement is the contract under test. */
|
||||
interface CoordinatorInternals {
|
||||
states: Map<unknown, unknown>
|
||||
buffers: Map<unknown, unknown>
|
||||
chains: Map<unknown, unknown>
|
||||
inits: Map<unknown, unknown>
|
||||
retirements: Set<Promise<void>>
|
||||
}
|
||||
|
||||
/**
|
||||
* Reference {@link PersistenceCoordinator} vehicle and abstract-service coverage, backed by a
|
||||
* dependency-free map with atomic writes and no torn-tail marker. Supplying the map lets multiple
|
||||
@@ -41,6 +80,10 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
|
||||
// --- service surface (delegated to the coordinator) ---
|
||||
|
||||
locate(_meta: SessionHeader): undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
create(m: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(m)
|
||||
}
|
||||
@@ -104,6 +147,49 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
}
|
||||
}
|
||||
|
||||
/** Controllable storage primitive for serialization and retirement failure tests. */
|
||||
class ControlledBackend implements PersistenceBackend<never> {
|
||||
readonly name = 'session-persistence-controlled'
|
||||
readonly store: MemoryStore = new Map()
|
||||
readonly lifecycle: string[] = []
|
||||
appendAttempts = 0
|
||||
loadAttempts = 0
|
||||
beforeAppend?: (attempt: number) => Promise<void>
|
||||
beforeLoadStored?: (attempt: number) => Promise<void>
|
||||
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<never> | undefined> {
|
||||
await this.beforeLoadStored?.(++this.loadAttempts)
|
||||
const entry = this.store.get(id)
|
||||
if (entry === undefined) return undefined
|
||||
return { meta: structuredClone(entry.meta), events: structuredClone(entry.events) }
|
||||
}
|
||||
|
||||
loadLive(id: SessionId, _cwd: string | undefined): Promise<StoredPrefix<never> | undefined> {
|
||||
return this.loadStored(id)
|
||||
}
|
||||
|
||||
async appendBatch(m: SessionHeader, events: readonly SessionEvent[], _isMaterialized: boolean): Promise<void> {
|
||||
const attempt = ++this.appendAttempts
|
||||
await this.beforeAppend?.(attempt)
|
||||
const entry = this.store.get(m.id)
|
||||
if (entry === undefined) {
|
||||
this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] })
|
||||
} else {
|
||||
entry.events.push(...structuredClone(events) as SessionEvent[])
|
||||
}
|
||||
}
|
||||
|
||||
async commitRepair(_m: SessionHeader, _tornMarker: undefined, _closers: readonly SessionEvent[]): Promise<void> {}
|
||||
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
return [...this.store.values()].map(entry => structuredClone(entry.meta))
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
this.lifecycle.push('close')
|
||||
}
|
||||
}
|
||||
|
||||
// Run the shared contract against the in-memory backend.
|
||||
runPersistenceContract('memory', async () => {
|
||||
const ctx = new Context()
|
||||
@@ -125,6 +211,230 @@ runCoordinatorContract('memory', async (): Promise<CoordinatorFixture> => {
|
||||
}
|
||||
})
|
||||
|
||||
describe('PersistenceCoordinator retirement', () => {
|
||||
it('a retiring unmaterialized owner without buffered events releases its id', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const backendFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const loadGate = Promise.withResolvers<boolean>()
|
||||
|
||||
try {
|
||||
const id = SessionId('retiring-lazy-owner')
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create(id)
|
||||
}, { inject: ['sessions'] }))
|
||||
await ctx.sessions.flush(first)
|
||||
|
||||
const baselineLoads = backend.loadAttempts
|
||||
backend.beforeLoadStored = async () => { await loadGate.promise }
|
||||
const blockingLoad = coordinator.load(id)
|
||||
await vi.waitFor(() => { expect(backend.loadAttempts).toBe(baselineLoads + 1) })
|
||||
await firstFiber.dispose()
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create(id)
|
||||
}, { inject: ['sessions'] }))
|
||||
await vi.waitFor(() => { expect(backend.loadAttempts).toBe(baselineLoads + 2) })
|
||||
|
||||
loadGate.resolve(true)
|
||||
await expect(blockingLoad).rejects.toThrow(/not found/)
|
||||
await expect(ctx.sessions.flush(reuse)).resolves.toBeUndefined()
|
||||
} finally {
|
||||
loadGate.resolve(true)
|
||||
await backendFiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('a retiring owner with buffered events still rejects same-id reuse', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const backendFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const loadGate = Promise.withResolvers<boolean>()
|
||||
|
||||
try {
|
||||
const id = SessionId('retiring-buffered-owner')
|
||||
let first!: Session
|
||||
const firstFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
first = inner.sessions.create(id)
|
||||
}, { inject: ['sessions'] }))
|
||||
await ctx.sessions.flush(first)
|
||||
first.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
first.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
|
||||
const baselineLoads = backend.loadAttempts
|
||||
backend.beforeLoadStored = async () => { await loadGate.promise }
|
||||
const blockingLoad = coordinator.load(id)
|
||||
await vi.waitFor(() => { expect(backend.loadAttempts).toBe(baselineLoads + 1) })
|
||||
await firstFiber.dispose()
|
||||
|
||||
let reuse!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
reuse = inner.sessions.create(id)
|
||||
}, { inject: ['sessions'] }))
|
||||
await expect(ctx.sessions.flush(reuse)).rejects.toThrow(/bound to a different live session/)
|
||||
|
||||
loadGate.resolve(true)
|
||||
await expect(blockingLoad).rejects.toThrow(/not found/)
|
||||
await vi.waitFor(() => {
|
||||
expect(backend.store.get(id)?.events.map(event => event.seq)).toEqual([0, 1])
|
||||
})
|
||||
} finally {
|
||||
loadGate.resolve(true)
|
||||
await backendFiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('a settled chain tail cannot delete a newer operation for the same id', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const internals = coordinator as unknown as CoordinatorInternals
|
||||
const first = Promise.withResolvers<boolean>()
|
||||
const second = Promise.withResolvers<boolean>()
|
||||
backend.beforeAppend = async (attempt) => {
|
||||
if (attempt === 1) await first.promise
|
||||
if (attempt === 2) await second.promise
|
||||
}
|
||||
|
||||
try {
|
||||
const id = SessionId('chain-tail')
|
||||
await coordinator.create(meta(id))
|
||||
const firstAppend = coordinator.append(id, [{
|
||||
type: 'turn/start',
|
||||
seq: 0,
|
||||
time: 1,
|
||||
data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } },
|
||||
}])
|
||||
const secondAppend = coordinator.append(id, [{
|
||||
type: 'turn/end',
|
||||
seq: 1,
|
||||
time: 2,
|
||||
data: { turn: 1, reason: { kind: 'completed' } },
|
||||
}])
|
||||
|
||||
await vi.waitFor(() => { expect(backend.appendAttempts).toBe(1) })
|
||||
first.resolve(true)
|
||||
await vi.waitFor(() => { expect(backend.appendAttempts).toBe(2) })
|
||||
expect(internals.chains.size).toBe(1)
|
||||
second.resolve(true)
|
||||
await Promise.all([firstAppend, secondAppend])
|
||||
await vi.waitFor(() => { expect(internals.chains.size).toBe(0) })
|
||||
expect(backend.store.get(id)?.events.map(event => event.seq)).toEqual([0, 1])
|
||||
} finally {
|
||||
first.resolve(true)
|
||||
second.resolve(true)
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('backend teardown retries a failed session retirement before close', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const backendFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const internals = coordinator as unknown as CoordinatorInternals
|
||||
backend.beforeAppend = async (attempt) => {
|
||||
if (attempt === 1) {
|
||||
backend.lifecycle.push('append-failed')
|
||||
throw new Error('transient append failure')
|
||||
}
|
||||
backend.lifecycle.push('append-committed')
|
||||
}
|
||||
|
||||
try {
|
||||
let session!: Session
|
||||
const sessionFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId('retry-retirement'))
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await sessionFiber.dispose()
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(backend.appendAttempts).toBe(1)
|
||||
expect(internals.retirements.size).toBe(0)
|
||||
})
|
||||
expect([...internals.buffers.values()]).toEqual([expect.arrayContaining([
|
||||
expect.objectContaining({ seq: 0 }),
|
||||
expect.objectContaining({ seq: 1 }),
|
||||
])])
|
||||
|
||||
await backendFiber.dispose()
|
||||
expect(backend.store.get(SessionId('retry-retirement'))?.events.map(event => event.seq)).toEqual([0, 1])
|
||||
expect(backend.lifecycle).toEqual(['append-failed', 'append-committed', 'close'])
|
||||
} finally {
|
||||
await backendFiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('backend teardown waits for an in-flight session retirement before close', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const backendFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const internals = coordinator as unknown as CoordinatorInternals
|
||||
const appendGate = Promise.withResolvers<boolean>()
|
||||
backend.beforeAppend = async () => {
|
||||
backend.lifecycle.push('append-started')
|
||||
await appendGate.promise
|
||||
backend.lifecycle.push('append-committed')
|
||||
}
|
||||
|
||||
try {
|
||||
let session!: Session
|
||||
const sessionFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId('inflight-retirement'))
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await sessionFiber.dispose()
|
||||
await vi.waitFor(() => {
|
||||
expect(backend.appendAttempts).toBe(1)
|
||||
expect(internals.retirements.size).toBe(1)
|
||||
})
|
||||
|
||||
let disposed = false
|
||||
const teardown = backendFiber.dispose().then(() => { disposed = true })
|
||||
await Promise.resolve()
|
||||
expect(disposed).toBe(false)
|
||||
expect(backend.lifecycle).toEqual(['append-started'])
|
||||
|
||||
appendGate.resolve(true)
|
||||
await teardown
|
||||
expect(backend.store.get(SessionId('inflight-retirement'))?.events.map(event => event.seq)).toEqual([0, 1])
|
||||
expect(backend.lifecycle).toEqual(['append-started', 'append-committed', 'close'])
|
||||
} finally {
|
||||
appendGate.resolve(true)
|
||||
await backendFiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistence service registration', () => {
|
||||
it('registers as ctx.sessionPersistence and is removed on fiber dispose (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
@@ -158,4 +468,108 @@ describe('SessionPersistence service registration', () => {
|
||||
.rejects.toThrow('session metadata must be losslessly JSON-serializable')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects a legacy header delta from a pre-change live producer', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const session = ctx.sessions.create(SessionId('legacy-live'), { meta: { cwd: '/legacy' } })
|
||||
// Model the runtime shape available to JavaScript or a hot-loaded plugin
|
||||
// compiled against the obsolete event vocabulary.
|
||||
const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent
|
||||
expect(() => appendLegacy('request/header-delta', { config: { model: 'legacy' } }))
|
||||
.toThrow(/unsupported legacy request\/header-delta format/)
|
||||
expect(session.events).toHaveLength(0)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects a legacy fallback header buffered by a pre-change live producer', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const session = ctx.sessions.create(SessionId('legacy-fallback-live'), { meta: { cwd: '/legacy' } })
|
||||
const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent
|
||||
|
||||
expect(() => appendLegacy('request/header', legacyFallbackHeader().data))
|
||||
.toThrow('unsupported legacy request/header reason "fallback"')
|
||||
expect(session.events).toHaveLength(0)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects a legacy stored prefix during live HMR adoption', async () => {
|
||||
const id = SessionId('legacy-hmr')
|
||||
const m = meta(id, '/legacy')
|
||||
const legacy = legacyHeaderDelta()
|
||||
const store: MemoryStore = new Map([[id, { meta: m, events: [legacy] }]])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// A current live session cannot carry the obsolete event in its seed, but
|
||||
// HMR still has to identify the persisted prefix as unsupported rather than
|
||||
// treating it as an ordinary live-prefix collision.
|
||||
const session = ctx.sessions.create(id, { meta: { cwd: '/legacy' } })
|
||||
const fiber = await ctx.plugin(MemoryPersistence, { store })
|
||||
|
||||
await expect(ctx.sessions.flush(session))
|
||||
.rejects.toThrow(/unsupported legacy request\/header-delta event at seq 0/)
|
||||
await Promise.allSettled([fiber.dispose()])
|
||||
})
|
||||
|
||||
it('rejects a stored legacy fallback header during load', async () => {
|
||||
const id = SessionId('legacy-fallback-load')
|
||||
const m = meta(id, '/legacy')
|
||||
const store: MemoryStore = new Map([[id, { meta: m, events: [legacyFallbackHeader()] }]])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence, { store })
|
||||
|
||||
await expect(ctx.sessionPersistence.load(id))
|
||||
.rejects.toThrow('unsupported legacy request/header reason "fallback" at seq 0')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects a stored legacy named-mode event during load', async () => {
|
||||
const id = SessionId('legacy-mode-load')
|
||||
const m = meta(id, '/legacy')
|
||||
const store: MemoryStore = new Map([[id, { meta: m, events: [legacyModeSet()] }]])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence, { store })
|
||||
|
||||
await expect(ctx.sessionPersistence.load(id))
|
||||
.rejects.toThrow('unsupported legacy mode/set event at seq 0')
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('retires all coordinator bookkeeping for disposed sessions', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(MemoryPersistence)
|
||||
const { coordinator } = ctx.sessionPersistence as unknown as { coordinator: CoordinatorInternals }
|
||||
|
||||
try {
|
||||
for (let index = 0; index < 3; index += 1) {
|
||||
let session!: Session
|
||||
const sessionFiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId(`disposed-${index}`))
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.sessions.flush(session)
|
||||
await sessionFiber.dispose()
|
||||
}
|
||||
|
||||
await vi.waitFor(() => {
|
||||
expect(ctx.sessions.list()).toHaveLength(0)
|
||||
expect({
|
||||
states: coordinator.states.size,
|
||||
buffers: coordinator.buffers.size,
|
||||
chains: coordinator.chains.size,
|
||||
inits: coordinator.inits.size,
|
||||
retirements: coordinator.retirements.size,
|
||||
}).toEqual({ states: 0, buffers: 0, chains: 0, inits: 0, retirements: 0 })
|
||||
})
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
@@ -19,6 +19,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user