Merge branch 'stack/agent-profiles-5-web-ui' into stack/agent-profiles-8-authoring
# Conflicts: # docs/cordis-catalog/services.md # docs/module-graph.md # packages/client/README.i18n.yaml # packages/core/tools/README.i18n.yaml
This commit is contained in:
Yichen Jiang
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session-persistence/session-persistence-sqlite/README.md
|
||||
README.md: 4961f62bf6854c343d35b8406c9d721590274534
|
||||
README.zh.md: 8610ef56f737bac8781536ea26a5b48dbf67ed48
|
||||
63
packages/session/session-persistence-sqlite/README.md
Normal file
63
packages/session/session-persistence-sqlite/README.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# @deepseek-ai/dsh-session-persistence-sqlite
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
A SQLite durable session-persistence backend — a second `SessionPersistence` implementation ([session persistence](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)), built to validate that the abstract seam and the shared `runPersistenceContract` suite are genuinely backend-agnostic. It satisfies the SAME contract as `dsh-session-persistence-jsonl` (append-only, contiguous-seq, lazy materialization, interrupted-turn close on load), expressed over `node:sqlite` rows instead of file bytes.
|
||||
|
||||
`locate(meta)` returns `undefined`: all sessions share one database, so there is no honest independent per-session transcript path.
|
||||
|
||||
> **TODO:** this backend talks to `node:sqlite` directly. If a cordis database service (`cordis/db` / a `@cordisjs` SQL driver plugin) is adopted, route through that instead of holding a raw `DatabaseSync` here — the contract surface (`SessionPersistence`) would not change, only the storage driver.
|
||||
|
||||
## Storage model
|
||||
|
||||
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)` — `data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../.agents/notes/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`), a per-materialization incarnation id, and a monotonic per-log revision live in a `sessions` row; `createdAt` is a non-negative safe integer stored in a strict `INTEGER` column. A singleton state row carries the immutable store id. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row).
|
||||
|
||||
The repository's Node range supports unflagged `node:sqlite`. The database enables foreign keys and uses the configured journal mode (`wal` by default; use a rollback mode where WAL shared-memory files are unsuitable). `PRAGMA application_id` identifies the canonical persistence database, and `PRAGMA user_version` stores its layout version. A fresh database must have no application identity or user-defined schema objects; initialization creates every table and stamps both pragmas in one transaction. Non-pristine unversioned databases, foreign application identities, and every non-current version reject before journal-mode mutation because this unreleased format has no migrations.
|
||||
|
||||
On filesystems with POSIX modes, the backend requests mode `0700` for missing directories and exclusively creates a missing database with mode `0600` before SQLite opens it; the process umask may further restrict both. New WAL, shared-memory, and persistent rollback-journal sidecars receive the database's resulting owner-only mode. Existing directories, database files, and sidecars keep their modes; filesystem setup errors other than an existing database fail initialization. These defaults prevent incidental exposure through a permissive process umask, but do not protect database confidentiality or integrity when another principal can replace the database entry in its parent directory.
|
||||
|
||||
## Contract semantics over rows
|
||||
|
||||
- **Append = a transaction.** `append` runs `BEGIN`/`COMMIT` around the batch: it materializes the `sessions` row (if still lazy) and INSERTs every event, asserting the contiguous-seq contract first (the first event's `seq` must equal the stored next-seq). A mid-batch failure (a UNIQUE violation on a duplicated seq) rolls back entirely, so the stored log and the in-memory cursor stay consistent. (`load()` already balanced the stored log, so `append` never has to repair a crash tail.)
|
||||
- **Lazy materialization.** `create()` records intent in memory only — no row is written until the first `append`. A created-but-never-appended session has no `sessions` row, so it is absent from `list()` (which reports exactly the sessions that have a row).
|
||||
- **Interrupted-turn close on load.** `load()` implements the shared [crash-recovery contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md): preserve the valid interrupted turn, append its synthetic closing events in one transaction, and remove only a torn tail row. Committed parse errors or sequence gaps make the session unloadable. Because recovery mutates stored rows, the next append starts from a balanced log and accurate cursor.
|
||||
- **Non-mutating inspection.** `inspect()` returns an immutable balanced logical view and may synthesize recovery closers in memory, without deleting a torn tail row, appending recovery rows, or changing the lightweight revision.
|
||||
- **Lightweight revisions.** `listSnapshots(signal?)` combines the immutable store and database-file identity, a per-materialization incarnation id, and a per-session counter incremented in each mutating transaction. A full-prefix read captures that revision and its event rows in one read transaction, while `readStoredRevision()` queries only the session row to validate retained preparations. This keeps unchanged observations stable without parsing event rows and distinguishes independent stores and recreated same-id logs. It checks cancellation before and after shared readiness and the synchronous metadata query; the query itself is non-preemptible.
|
||||
|
||||
## Configuration (schemastery)
|
||||
|
||||
```ts
|
||||
interface Config {
|
||||
path: string // SQLite database file path, or ':memory:' for an in-process DB
|
||||
journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' // journal_mode pragma; default 'wal'
|
||||
preparedSessionCacheSize?: number // positive integer; default 5
|
||||
writeBatchMaxDelayMs?: number // positive integer; default 200; maximum 2_147_483_647
|
||||
}
|
||||
```
|
||||
|
||||
## Write path
|
||||
|
||||
Like the JSONL backend, the plugin copies each frozen `session/event` into one controller per live session. The first pending event starts the configured fixed batching window, and later events join without resetting it. Expiry starts one transaction; events admitted during that write form a separately bounded follow-up batch. `session/flush` cancels the wait and drains current and pending batches. The controller persists a fork's seed once, keeps a write cursor so resume never re-appends stored events, and seeds live sessions on apply because HMR does not replay `session/created`. Dispose drains every retained controller before closing the database. Every event remains a separate SQLite row; batching only groups more INSERTs into one transaction and revision increment.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Resumed conversation history
|
||||
|
||||
#### What the model sees
|
||||
|
||||
SQLite storage contributes no live prompt or schema. Loading restores the same surface history as JSONL and preserves prior headers for reconstruction; the new loop composes its current envelope. Recovery balances an assistant request without a durable call with `TOOL_NOT_STARTED`; a durable call without a result becomes `TOOL_OUTCOME_UNKNOWN`, which tells the model to retry only read-only or idempotent work and to verify possible side effects or ask the user. Row metadata and raw chunks are not messages.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Zero live-request tokens. Resume restores retained history and pays the current envelope, plus the quoted repair result for each interrupted call.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
SQLite storage does not mutate live request prefixes. A resumed loop can reuse provider cache only when its reconstructed history, current envelope, and model route match; crash-repair results append.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **`DatabaseSync` is synchronous** — every append transaction blocks the event loop for its duration; acceptable for local stores, a throughput ceiling for busy multi-session servers.
|
||||
- **Write contention has no wait or retry policy** — the backend sets no busy timeout and retries no locked-database error, so another connection holding a write transaction makes the operation reject immediately.
|
||||
- **Only a pristine new database or the current owned `SCHEMA_VERSION` opens** — unversioned schema objects, foreign application identities, and every other schema version are rejected rather than migrated (unreleased software; no persisted user data to preserve).
|
||||
- **Nothing deletes stored sessions** — rows accumulate until removed externally (the seam has no deletion surface; `ON DELETE CASCADE` is wired for such out-of-band cleanup).
|
||||
63
packages/session/session-persistence-sqlite/README.zh.md
Normal file
63
packages/session/session-persistence-sqlite/README.zh.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# @deepseek-ai/dsh-session-persistence-sqlite
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
SQLite 持久会话存储后端：第二个 `SessionPersistence` 实现（见[会话持久化](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)），用于验证抽象 seam 和共享 `runPersistenceContract` 套件真正与后端无关。它满足与 `dsh-session-persistence-jsonl` 相同的契约（仅追加、连续 seq、延迟实体化、在 load 时关闭中断轮次），但用 `node:sqlite` 行而非文件字节表达。
|
||||
|
||||
`locate(meta)` 返回 `undefined`：所有会话共享一个数据库，因此不存在真实、独立的逐会话 transcript（文本记录）路径。
|
||||
|
||||
> **TODO：** 该后端直接调用 `node:sqlite`。如果采用 Cordis 数据库服务（`cordis/db` / `@cordisjs` SQL driver 插件），应改为通过该服务路由，而不在此直接持有 `DatabaseSync`；契约接口（`SessionPersistence`）不会变，只更换存储驱动。
|
||||
|
||||
## 存储模型
|
||||
|
||||
每个 `SessionEvent` 1:1 映射到 `events` 表中的一行 `(session_id, seq, type, time, data, source_event_seqs, surface_op)`；`data` 是作为 JSON 文本的事件 payload，因此行结构就是原始事件本身（包括 `assistant/chunk`，保持 `seq` 连续）。两个 `TEXT` 列 `source_event_seqs` 和 `surface_op` 可为空，存储事件可选接口元数据字段（见[会话接口](../../../.agents/notes/implemented/architecture/2026-06-18-session-surface.md)）。日志外元数据（`SessionHeader`）、每实体化 incarnation id 和每日志单调修订位于 `sessions` 行；`createdAt` 是存储在 strict `INTEGER` 列中的非负安全整数。单例状态行携带不可变存储 id。`sessions` 行只由第一次 `append` 写入，其存在性是延迟实体化信号（`list` 精确报告有行的会话）。
|
||||
|
||||
仓库支持的 Node 范围可不加 flag 使用 `node:sqlite`。数据库启用外键，并使用已配置 journal mode（默认 `wal`；WAL 共享内存文件不适用时使用 rollback mode）。`PRAGMA application_id` 标识规范持久化数据库，`PRAGMA user_version` 存储布局版本。新数据库必须没有 application identity 或用户定义 schema 对象；初始化在一个事务中创建全部表并盖上两个 pragma。非 pristine 无版本数据库、外部 application identity 和所有非当前版本在 journal-mode 变更前均会被拒绝，因为该未发布格式无迁移。
|
||||
|
||||
在具有 POSIX mode 的文件系统上，后端为缺失目录请求 mode `0700`，并在 SQLite 打开前以 mode `0600` 排他创建缺失数据库；进程 umask 可进一步限制两者。新 WAL、共享内存和持久 rollback-journal sidecar 获得数据库最终的仅所有者 mode。现有目录、数据库文件和 sidecar 保留原 mode；除已存在数据库外的文件系统设置错误会使初始化失败。这些默认值防止宽松进程 umask 造成的意外暴露，但当其他 principal 能替换父目录中的数据库条目时，不保护数据库机密性或完整性。
|
||||
|
||||
## 行上的契约语义
|
||||
|
||||
- **Append = 事务。**`append` 围绕批次运行 `BEGIN`/`COMMIT`：它实体化 `sessions` 行（如果仍延迟），并 INSERT 每个事件，首先断言连续 seq 契约（第一个事件 `seq` 必须等于已存储 next-seq）。批次中失败（重复 seq 上的 UNIQUE 违规）会完全回滚，使已存储日志和内存游标保持一致。（`load()` 已平衡已存储日志，因此 `append` 不必修复崩溃尾部。）
|
||||
- **延迟实体化。**`create()` 只在内存记录意图，第一次 `append` 前不写行。从未 append 的会话没有 `sessions` 行，因此不在 `list()` 中（它精确报告有行的会话）。
|
||||
- **在 load 时关闭中断轮次。**`load()` 实现共享[崩溃恢复契约](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md)：保留有效中断轮次，在一个事务中追加合成关闭事件，并只移除撕裂尾部行。已提交解析错误或序列缺口使会话无法加载。恢复会变更已存储行，因此下一次 append 从平衡日志和准确游标开始。
|
||||
- **非变更检查。**`inspect()` 返回不可变、平衡的逻辑视图，并可在内存中合成恢复 closer，但不会删除撕裂尾部行、追加恢复行或更改轻量修订。
|
||||
- **轻量修订。**`listSnapshots(signal?)` 组合不可变存储与数据库文件身份、每实体化 incarnation id，以及在每个变更事务中递增的每会话计数器。完整前缀读取在同一个读事务中捕获该 revision 及其事件行，`readStoredRevision()` 则只查询 session 行来校验保留的 preparation。它在不解析事件行的情况下保持未变观察稳定，并区分独立存储和重建的同 id 日志。它在共享就绪和同步元数据查询前后检查取消；查询本身不可抢占。
|
||||
|
||||
## 配置（schemastery）
|
||||
|
||||
```ts
|
||||
interface Config {
|
||||
path: string // SQLite database file path, or ':memory:' for an in-process DB
|
||||
journalMode?: 'wal' | 'delete' | 'truncate' | 'persist' // journal_mode pragma; default 'wal'
|
||||
preparedSessionCacheSize?: number // positive integer; default 5
|
||||
writeBatchMaxDelayMs?: number // positive integer; default 200; maximum 2_147_483_647
|
||||
}
|
||||
```
|
||||
|
||||
## 写入路径
|
||||
|
||||
与 JSONL 后端一样，插件将每个冻结的 `session/event` 复制到每个活动会话各自的 controller。第一个待处理事件会开启配置的固定批处理窗口，后续事件会加入但不会重置截止时间。窗口到期后会启动一个事务；该次写入期间接纳的事件会形成另一个独立有界的后续批次。`session/flush` 会取消等待并排空当前与待处理批次。Controller 会持久化一次 fork 种子，并保留写入游标，使恢复操作绝不重新 append 已存储事件；它还会在 apply 时为活动会话设置初始状态，因为 HMR（热模块替换）不回放 `session/created`。dispose（资源释放）会在关闭数据库前排空每个保留的 controller。每个事件仍各占一行 SQLite 记录；批处理只把更多 INSERT 归入同一个事务和同一次修订版本递增。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### 恢复的对话历史
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
SQLite 存储不影响当前提示词或 schema。加载会恢复与 JSONL 相同的呈现历史，并保留之前的 header 用于重建；新 loop 组合当前 envelope。恢复会用 `TOOL_NOT_STARTED` 平衡没有已持久化调用的 assistant 请求；已有已持久化调用但无结果时则变为 `TOOL_OUTCOME_UNKNOWN`，它要求模型只重试只读或幂等工作，并验证可能的副作用或询问用户。行元数据和原始分片不会成为消息。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
当前请求不会新增 token。恢复会还原已保留的历史，并产生当前 envelope 以及每个中断调用所附修复结果文本的 token 开销。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
SQLite 存储不修改当前请求前缀。只有重建历史、当前 envelope 和模型路由匹配时，恢复 loop 才能重用提供方缓存；崩溃修复结果会追加到末尾。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **`DatabaseSync` 是同步的**：每个 append 事务在整个期间阻塞事件循环；对本地存储可接受，对繁忙多会话服务器是吞吐上限。
|
||||
- **写入争用无等待或重试策略**：后端不设置 busy timeout，也不重试 locked-database 错误，因此其他连接持有写事务时操作立即拒绝。
|
||||
- **只有 pristine 新数据库或当前自有 `SCHEMA_VERSION` 才能打开**：无版本 schema 对象、外部 application identity 和所有其他 schema 版本被拒绝，而不是迁移（未发布软件，无持久用户数据需要保留）。
|
||||
- **不删除已存储会话**：行会累积，直到外部移除（seam 无删除接口；`ON DELETE CASCADE` 已为这种带外清理配置）。
|
||||
42
packages/session/session-persistence-sqlite/package.json
Normal file
42
packages/session/session-persistence-sqlite/package.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-session-persistence-sqlite",
|
||||
"description": "SQLite durable session persistence backend for the DeepSeek Harness",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
409
packages/session/session-persistence-sqlite/src/index.ts
Normal file
409
packages/session/session-persistence-sqlite/src/index.ts
Normal file
@@ -0,0 +1,409 @@
|
||||
/**
|
||||
* SQLite durable session-persistence backend. It maps each session header and
|
||||
* event to rows, and delegates write-path orchestration to
|
||||
* {@link PersistenceCoordinator}. It has no independent per-session artifact,
|
||||
* so its locator returns `undefined`.
|
||||
* @module @deepseek-ai/dsh-session-persistence-sqlite
|
||||
*/
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { statSync } from 'node:fs'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { mkdir, open } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import {
|
||||
DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS,
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator,
|
||||
type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot,
|
||||
type SessionInspection, type SessionPersistenceRevision as PersistenceRevision,
|
||||
type StoredPrefix, type StoredSuffix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { SessionEvent, SurfaceEventType, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
type JournalMode, openDatabase, rowToMeta, scanRows, type EventRow, type SessionRow,
|
||||
} from './schema.ts'
|
||||
|
||||
export { SCHEMA_VERSION } from './schema.ts'
|
||||
|
||||
/**
|
||||
* Serialize an event's surface-metadata fields for SQL binding. Both fields are
|
||||
* nullable TEXT columns — null when the event has no surface metadata (non-surface
|
||||
* events, events written before surface support).
|
||||
*/
|
||||
function surfaceBindings(event: SessionEvent): [string | null, string | null] {
|
||||
const se = event as SessionEvent<SurfaceEventType>
|
||||
return [
|
||||
se.sourceEventSeqs ? JSON.stringify(se.sourceEventSeqs) : null,
|
||||
se.surfaceOp !== undefined ? JSON.stringify(se.surfaceOp) : null,
|
||||
]
|
||||
}
|
||||
|
||||
/** Build the source-qualified revision shared by full and lightweight reads. */
|
||||
function sqliteRevision(storeIdentity: string, row: SessionRow): PersistenceRevision {
|
||||
return SessionPersistenceRevision(
|
||||
`${storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}`,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Exclusively create a missing database file with owner-only permissions.
|
||||
* Existing files retain their modes, and errors other than `EEXIST` propagate.
|
||||
* `DatabaseSync` reopens by path, so this does not protect confidentiality or
|
||||
* integrity when another principal can replace the database entry in its parent
|
||||
* directory.
|
||||
*/
|
||||
async function createDatabaseFile(path: string): Promise<void> {
|
||||
try {
|
||||
const handle = await open(path, 'wx', 0o600)
|
||||
await handle.close()
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error
|
||||
}
|
||||
}
|
||||
|
||||
/** Plugin configuration. */
|
||||
export interface Config {
|
||||
/**
|
||||
* Filesystem path to the SQLite database file. The special value `:memory:`
|
||||
* opens an in-process database (tests). On filesystems with POSIX modes,
|
||||
* missing directories and databases are created owner-only; existing path
|
||||
* modes are preserved. Filesystem setup errors other than an existing database
|
||||
* fail initialization. The backend does not protect confidentiality or
|
||||
* integrity when another principal can replace the database entry in its
|
||||
* parent directory.
|
||||
*/
|
||||
path: string
|
||||
/**
|
||||
* SQLite `journal_mode` pragma. `wal` (the default) is the recorded
|
||||
* durability model; pick a rollback-journal mode (`delete`/`truncate`/
|
||||
* `persist`) on filesystems where WAL's shared-memory files do not work
|
||||
* (network mounts). See {@link JournalMode}.
|
||||
*/
|
||||
journalMode?: JournalMode
|
||||
/** Maximum cold Session preparations retained for history-to-resume reuse. */
|
||||
preparedSessionCacheSize?: number
|
||||
/** Fixed live-event coalescing window; not a backend completion deadline. */
|
||||
writeBatchMaxDelayMs?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* The SQLite persistence backend. Load as a plugin; it registers as
|
||||
* `ctx.sessionPersistence` and (via the coordinator) installs the write-path
|
||||
* listeners. Its torn-tail marker is the seq to delete from.
|
||||
*/
|
||||
export class SessionPersistenceSqlite extends SessionPersistence implements PersistenceBackend<number> {
|
||||
static inject = ['sessions']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
path: z.string().required(),
|
||||
journalMode: z.union(['wal', 'delete', 'truncate', 'persist'] as const).default('wal'),
|
||||
preparedSessionCacheSize: z.number().step(1).min(1).default(DEFAULT_PREPARED_SESSION_CACHE_SIZE),
|
||||
writeBatchMaxDelayMs: z.number().step(1).min(1).max(MAX_WRITE_BATCH_DELAY_MS)
|
||||
.default(DEFAULT_WRITE_BATCH_MAX_DELAY_MS),
|
||||
})
|
||||
|
||||
/**
|
||||
* Backend label for the coordinator's dispose diagnostics. Intentionally
|
||||
* shadows cordis `Service.name` (set to `'sessionPersistence'` by the base);
|
||||
* see the JSONL backend for why this does not affect service resolution.
|
||||
*/
|
||||
override readonly name = 'session-persistence-sqlite'
|
||||
|
||||
private db!: DatabaseSync
|
||||
private storeIdentity!: string
|
||||
private ready: Promise<void>
|
||||
private coordinator: PersistenceCoordinator<number>
|
||||
|
||||
constructor(ctx: Context, public config: Config) {
|
||||
super(ctx)
|
||||
// Programmatic wrappers may construct the backend without Schemastery normalization.
|
||||
const preparedSessionCacheSize = config.preparedSessionCacheSize
|
||||
?? DEFAULT_PREPARED_SESSION_CACHE_SIZE
|
||||
const writeBatchMaxDelayMs = config.writeBatchMaxDelayMs
|
||||
?? DEFAULT_WRITE_BATCH_MAX_DELAY_MS
|
||||
// Open asynchronously so directory creation does not block plugin apply;
|
||||
// every storage hook awaits the same readiness promise.
|
||||
this.ready = this.openDb(config.path, (config as Required<Config>).journalMode)
|
||||
this.coordinator = new PersistenceCoordinator<number>(this.ctx, this, {
|
||||
preparedSessionCacheSize,
|
||||
writeBatchMaxDelayMs,
|
||||
})
|
||||
}
|
||||
|
||||
private async openDb(path: string, journalMode: JournalMode): Promise<void> {
|
||||
const actual = path === ':memory:' ? path : resolve(path)
|
||||
if (actual !== ':memory:') {
|
||||
await mkdir(dirname(actual), { recursive: true, mode: 0o700 })
|
||||
await createDatabaseFile(actual)
|
||||
}
|
||||
this.db = openDatabase(actual, journalMode)
|
||||
try {
|
||||
const row = this.db.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string } | undefined
|
||||
/* v8 ignore next -- openDatabase inserts the singleton before returning. */
|
||||
if (row === undefined) {
|
||||
throw new Error(`session database at "${actual}" has no store identity`)
|
||||
}
|
||||
if (row.store_id.length === 0) {
|
||||
throw new Error(`session database at "${actual}" has no valid store identity`)
|
||||
}
|
||||
if (actual !== ':memory:') {
|
||||
const identity = statSync(actual, { bigint: true })
|
||||
this.storeIdentity = `file:${identity.dev}:${identity.ino}:${identity.birthtimeNs}:store:${row.store_id}`
|
||||
} else {
|
||||
this.storeIdentity = `memory:store:${row.store_id}`
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
this.db.close()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
// --- SessionPersistence service surface (delegated to the coordinator) ---
|
||||
|
||||
/** SQLite has one database, not an independent local artifact per session. */
|
||||
locate(_meta: SessionHeader): SessionLocation | undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
return this.coordinator.create(meta)
|
||||
}
|
||||
|
||||
append(id: SessionId, events: readonly SessionEvent[]): Promise<void> {
|
||||
return this.coordinator.append(id, events)
|
||||
}
|
||||
|
||||
override prepare(id: SessionId, signal?: AbortSignal): Promise<SessionPreparation> {
|
||||
return this.coordinator.prepare(id, signal)
|
||||
}
|
||||
|
||||
load(id: SessionId): Promise<SessionInspection> {
|
||||
return this.coordinator.load(id)
|
||||
}
|
||||
|
||||
inspect(id: SessionId, signal?: AbortSignal): Promise<SessionInspection> {
|
||||
return this.coordinator.inspect(id, signal)
|
||||
}
|
||||
|
||||
readFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.readFrom(id, fromSeq, signal)
|
||||
}
|
||||
|
||||
// One method serves both public `list` and the backend hook; delegating it to
|
||||
// the coordinator would call this hook recursively.
|
||||
|
||||
// --- PersistenceBackend hooks (the SQLite storage primitives) ---
|
||||
|
||||
/** Read a stored prefix by id (ids are globally unique — no scope to scan). */
|
||||
loadStored(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<number> | undefined> {
|
||||
return this.readPrefix(id, signal)
|
||||
}
|
||||
|
||||
/** Read one row's revision without loading its events. */
|
||||
async readStoredRevision(id: SessionId, signal?: AbortSignal): Promise<PersistenceRevision | undefined> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ready
|
||||
signal?.throwIfAborted()
|
||||
const row = this.rowFor(id)
|
||||
return row === undefined ? undefined : sqliteRevision(this.storeIdentity, row)
|
||||
}
|
||||
|
||||
/**
|
||||
* Seek-capable suffix read: SQL selects `seq >= fromSeq` directly, so the
|
||||
* read scales with the suffix, not the log. Torn rows past the preserved
|
||||
* region are dropped, never repaired (non-mutating read).
|
||||
*/
|
||||
async loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<StoredSuffix | undefined> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ready
|
||||
signal?.throwIfAborted()
|
||||
const row = this.rowFor(id)
|
||||
if (row === undefined) return undefined
|
||||
const meta = rowToMeta(row)
|
||||
const eventRows = this.db
|
||||
.prepare('SELECT seq, type, time, data, source_event_seqs, surface_op FROM events WHERE session_id = ? AND seq >= ? ORDER BY seq')
|
||||
.all(id, fromSeq) as unknown as EventRow[]
|
||||
signal?.throwIfAborted()
|
||||
const { preserved } = scanRows(eventRows, fromSeq)
|
||||
return { meta, events: preserved }
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a session's row + ordered events into a {@link StoredPrefix}. The
|
||||
* torn-tail marker is the seq from which a never-committed tail must be deleted
|
||||
* (`scanRows` already returns it as `number | undefined`).
|
||||
*/
|
||||
private async readPrefix(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<number> | undefined> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ready
|
||||
signal?.throwIfAborted()
|
||||
this.db.exec('BEGIN')
|
||||
let snapshot: { row: SessionRow; eventRows: EventRow[] } | undefined
|
||||
try {
|
||||
const row = this.rowFor(id)
|
||||
if (row !== undefined) {
|
||||
const eventRows = this.db
|
||||
.prepare('SELECT seq, type, time, data, source_event_seqs, surface_op FROM events WHERE session_id = ? ORDER BY seq')
|
||||
.all(id) as unknown as EventRow[]
|
||||
snapshot = { row, eventRows }
|
||||
}
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore start -- synchronous read failures only need transaction cleanup before propagation. */
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
if (snapshot === undefined) return undefined
|
||||
const { row, eventRows } = snapshot
|
||||
const { preserved, tornFrom } = scanRows(eventRows)
|
||||
return {
|
||||
meta: rowToMeta(row),
|
||||
events: preserved,
|
||||
revision: sqliteRevision(this.storeIdentity, row),
|
||||
...tornFrom !== undefined ? { tornMarker: tornFrom } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Durably append a batch in ONE transaction: materialize the sessions row (if
|
||||
* lazy) and INSERT every event, or roll back entirely. The transaction is the
|
||||
* atomicity + durability boundary, so a mid-batch failure (a UNIQUE violation
|
||||
* on a duplicated seq) leaves the stored log untouched.
|
||||
*/
|
||||
async appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void> {
|
||||
await this.ready
|
||||
const insertEvent = this.db.prepare(
|
||||
'INSERT INTO events (session_id, seq, type, time, data, source_event_seqs, surface_op) VALUES (?, ?, ?, ?, ?, ?, ?)',
|
||||
)
|
||||
this.db.exec('BEGIN')
|
||||
try {
|
||||
if (!isMaterialized) this.writeRow(meta)
|
||||
for (const event of events) {
|
||||
const [surfaceSeqs, surfaceOp] = surfaceBindings(event)
|
||||
insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data), surfaceSeqs, surfaceOp)
|
||||
}
|
||||
this.db.prepare('UPDATE sessions SET revision = revision + 1 WHERE id = ?').run(meta.id)
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error) {
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Make a crash repair durable in ONE transaction: DELETE the torn tail (from
|
||||
* `tornMarker`) and INSERT the synthetic `closers`. After COMMIT the stored rows
|
||||
* == the balanced log.
|
||||
*/
|
||||
async commitRepair(meta: SessionHeader, tornMarker: number | undefined, closers: readonly SessionEvent[]): Promise<void> {
|
||||
await this.ready
|
||||
this.db.exec('BEGIN')
|
||||
try {
|
||||
if (tornMarker !== undefined) {
|
||||
this.db.prepare('DELETE FROM events WHERE session_id = ? AND seq >= ?').run(meta.id, tornMarker)
|
||||
}
|
||||
if (closers.length > 0) {
|
||||
const insertEvent = this.db.prepare(
|
||||
'INSERT INTO events (session_id, seq, type, time, data, source_event_seqs, surface_op) VALUES (?, ?, ?, ?, ?, ?, ?)',
|
||||
)
|
||||
for (const event of closers) {
|
||||
const [surfaceSeqs, surfaceOp] = surfaceBindings(event)
|
||||
insertEvent.run(meta.id, event.seq, event.type, event.time, JSON.stringify(event.data), surfaceSeqs, surfaceOp)
|
||||
}
|
||||
}
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
this.db.prepare('UPDATE sessions SET revision = revision + 1 WHERE id = ?').run(meta.id)
|
||||
}
|
||||
this.db.exec('COMMIT')
|
||||
} catch (error) {
|
||||
// The DELETE+INSERT cannot collide (a row at a closer's seq is preserved or
|
||||
// deleted as torn first); this rolls back a DB-level failure (disk full,
|
||||
// etc.), unreachable in test.
|
||||
/* v8 ignore start */
|
||||
this.db.exec('ROLLBACK')
|
||||
throw error
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
|
||||
/** List all materialized sessions' metadata (every row is a materialized session). */
|
||||
async list(signal?: AbortSignal): Promise<SessionHeader[]> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ready
|
||||
signal?.throwIfAborted()
|
||||
const rows = this.db
|
||||
.prepare('SELECT * FROM sessions')
|
||||
.all() as unknown as SessionRow[]
|
||||
signal?.throwIfAborted()
|
||||
return rows.map(rowToMeta)
|
||||
}
|
||||
|
||||
/** List metadata with a source-qualified monotonic revision per session. */
|
||||
async listSnapshots(signal?: AbortSignal): Promise<SessionPersistenceSnapshot[]> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ready
|
||||
signal?.throwIfAborted()
|
||||
const rows = this.db.prepare('SELECT * FROM sessions').all() as unknown as SessionRow[]
|
||||
signal?.throwIfAborted()
|
||||
return rows.map(row => ({
|
||||
header: rowToMeta(row),
|
||||
revision: SessionPersistenceRevision(
|
||||
`${this.storeIdentity}:incarnation:${row.incarnation}:revision:${row.revision}`,
|
||||
),
|
||||
}))
|
||||
}
|
||||
|
||||
/** Close the database handle (awaited by the coordinator's dispose, post-drain). */
|
||||
async close(): Promise<void> {
|
||||
await this.ready
|
||||
this.db.close()
|
||||
}
|
||||
|
||||
// --- row helpers ---
|
||||
|
||||
/** Fetch a session's row, or undefined if absent. */
|
||||
private rowFor(id: SessionId): SessionRow | undefined {
|
||||
return this.db.prepare('SELECT * FROM sessions WHERE id = ?').get(id) as unknown as SessionRow | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert-or-replace a session's metadata row. The only caller is the first
|
||||
* materializing `appendBatch`, so writing the row IS the materialization (its
|
||||
* existence is the signal `list` reads).
|
||||
*/
|
||||
private writeRow(meta: SessionHeader): void {
|
||||
this.db.prepare(`
|
||||
INSERT INTO sessions
|
||||
(id, version, created_at, cwd, parent_session, seed_length, origin, delegation_depth, agent_preset, incarnation, revision)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
version = excluded.version,
|
||||
created_at = excluded.created_at,
|
||||
cwd = excluded.cwd,
|
||||
parent_session = excluded.parent_session,
|
||||
seed_length = excluded.seed_length,
|
||||
origin = excluded.origin,
|
||||
delegation_depth = excluded.delegation_depth,
|
||||
agent_preset = excluded.agent_preset
|
||||
`).run(
|
||||
meta.id,
|
||||
meta.version,
|
||||
meta.createdAt,
|
||||
meta.cwd ?? null,
|
||||
meta.parentSession ?? null,
|
||||
meta.seedLength ?? null,
|
||||
meta.origin ?? null,
|
||||
meta.delegationDepth ?? null,
|
||||
meta.agentPreset ?? null,
|
||||
randomUUID(),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
export default SessionPersistenceSqlite
|
||||
30
packages/session/session-persistence-sqlite/src/invariant.ts
Normal file
30
packages/session/session-persistence-sqlite/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-persistence-sqlite`.
|
||||
* @module @deepseek-ai/dsh-session-persistence-sqlite/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-persistence-sqlite-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: persistence correctness requires backend round-trip and crash-tail tests;
|
||||
* this package exposes no continuously observable in-process relation.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
265
packages/session/session-persistence-sqlite/src/schema.ts
Normal file
265
packages/session/session-persistence-sqlite/src/schema.ts
Normal file
@@ -0,0 +1,265 @@
|
||||
/**
|
||||
* Schema + load-time helpers for the SQLite session-persistence backend: the
|
||||
* DDL (a store-identity row, `sessions` metadata, and a 1:1 `events` row per
|
||||
* `SessionEvent`), the database open/configure step, and the last-`turn/end`
|
||||
* cut that gives the SQLite backend the SAME crash-tail-on-load semantics as
|
||||
* the JSONL backend.
|
||||
*
|
||||
* @module dsh-session-persistence-sqlite/schema
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import type { SessionEvent, SessionId, SessionHeader, SurfaceOp } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* The on-disk schema version. Bumped only on a breaking change to the table
|
||||
* layout; orthogonal to a session's own `version` (which versions the EVENT
|
||||
* vocabulary, stored per session in the `sessions` row).
|
||||
*/
|
||||
export const SCHEMA_VERSION = 14
|
||||
|
||||
/** SQLite application id protecting unrelated databases from persistence writes. */
|
||||
export const SESSION_PERSISTENCE_SQLITE_APPLICATION_ID = 0x44534850
|
||||
|
||||
/**
|
||||
* A row of the `sessions` table — the out-of-log metadata ({@link SessionHeader}).
|
||||
* The row's EXISTENCE is the materialization signal: it is written only by the
|
||||
* first `append` (lazy materialization), so a created-but-never-appended
|
||||
* session has no row and is absent from `list`, mirroring the JSONL
|
||||
* backend's "no file until first append".
|
||||
*/
|
||||
export interface SessionRow {
|
||||
id: string
|
||||
version: number
|
||||
created_at: number
|
||||
cwd: string | null
|
||||
parent_session: string | null
|
||||
seed_length: number | null
|
||||
origin: 'subagent' | null
|
||||
/** Stable identity assigned when this log is materialized. */
|
||||
incarnation: string
|
||||
/** Monotonic log-change token incremented in each mutating transaction. */
|
||||
revision: number
|
||||
delegation_depth: number | null
|
||||
agent_preset: string | null
|
||||
}
|
||||
|
||||
/** An `events` table row: one `SessionEvent` mapped 1:1 (`data` is JSON text). */
|
||||
export interface EventRow {
|
||||
seq: number
|
||||
type: string
|
||||
time: number
|
||||
data: string
|
||||
/** JSON-encoded `number[]` — the event's sourceEventSeqs, or null. */
|
||||
source_event_seqs: string | null
|
||||
/** JSON-encoded `SurfaceOp` — how the event entered the surface, or null. */
|
||||
surface_op: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Journal modes the backend will run under. `wal` is the default and the
|
||||
* durability model the persistence ADR records; the rollback-journal modes
|
||||
* (`delete`/`truncate`/`persist`) exist for filesystems where WAL's
|
||||
* shared-memory files do not work (network mounts). `memory`/`off` are
|
||||
* excluded: dropping journal durability silently contradicts what this
|
||||
* backend promises.
|
||||
*/
|
||||
export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist'
|
||||
|
||||
/**
|
||||
* Open the database and apply its schema and pragmas. An empty database with a
|
||||
* zero `user_version` is initialized at {@link SCHEMA_VERSION}; a nonempty
|
||||
* unversioned database and every other non-current version reject rather than
|
||||
* being migrated in place.
|
||||
* @param path - the SQLite database file to open (created when absent).
|
||||
* @param journalMode - validated journal pragma.
|
||||
* @returns the open handle with pragmas applied and all three tables ensured.
|
||||
*/
|
||||
export function openDatabase(path: string, journalMode: JournalMode): DatabaseSync {
|
||||
const db = new DatabaseSync(path)
|
||||
try {
|
||||
configureDatabase(db, path, journalMode)
|
||||
return db
|
||||
} catch (error: unknown) {
|
||||
db.close()
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
function configureDatabase(db: DatabaseSync, path: string, journalMode: JournalMode): void {
|
||||
db.exec('PRAGMA foreign_keys = ON')
|
||||
let began = false
|
||||
try {
|
||||
db.exec('BEGIN IMMEDIATE')
|
||||
began = true
|
||||
// Validate while holding the write lock so no other connection can change
|
||||
// schema ownership between inspection and initialization.
|
||||
const { user_version: onDisk } = db.prepare('PRAGMA user_version').get() as { user_version: number }
|
||||
const { application_id: applicationId } = db.prepare('PRAGMA application_id').get() as { application_id: number }
|
||||
const { count: userObjectCount } = db.prepare(
|
||||
"SELECT COUNT(*) AS count FROM sqlite_schema WHERE name NOT GLOB 'sqlite_*'",
|
||||
).get() as { count: number }
|
||||
if (onDisk === 0 && (applicationId !== 0 || userObjectCount > 0)) {
|
||||
throw new Error(`session database at "${path}" has an unversioned schema or application identity`)
|
||||
}
|
||||
if (onDisk !== 0 && onDisk !== SCHEMA_VERSION) {
|
||||
throw new Error(`session database at "${path}" has schema version ${onDisk}, incompatible with this build (${SCHEMA_VERSION})`)
|
||||
}
|
||||
if (onDisk === SCHEMA_VERSION && applicationId !== SESSION_PERSISTENCE_SQLITE_APPLICATION_ID) {
|
||||
throw new Error(
|
||||
`session database at "${path}" has application id ${applicationId}, expected ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`,
|
||||
)
|
||||
}
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS persistence_state (
|
||||
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
|
||||
store_id TEXT NOT NULL
|
||||
) STRICT;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
id TEXT PRIMARY KEY,
|
||||
version INTEGER NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
cwd TEXT,
|
||||
parent_session TEXT,
|
||||
seed_length INTEGER,
|
||||
origin TEXT,
|
||||
delegation_depth INTEGER,
|
||||
agent_preset TEXT,
|
||||
incarnation TEXT NOT NULL,
|
||||
revision INTEGER NOT NULL
|
||||
) STRICT;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE,
|
||||
seq INTEGER NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
time INTEGER NOT NULL,
|
||||
data TEXT NOT NULL,
|
||||
source_event_seqs TEXT,
|
||||
surface_op TEXT,
|
||||
PRIMARY KEY (session_id, seq)
|
||||
) STRICT
|
||||
`)
|
||||
db.prepare(
|
||||
'INSERT OR IGNORE INTO persistence_state (singleton, store_id) VALUES (1, ?)',
|
||||
).run(randomUUID())
|
||||
if (onDisk === 0) {
|
||||
db.exec(`PRAGMA application_id = ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`)
|
||||
db.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`)
|
||||
}
|
||||
db.exec('COMMIT')
|
||||
began = false
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- a BEGIN failure leaves no transaction to roll back. */
|
||||
if (began) {
|
||||
/* v8 ignore next 5 -- preserve the original schema failure if SQLite also refuses rollback. */
|
||||
try {
|
||||
db.exec('ROLLBACK')
|
||||
} catch {
|
||||
// The original SQLite failure remains the actionable cause.
|
||||
}
|
||||
}
|
||||
throw error
|
||||
}
|
||||
// The validated union is safe to interpolate into a non-bindable PRAGMA.
|
||||
// Apply it only after ownership validation and initialization commit.
|
||||
db.exec(`PRAGMA journal_mode = ${journalMode.toUpperCase()}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconstruct the {@link SessionHeader} from a `sessions` row.
|
||||
* @param row - the `sessions` table row.
|
||||
* @returns the header, `NULL` columns mapped to omitted optional fields.
|
||||
*/
|
||||
export function rowToMeta(row: SessionRow): SessionHeader {
|
||||
if (!Number.isSafeInteger(row.created_at) || row.created_at < 0) {
|
||||
throw new Error('stored session createdAt must be a non-negative safe integer')
|
||||
}
|
||||
return {
|
||||
version: row.version,
|
||||
id: row.id as SessionId,
|
||||
createdAt: row.created_at,
|
||||
...row.cwd !== null ? { cwd: row.cwd } : {},
|
||||
...row.parent_session !== null ? { parentSession: row.parent_session as SessionId } : {},
|
||||
...row.seed_length !== null ? { seedLength: row.seed_length } : {},
|
||||
...row.origin !== null ? { origin: row.origin } : {},
|
||||
...row.delegation_depth !== null ? { delegationDepth: row.delegation_depth } : {},
|
||||
...row.agent_preset !== null ? { agentPreset: row.agent_preset } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconstruct a {@link SessionEvent} from an `events` row (parses `data`).
|
||||
* @param row - the `events` table row; `data` and the surface columns hold JSON text.
|
||||
* @returns the reconstructed event; throws when a JSON column fails to parse
|
||||
* ({@link scanRows} treats that as a hole, not corruption, in the tail).
|
||||
*/
|
||||
export function rowToEvent(row: EventRow): SessionEvent {
|
||||
// Surface-metadata fields are conditional on the event type in the type
|
||||
// system; spread them so each variant gets only the fields it declares.
|
||||
const surfaceFields = {
|
||||
...row.source_event_seqs !== null ? { sourceEventSeqs: JSON.parse(row.source_event_seqs) as number[] } : {},
|
||||
...row.surface_op !== null ? { surfaceOp: JSON.parse(row.surface_op) as SurfaceOp } : {},
|
||||
}
|
||||
return {
|
||||
type: row.type as SessionEvent['type'],
|
||||
seq: row.seq,
|
||||
time: row.time,
|
||||
data: JSON.parse(row.data) as SessionEvent['data'],
|
||||
...surfaceFields,
|
||||
} as SessionEvent
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the preserved prefix of ordered event rows. Fully written rows in an
|
||||
* interrupted final turn remain in the prefix. The first unparsable row or seq
|
||||
* gap after the last `turn/end` marks a tolerated torn tail; the same hole in
|
||||
* the committed region rejects.
|
||||
*
|
||||
* @param rows - one session's event rows, ordered by seq ascending.
|
||||
* @param base - the seq the first row is expected to carry; `0` for a whole
|
||||
* log, the requested `fromSeq` for a suffix read (`loadStoredFrom`).
|
||||
* @returns the preserved event prefix, plus `tornFrom` — the seq the physical
|
||||
* delete starts at — when a torn tail exists.
|
||||
*/
|
||||
export function scanRows(rows: readonly EventRow[], base = 0): { preserved: SessionEvent[]; tornFrom?: number } {
|
||||
// Pass 1: parse each row's data; a row whose data is not valid JSON is a hole.
|
||||
// (The seq/type COLUMNS are always present even when `data` is corrupt.)
|
||||
interface Parsed { ok: boolean; event?: SessionEvent }
|
||||
const parsed: Parsed[] = rows.map((row) => {
|
||||
try {
|
||||
return { ok: true, event: rowToEvent(row) }
|
||||
} catch {
|
||||
return { ok: false }
|
||||
}
|
||||
})
|
||||
|
||||
// The last index that is a valid `turn/end` — holes through a closed turn
|
||||
// are always committed corruption.
|
||||
let lastTurnEnd = -1
|
||||
for (let i = parsed.length - 1; i >= 0; i--) {
|
||||
if (parsed[i]?.ok && rows[i]?.type === 'turn/end') { lastTurnEnd = i; break }
|
||||
}
|
||||
|
||||
// Preserve the contiguous prefix, including a complete interrupted turn;
|
||||
// holes through the last committed boundary throw, while later holes stop.
|
||||
const preserved: SessionEvent[] = []
|
||||
for (let i = 0; i < rows.length; i++) {
|
||||
const p = parsed[i]
|
||||
if (!p?.ok || p.event === undefined) {
|
||||
if (i <= lastTurnEnd) throw new Error(`corrupt session log: unparsable committed event at seq ${rows[i]?.seq}`)
|
||||
break // torn tail fragment after the last turn/end — stop, tolerate
|
||||
}
|
||||
if (p.event.seq !== base + i) {
|
||||
if (i <= lastTurnEnd) throw new Error(`corrupt session log: seq gap in committed region (expected ${base + i}, got ${p.event.seq})`)
|
||||
break // gap after the last turn/end — torn tail, stop
|
||||
}
|
||||
preserved.push(p.event)
|
||||
}
|
||||
|
||||
// Any rows past the preserved prefix are a never-committed torn tail; their
|
||||
// first seq is the deletion point for load's physical repair.
|
||||
return preserved.length < rows.length ? { preserved, tornFrom: base + preserved.length } : { preserved }
|
||||
}
|
||||
946
packages/session/session-persistence-sqlite/tests/sqlite.spec.ts
Normal file
946
packages/session/session-persistence-sqlite/tests/sqlite.spec.ts
Normal file
@@ -0,0 +1,946 @@
|
||||
import { createUserMessage, createMessage } from '@deepseek-ai/dsh-llm'
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { chmod, mkdtemp, rm, stat, symlink, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SurfaceEvent, SurfaceEventType } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistenceSqlite, { SCHEMA_VERSION } from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import {
|
||||
openDatabase,
|
||||
rowToEvent,
|
||||
rowToMeta,
|
||||
scanRows,
|
||||
SESSION_PERSISTENCE_SQLITE_APPLICATION_ID,
|
||||
type EventRow,
|
||||
} from '../src/schema.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
|
||||
const dirs: string[] = []
|
||||
afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) })
|
||||
|
||||
async function expectFlushError(promise: Promise<unknown>, message: RegExp): Promise<void> {
|
||||
try {
|
||||
await promise
|
||||
} catch (error) {
|
||||
expect(error).toBeInstanceOf(Error)
|
||||
expect((error as Error).message).toMatch(message)
|
||||
return
|
||||
}
|
||||
throw new Error('expected flush to reject')
|
||||
}
|
||||
|
||||
async function freshDbPath(): Promise<string> {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-'))
|
||||
dirs.push(dir)
|
||||
return join(dir, 'sessions.db')
|
||||
}
|
||||
|
||||
/** A context with the session store + SQLite backend, plus a teardown. */
|
||||
async function backend(path = ':memory:'): Promise<{ ctx: Context; dispose: () => Promise<void> }> {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
return { ctx, dispose: () => fiber.dispose() }
|
||||
}
|
||||
|
||||
// Run the same backend-agnostic contract as JSONL to pin identical semantics.
|
||||
runPersistenceContract('sqlite', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
return {
|
||||
persistence: ctx.sessionPersistence,
|
||||
dispose: async () => { await fiber.dispose() },
|
||||
}
|
||||
})
|
||||
|
||||
// A file-backed database lets two mounts share rows across reload. `corruptTail` inserts invalid
|
||||
// JSON past the committed seq, exercising coordinator repair against real database rows.
|
||||
runCoordinatorContract('sqlite', async (): Promise<CoordinatorFixture> => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-sqlite-coord-'))
|
||||
const path = join(dir, 'sessions.db')
|
||||
return {
|
||||
mount: async ctx => ctx.plugin(SessionPersistenceSqlite, { path }),
|
||||
corruptTail: async (id) => {
|
||||
// A row past the committed region whose `data` does not parse: scanRows
|
||||
// bounds the preserved prefix at it and returns its seq as tornFrom, which
|
||||
// the backend surfaces to the coordinator as the tornMarker to delete from.
|
||||
const db = openDatabase(path, 'wal')
|
||||
const next = (db.prepare('SELECT COALESCE(MAX(seq), -1) + 1 AS n FROM events WHERE session_id = ?')
|
||||
.get(id) as { n: number }).n
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
.run(id, next, 'assistant/chunk', 99, '{not valid json')
|
||||
db.close()
|
||||
},
|
||||
cleanup: async () => { await rm(dir, { recursive: true, force: true }) },
|
||||
}
|
||||
})
|
||||
|
||||
describe('scanRows', () => {
|
||||
// scanRows works off EventRows (data is a JSON string column); build them from SessionEvents
|
||||
// so the unit tests read in terms of the event vocabulary. Surface metadata is serialized to
|
||||
// its nullable columns so the conversion remains faithful.
|
||||
const rows = (events: SessionEvent[]): EventRow[] =>
|
||||
events.map((e) => {
|
||||
const se = e as SessionEvent<SurfaceEventType>
|
||||
return {
|
||||
seq: e.seq, type: e.type, time: e.time, data: JSON.stringify(e.data),
|
||||
source_event_seqs: se.sourceEventSeqs !== undefined ? JSON.stringify(se.sourceEventSeqs) : null,
|
||||
surface_op: se.surfaceOp !== undefined ? JSON.stringify(se.surfaceOp) : null,
|
||||
}
|
||||
})
|
||||
|
||||
it('preserves the full log when it ends exactly on a turn/end (no torn tail)', () => {
|
||||
const { preserved, tornFrom } = scanRows(rows(oneTurnLog()))
|
||||
expect(preserved).toEqual(oneTurnLog())
|
||||
expect(tornFrom).toBeUndefined()
|
||||
})
|
||||
|
||||
it('PRESERVES the real events of an interrupted turn after the last turn/end', () => {
|
||||
// turn 1 committed (0..5) + a crashed turn 2 (turn/start 6, step/start 7, no
|
||||
// close): all 8 rows are intact, so the whole prefix is preserved and there
|
||||
// is no torn fragment to delete. (load() then synthesizes the closers.)
|
||||
const withOpenTurn: SessionEvent[] = [
|
||||
...oneTurnLog(),
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
]
|
||||
const { preserved, tornFrom } = scanRows(rows(withOpenTurn))
|
||||
expect(preserved.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
expect(tornFrom).toBeUndefined()
|
||||
})
|
||||
|
||||
it('preserves the contiguous prefix and flags a torn tail at a seq gap', () => {
|
||||
// A gap after seq 0 (no committed turn/end): seq 0 is the preserved
|
||||
// interrupted-turn event; the gap bounds it and marks the torn fragment.
|
||||
const gapped: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing
|
||||
]
|
||||
const { preserved, tornFrom } = scanRows(rows(gapped))
|
||||
expect(preserved.map(e => e.seq)).toEqual([0])
|
||||
expect(tornFrom).toBe(1)
|
||||
})
|
||||
|
||||
it('an empty log preserves nothing and has no torn tail', () => {
|
||||
expect(scanRows([])).toEqual({ preserved: [] })
|
||||
})
|
||||
|
||||
it('throws on a seq gap inside the committed region (before the last turn/end)', () => {
|
||||
const gapped: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }, // seq 1 missing
|
||||
{ type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } },
|
||||
]
|
||||
expect(() => scanRows(rows(gapped))).toThrow(/seq gap in committed region/)
|
||||
})
|
||||
|
||||
it('throws on an unparsable row inside the committed region', () => {
|
||||
const withCorruptCommitted: EventRow[] = [
|
||||
{ seq: 0, type: 'turn/start', time: 1, data: '{not json', source_event_seqs: null, surface_op: null }, // corrupt, sits before a turn/end
|
||||
{ seq: 1, type: 'turn/end', time: 2, data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }), source_event_seqs: null, surface_op: null },
|
||||
]
|
||||
expect(() => scanRows(withCorruptCommitted)).toThrow(/unparsable committed event/)
|
||||
})
|
||||
|
||||
it('tolerates an unparsable torn-tail row after the last turn/end', () => {
|
||||
const withCorruptTail: EventRow[] = [
|
||||
...rows(oneTurnLog()),
|
||||
{ seq: 6, type: 'turn/start', time: 7, data: '{not json', source_event_seqs: null, surface_op: null }, // torn fragment, no committed turn/end after
|
||||
]
|
||||
const { preserved, tornFrom } = scanRows(withCorruptTail)
|
||||
expect(preserved).toEqual(oneTurnLog())
|
||||
expect(tornFrom).toBe(6)
|
||||
})
|
||||
})
|
||||
|
||||
describe('rowToMeta', () => {
|
||||
it('restores optional origin metadata', () => {
|
||||
expect(rowToMeta({
|
||||
id: 'with-origin',
|
||||
version: 0,
|
||||
created_at: 1,
|
||||
cwd: null,
|
||||
parent_session: null,
|
||||
seed_length: null,
|
||||
origin: 'subagent',
|
||||
incarnation: 'with-origin',
|
||||
revision: 1,
|
||||
delegation_depth: null,
|
||||
agent_preset: null,
|
||||
})).toMatchObject({ id: 'with-origin', origin: 'subagent' })
|
||||
})
|
||||
|
||||
it('rejects fractional stored creation metadata', () => {
|
||||
expect(() => rowToMeta({
|
||||
id: 'fractional',
|
||||
version: 0,
|
||||
created_at: 1.5,
|
||||
cwd: null,
|
||||
parent_session: null,
|
||||
seed_length: null,
|
||||
origin: null,
|
||||
incarnation: 'fractional',
|
||||
revision: 1,
|
||||
delegation_depth: null,
|
||||
agent_preset: null,
|
||||
})).toThrow('stored session createdAt must be a non-negative safe integer')
|
||||
})
|
||||
|
||||
it('restores the agent preset a session was composed from', () => {
|
||||
// The preset decides the resumed session's tools and prompt; a row that
|
||||
// dropped it would rebuild a composition the stored history contradicts.
|
||||
expect(rowToMeta({
|
||||
id: 'composed',
|
||||
version: 0,
|
||||
created_at: 1,
|
||||
cwd: null,
|
||||
parent_session: null,
|
||||
seed_length: null,
|
||||
origin: null,
|
||||
incarnation: 'composed',
|
||||
revision: 1,
|
||||
delegation_depth: null,
|
||||
agent_preset: 'minimal',
|
||||
})).toMatchObject({ agentPreset: 'minimal' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: durability and crash semantics', () => {
|
||||
it('rejects a stored v0 log containing a legacy request/header-delta event', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('legacy-header-delta', '/legacy')
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)')
|
||||
.run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-delta')
|
||||
const insert = db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
insert.run(m.id, 0, 'turn/start', 1, JSON.stringify({ turn: 1 }))
|
||||
insert.run(m.id, 1, 'request/header-delta', 2, JSON.stringify({ config: { model: 'legacy' } }))
|
||||
insert.run(m.id, 2, 'turn/end', 3, JSON.stringify({ turn: 1, reason: { kind: 'completed' } }))
|
||||
db.close()
|
||||
|
||||
const mounted = await backend(path)
|
||||
await expect(mounted.ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1/)
|
||||
await mounted.dispose()
|
||||
})
|
||||
|
||||
it('rejects a stored v0 full header carrying the legacy fallback reason', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('legacy-header-fallback', '/legacy')
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO sessions (id, version, created_at, cwd, parent_session, seed_length, delegation_depth, incarnation, revision) VALUES (?, ?, ?, ?, NULL, NULL, NULL, ?, 1)')
|
||||
.run(m.id, m.version, m.createdAt, m.cwd ?? null, 'legacy-header-fallback')
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, ?, ?, ?, ?)')
|
||||
.run(m.id, 0, 'request/header', 1, JSON.stringify({
|
||||
header: { config: { model: 'legacy' } },
|
||||
reason: 'fallback',
|
||||
}))
|
||||
db.close()
|
||||
|
||||
const mounted = await backend(path)
|
||||
await expect(mounted.ctx.sessionPersistence.load(m.id))
|
||||
.rejects.toThrow(/unsupported legacy request\/header reason "fallback" at seq 0/)
|
||||
await mounted.dispose()
|
||||
})
|
||||
|
||||
it('has no independent per-session log location', async () => {
|
||||
const { ctx, dispose } = await backend()
|
||||
expect(ctx.sessionPersistence.locate(meta('sqlite-location'))).toBeUndefined()
|
||||
await dispose()
|
||||
})
|
||||
|
||||
it('an interrupted turn (rows after the last turn/end) is PRESERVED and closed during load', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('crash')
|
||||
// Run 1: persist a complete turn, then a half-written second turn (no turn/end).
|
||||
const ctx1 = new Context()
|
||||
await ctx1.plugin(SessionStore)
|
||||
const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path })
|
||||
await ctx1.sessionPersistence.create(m)
|
||||
await ctx1.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await ctx1.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } },
|
||||
{ type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } },
|
||||
])
|
||||
await fiber1.dispose()
|
||||
|
||||
// Run 2: load PRESERVES the interrupted turn's real events (a turn can be huge
|
||||
// — never truncated) and closes the orphaned turn with synthetic boundary
|
||||
// events: step/end (the step was open) then turn/end {interrupted}.
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
const loaded = await ctx2.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.type)).toEqual([
|
||||
'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1
|
||||
'turn/start', 'step/start', 'step/end', 'turn/end', // turn 2: real events + synthetic closers
|
||||
])
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
|
||||
const last = loaded.events.at(-1)!
|
||||
expect(last.type === 'turn/end' && last.data.reason).toEqual({ kind: 'interrupted' })
|
||||
|
||||
// load durably closed the turn, so the next append continues at the balanced
|
||||
// length (seq 10) and a reload round-trips identically.
|
||||
await ctx2.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 10, time: 9, data: { turn: 3 } },
|
||||
{ type: 'turn/end', seq: 11, time: 10, data: { turn: 3, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await ctx2.sessionPersistence.load(m.id)
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('load() durably closes the interrupted turn: the synthetic closers are on disk after load', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('load-closes')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5
|
||||
await b1.dispose()
|
||||
// Hand-write an interrupted turn (turn/start seq 6, no turn/end).
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)')
|
||||
.run(m.id, 'turn/start', JSON.stringify({ turn: 2 }))
|
||||
db.close()
|
||||
|
||||
const b2 = await backend(path)
|
||||
const loaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
// turn 2's real turn/start (seq 6) is preserved + a synthetic turn/end (seq 7).
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
expect(loaded.events.at(-1)!.type).toBe('turn/end')
|
||||
// load() is mutating: the synthetic turn/end MUST be on disk so the stored log
|
||||
// is balanced and the cursor is truthful (contract: load closes, not defers).
|
||||
const probe = openDatabase(path, 'wal')
|
||||
const stored = probe.prepare('SELECT seq, type FROM events WHERE session_id = ? ORDER BY seq').all(m.id) as { seq: number; type: string }[]
|
||||
probe.close()
|
||||
expect(stored.map(r => r.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
expect(stored.at(-1)!.type).toBe('turn/end')
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('all-tail load: a session whose only turn never closed is preserved and closed on load', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('all-tail')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
// A first turn that NEVER completed: turn/start + user/message, no turn/end.
|
||||
await b1.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
{ type: 'user/message', seq: 1, time: 2, data: createUserMessage({
|
||||
content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' },
|
||||
}), surfaceOp: 'append' },
|
||||
])
|
||||
await b1.dispose()
|
||||
|
||||
// A fresh backend loads it: the interrupted (only) turn's real events are
|
||||
// preserved and closed with a synthetic turn/end {interrupted} — NOT
|
||||
// truncated. The session was materialized, so list() reports it present.
|
||||
const b2 = await backend(path)
|
||||
const loaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.type)).toEqual(['turn/start', 'user/message', 'turn/end'])
|
||||
expect(loaded.events.at(-1)!.type === 'turn/end' && loaded.events.at(-1)!.data).toMatchObject({ reason: { kind: 'interrupted' } })
|
||||
expect((await b2.ctx.sessionPersistence.list()).map(x => x.id)).toContain(m.id)
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('rejects opening a database whose schema version is not the current build (newer OR older)', async () => {
|
||||
const path = await freshDbPath()
|
||||
openDatabase(path, 'wal').close() // stamp user_version = SCHEMA_VERSION
|
||||
// Bump user_version past what this build supports.
|
||||
const dbNewer = openDatabase(path, 'wal')
|
||||
dbNewer.exec(`PRAGMA user_version = ${SCHEMA_VERSION + 1}`)
|
||||
dbNewer.close()
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/incompatible with this build/)
|
||||
|
||||
// The immediately preceding layout lacks the required store identity and is
|
||||
// rejected rather than migrated (unreleased software, no backward-compat).
|
||||
const olderPath = await freshDbPath()
|
||||
openDatabase(olderPath, 'wal').close()
|
||||
const dbOlder = openDatabase(olderPath, 'wal')
|
||||
dbOlder.exec(`PRAGMA user_version = ${SCHEMA_VERSION - 1}`)
|
||||
dbOlder.close()
|
||||
expect(() => openDatabase(olderPath, 'wal')).toThrow(/incompatible with this build/)
|
||||
})
|
||||
|
||||
it('rejects a table-backed unversioned database before stamping or changing journal mode', async () => {
|
||||
const path = await freshDbPath()
|
||||
const legacy = new DatabaseSync(path)
|
||||
legacy.exec('CREATE TABLE sessions (id TEXT PRIMARY KEY)')
|
||||
legacy.close()
|
||||
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/unversioned schema or application identity/)
|
||||
|
||||
const unchanged = new DatabaseSync(path)
|
||||
expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 })
|
||||
expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
expect(unchanged.prepare(
|
||||
"SELECT name FROM sqlite_schema WHERE type = 'table' AND name = 'sessions'",
|
||||
).get()).toEqual({ name: 'sessions' })
|
||||
unchanged.close()
|
||||
})
|
||||
|
||||
it('counts a sqliteX table as user-owned instead of mistaking it for SQLite metadata', async () => {
|
||||
const path = await freshDbPath()
|
||||
const unrelated = new DatabaseSync(path)
|
||||
unrelated.exec('CREATE TABLE sqliteX (value TEXT)')
|
||||
unrelated.exec("INSERT INTO sqliteX VALUES ('safe')")
|
||||
unrelated.close()
|
||||
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/unversioned schema or application identity/)
|
||||
|
||||
const unchanged = new DatabaseSync(path)
|
||||
expect(unchanged.prepare('SELECT value FROM sqliteX').get()).toEqual({ value: 'safe' })
|
||||
expect(unchanged.prepare('PRAGMA application_id').get()).toEqual({ application_id: 0 })
|
||||
expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 })
|
||||
expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
unchanged.close()
|
||||
})
|
||||
|
||||
it('rejects view-only and foreign-application unversioned databases without mutation', async () => {
|
||||
const viewPath = await freshDbPath()
|
||||
const viewOnly = new DatabaseSync(viewPath)
|
||||
viewOnly.exec('CREATE VIEW foreign_view AS SELECT 1 AS value')
|
||||
viewOnly.close()
|
||||
|
||||
expect(() => openDatabase(viewPath, 'wal')).toThrow(/unversioned schema or application identity/)
|
||||
const unchangedView = new DatabaseSync(viewPath)
|
||||
expect(unchangedView.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
expect(unchangedView.prepare(
|
||||
"SELECT type FROM sqlite_schema WHERE name = 'foreign_view'",
|
||||
).get()).toEqual({ type: 'view' })
|
||||
unchangedView.close()
|
||||
|
||||
const applicationPath = await freshDbPath()
|
||||
const foreignApplication = new DatabaseSync(applicationPath)
|
||||
foreignApplication.exec('PRAGMA application_id = 12345')
|
||||
foreignApplication.close()
|
||||
|
||||
expect(() => openDatabase(applicationPath, 'wal')).toThrow(/unversioned schema or application identity/)
|
||||
const unchangedApplication = new DatabaseSync(applicationPath)
|
||||
expect(unchangedApplication.prepare('PRAGMA application_id').get()).toEqual({ application_id: 12345 })
|
||||
expect(unchangedApplication.prepare('PRAGMA user_version').get()).toEqual({ user_version: 0 })
|
||||
expect(unchangedApplication.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
unchangedApplication.close()
|
||||
})
|
||||
|
||||
it('rejects a current-version database with a foreign application identity', async () => {
|
||||
const path = await freshDbPath()
|
||||
const foreign = new DatabaseSync(path)
|
||||
foreign.exec('PRAGMA application_id = 12345')
|
||||
foreign.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`)
|
||||
foreign.close()
|
||||
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/has application id 12345/)
|
||||
|
||||
const unchanged = new DatabaseSync(path)
|
||||
expect(unchanged.prepare('PRAGMA application_id').get()).toEqual({ application_id: 12345 })
|
||||
expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION })
|
||||
expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
unchanged.close()
|
||||
})
|
||||
|
||||
it('rolls back schema objects and identity stamps when initialization fails', async () => {
|
||||
const path = await freshDbPath()
|
||||
const conflicting = new DatabaseSync(path)
|
||||
conflicting.exec(`PRAGMA application_id = ${SESSION_PERSISTENCE_SQLITE_APPLICATION_ID}`)
|
||||
conflicting.exec(`PRAGMA user_version = ${SCHEMA_VERSION}`)
|
||||
conflicting.exec("CREATE VIEW persistence_state AS SELECT 1 AS singleton, 'foreign' AS store_id")
|
||||
conflicting.close()
|
||||
|
||||
expect(() => openDatabase(path, 'wal')).toThrow()
|
||||
|
||||
const unchanged = new DatabaseSync(path)
|
||||
expect(unchanged.prepare(
|
||||
"SELECT type FROM sqlite_schema WHERE name = 'persistence_state'",
|
||||
).get()).toEqual({ type: 'view' })
|
||||
expect(unchanged.prepare(
|
||||
"SELECT type FROM sqlite_schema WHERE name = 'sessions'",
|
||||
).get()).toBeUndefined()
|
||||
expect(unchanged.prepare(
|
||||
"SELECT type FROM sqlite_schema WHERE name = 'events'",
|
||||
).get()).toBeUndefined()
|
||||
expect(unchanged.prepare('PRAGMA application_id').get())
|
||||
.toEqual({ application_id: SESSION_PERSISTENCE_SQLITE_APPLICATION_ID })
|
||||
expect(unchanged.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION })
|
||||
expect(unchanged.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'delete' })
|
||||
unchanged.close()
|
||||
})
|
||||
|
||||
it('stamps the persistence application identity with the schema version', async () => {
|
||||
const path = await freshDbPath()
|
||||
openDatabase(path, 'wal').close()
|
||||
|
||||
const db = new DatabaseSync(path)
|
||||
expect(db.prepare('PRAGMA application_id').get())
|
||||
.toEqual({ application_id: SESSION_PERSISTENCE_SQLITE_APPLICATION_ID })
|
||||
expect(db.prepare('PRAGMA user_version').get()).toEqual({ user_version: SCHEMA_VERSION })
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('rejects a sibling v3 database (the merge-collided version) rather than opening it against missing columns', async () => {
|
||||
// Version 3 identified two incompatible sibling layouts, so it is always rejected.
|
||||
const path = await freshDbPath()
|
||||
openDatabase(path, 'wal').close() // creates + stamps user_version = SCHEMA_VERSION
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.exec('PRAGMA user_version = 3')
|
||||
db.close()
|
||||
expect(() => openDatabase(path, 'wal')).toThrow(/schema version 3, incompatible with this build/)
|
||||
})
|
||||
|
||||
it('a corrupt-JSON row in the uncommitted tail is discarded on load, not unloadable', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('corrupt-tail')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog()) // committed: seqs 0..5
|
||||
await b1.dispose()
|
||||
|
||||
// A torn row after the last committed turn has invalid JSON. `scanRows` locates the boundary
|
||||
// from seq/type columns without parsing the tail, preserves the committed prefix, and load
|
||||
// deletes the row; invalid JSON inside the committed region would remain fatal.
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.prepare('INSERT INTO events (session_id, seq, type, time, data) VALUES (?, 6, ?, 7, ?)')
|
||||
.run(m.id, 'turn/start', '{not valid json')
|
||||
db.close()
|
||||
|
||||
const b2 = await backend(path)
|
||||
const loaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toEqual(oneTurnLog()) // torn tail discarded, committed intact (turn 1 already balanced → no closers)
|
||||
// load physically deleted the corrupt tail row, so a fresh append continues.
|
||||
await b2.ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: 6, time: 8, data: { turn: 2 } },
|
||||
{ type: 'turn/end', seq: 7, time: 9, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
])
|
||||
const reloaded = await b2.ctx.sessionPersistence.load(m.id)
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('append rolls back the whole batch on a mid-batch seq collision (transaction)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const m = meta('rollback')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog()) // seqs 0..5
|
||||
|
||||
// A batch that re-states an already-stored seq must be rejected and leave
|
||||
// the stored log unchanged (the UNIQUE (session_id, seq) constraint fires
|
||||
// inside the transaction → ROLLBACK).
|
||||
await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toThrow()
|
||||
const loaded = await ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events).toEqual(oneTurnLog()) // unchanged
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('persists across separate backend instances over the same file', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('persist', '/proj')
|
||||
const ctx1 = new Context()
|
||||
await ctx1.plugin(SessionStore)
|
||||
const fiber1 = await ctx1.plugin(SessionPersistenceSqlite, { path })
|
||||
await ctx1.sessionPersistence.create(m)
|
||||
await ctx1.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await fiber1.dispose()
|
||||
|
||||
const ctx2 = new Context()
|
||||
await ctx2.plugin(SessionStore)
|
||||
const fiber2 = await ctx2.plugin(SessionPersistenceSqlite, { path })
|
||||
expect((await ctx2.sessionPersistence.list()).map(x => x.id)).toContain(m.id)
|
||||
const loaded = await ctx2.sessionPersistence.load(m.id)
|
||||
expect(loaded.meta).toMatchObject({ id: m.id, cwd: '/proj' })
|
||||
expect(loaded.events).toEqual(oneTurnLog())
|
||||
await fiber2.dispose()
|
||||
})
|
||||
|
||||
it('source-qualifies revisions across stores while preserving same-file reopen identity', async () => {
|
||||
const pathA = await freshDbPath()
|
||||
const pathB = await freshDbPath()
|
||||
const m = meta('revision-source')
|
||||
const a = await backend(pathA)
|
||||
await a.ctx.sessionPersistence.create(m)
|
||||
await a.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const revisionA = (await a.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
await a.dispose()
|
||||
|
||||
const probeA = openDatabase(pathA, 'wal')
|
||||
const storeIdA = (probeA.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string }).store_id
|
||||
probeA.close()
|
||||
|
||||
const aliasA = `${pathA}.alias`
|
||||
await symlink(pathA, aliasA)
|
||||
const reopenedA = await backend(aliasA)
|
||||
expect((await reopenedA.ctx.sessionPersistence.listSnapshots())[0]?.revision).toBe(revisionA)
|
||||
await reopenedA.dispose()
|
||||
|
||||
const b = await backend(pathB)
|
||||
await b.ctx.sessionPersistence.create(m)
|
||||
await b.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const revisionB = (await b.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
const probeB = openDatabase(pathB, 'wal')
|
||||
const storeIdB = (probeB.prepare(
|
||||
'SELECT store_id FROM persistence_state WHERE singleton = 1',
|
||||
).get() as { store_id: string }).store_id
|
||||
probeB.close()
|
||||
expect(storeIdB).not.toBe(storeIdA)
|
||||
expect(revisionB).not.toBe(revisionA)
|
||||
expect(String(revisionA)).toMatch(/:revision:1$/)
|
||||
expect(String(revisionB)).toMatch(/:revision:1$/)
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('binds a full stored prefix to the same revision as a lightweight read', async () => {
|
||||
const b = await backend()
|
||||
const m = meta('stored-prefix-revision')
|
||||
await b.ctx.sessionPersistence.create(m)
|
||||
await b.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = b.ctx.sessionPersistence as SessionPersistenceSqlite
|
||||
|
||||
const stored = await persistence.loadStored(m.id)
|
||||
expect(stored?.revision).toBe(await persistence.readStoredRevision(m.id))
|
||||
expect(await persistence.readStoredRevision(SessionId('missing-revision'))).toBeUndefined()
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('changes revisions when a deleted session id is materialized again in the same database', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('recreated-revision')
|
||||
const first = await backend(path)
|
||||
await first.ctx.sessionPersistence.create(m)
|
||||
await first.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const before = (await first.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
await first.dispose()
|
||||
|
||||
const cleanup = openDatabase(path, 'wal')
|
||||
cleanup.prepare('DELETE FROM sessions WHERE id = ?').run(m.id)
|
||||
cleanup.close()
|
||||
|
||||
const second = await backend(path)
|
||||
await second.ctx.sessionPersistence.create(m)
|
||||
await second.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const after = (await second.ctx.sessionPersistence.listSnapshots())[0]?.revision
|
||||
expect(after).not.toBe(before)
|
||||
expect(String(before)).toMatch(/:revision:1$/)
|
||||
expect(String(after)).toMatch(/:revision:1$/)
|
||||
await second.dispose()
|
||||
})
|
||||
|
||||
it('awaits in-flight readiness before surfacing snapshot-list cancellation', async () => {
|
||||
const b = await backend()
|
||||
const internals = b.ctx.sessionPersistence as unknown as { ready: Promise<void> }
|
||||
const originalReady = internals.ready
|
||||
const readiness = Promise.withResolvers<undefined>()
|
||||
internals.ready = readiness.promise
|
||||
const reason = new Error('SQLite snapshot readiness cancelled')
|
||||
const controller = new AbortController()
|
||||
const pending = b.ctx.sessionPersistence.listSnapshots(controller.signal)
|
||||
let settled = false
|
||||
void pending.then(
|
||||
() => { settled = true },
|
||||
() => { settled = true },
|
||||
)
|
||||
|
||||
controller.abort(reason)
|
||||
await Promise.resolve()
|
||||
expect(settled).toBe(false)
|
||||
|
||||
readiness.resolve(undefined)
|
||||
await expect(pending).rejects.toBe(reason)
|
||||
internals.ready = originalReady
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('exposes the schema version constant', () => {
|
||||
expect(SCHEMA_VERSION).toBe(14)
|
||||
})
|
||||
|
||||
it('keeps the revision stable for an empty repair hook', async () => {
|
||||
const b = await backend()
|
||||
const m = meta('empty-repair')
|
||||
await b.ctx.sessionPersistence.create(m)
|
||||
await b.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const before = await b.ctx.sessionPersistence.listSnapshots()
|
||||
await (b.ctx.sessionPersistence as SessionPersistenceSqlite).commitRepair(m, undefined, [])
|
||||
expect(await b.ctx.sessionPersistence.listSnapshots()).toEqual(before)
|
||||
await b.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite: edge cases', () => {
|
||||
it('resolves the preparation-cache default without schema normalization', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
let persistence!: SessionPersistenceSqlite
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
persistence = new SessionPersistenceSqlite(inner, {
|
||||
path: ':memory:',
|
||||
journalMode: 'wal',
|
||||
})
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
expect(await persistence.list()).toEqual([])
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('uses the configured preparation cache through the public service', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, {
|
||||
path: ':memory:',
|
||||
preparedSessionCacheSize: 1,
|
||||
writeBatchMaxDelayMs: 1,
|
||||
})
|
||||
const m = meta('sqlite-preparation-cache')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
|
||||
const preparation = await ctx.sessionPersistence.prepare(m.id)
|
||||
expect(preparation.session.header).toEqual(m)
|
||||
preparation[Symbol.dispose]()
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('rejects and closes a current-schema database with an invalid store identity', async () => {
|
||||
const path = await freshDbPath()
|
||||
const db = openDatabase(path, 'wal')
|
||||
db.exec("UPDATE persistence_state SET store_id = '' WHERE singleton = 1")
|
||||
db.close()
|
||||
|
||||
const b = await backend(path)
|
||||
await expect(b.ctx.sessionPersistence.listSnapshots()).rejects.toThrow(/no valid store identity/)
|
||||
await expect(b.dispose()).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('creates a new database and WAL sidecars with owner-only modes without changing its parent mode', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
const dir = dirname(path)
|
||||
await chmod(dir, 0o755)
|
||||
|
||||
const b = await backend(path)
|
||||
await b.ctx.sessionPersistence.list()
|
||||
|
||||
expect((await stat(dir)).mode & 0o777).toBe(0o755)
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-wal`)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-shm`)).mode & 0o777).toBe(0o600)
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('creates a persistent rollback journal with owner-only mode', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path, journalMode: 'persist' })
|
||||
const m = meta('persist-permissions')
|
||||
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o600)
|
||||
expect((await stat(`${path}-journal`)).mode & 0o777).toBe(0o600)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('preserves the mode of an existing database file', async () => {
|
||||
if (process.platform === 'win32') return
|
||||
const path = await freshDbPath()
|
||||
await writeFile(path, '', { mode: 0o644 })
|
||||
await chmod(path, 0o644)
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path, journalMode: 'delete' })
|
||||
await ctx.sessionPersistence.list()
|
||||
|
||||
expect((await stat(path)).mode & 0o777).toBe(0o644)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('surfaces an invalid database path during pre-creation', async () => {
|
||||
const path = await freshDbPath()
|
||||
const b = await backend(`${path}\0`)
|
||||
|
||||
await expect(b.ctx.sessionPersistence.list()).rejects.toMatchObject({ code: 'ERR_INVALID_ARG_VALUE' })
|
||||
await b.dispose()
|
||||
})
|
||||
|
||||
it('append rolls back and rethrows when an event INSERT fails inside the transaction', async () => {
|
||||
const path = await freshDbPath()
|
||||
const m = meta('rollback-insert')
|
||||
const b1 = await backend(path)
|
||||
await b1.ctx.sessionPersistence.create(m)
|
||||
await b1.ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
|
||||
// A SECOND backend over the same file loads the session first, so it adopts
|
||||
// cursor 6 (the committed length) into its OWN in-memory state.
|
||||
const b2 = await backend(path)
|
||||
await b2.ctx.sessionPersistence.load(m.id) // cursor 6 in b2
|
||||
const turn2: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
]
|
||||
// b1 commits seq 6..7 first.
|
||||
await b1.ctx.sessionPersistence.append(m.id, turn2)
|
||||
// b2 still thinks its cursor is 6, so this batch passes the contiguity check
|
||||
// but its INSERT of seq 6 hits the UNIQUE (session_id, seq) constraint
|
||||
// mid-transaction → ROLLBACK + rethrow.
|
||||
await expect(b2.ctx.sessionPersistence.append(m.id, turn2)).rejects.toThrow(/UNIQUE/)
|
||||
// b1's turn is intact; b2's rolled-back attempt left nothing extra.
|
||||
const loaded = await b1.ctx.sessionPersistence.load(m.id)
|
||||
expect(loaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
await b1.dispose()
|
||||
await b2.dispose()
|
||||
})
|
||||
|
||||
it('journalMode config reaches the database (default wal, rollback modes selectable)', async () => {
|
||||
// :memory: databases always report journal_mode=memory, so probe file DBs.
|
||||
const walPath = await freshDbPath()
|
||||
const bWal = await backend(walPath)
|
||||
await bWal.ctx.sessionPersistence.create(meta('jm-wal'))
|
||||
const probe = openDatabase(walPath, 'wal')
|
||||
expect((probe.prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('wal')
|
||||
probe.close()
|
||||
await bWal.dispose()
|
||||
|
||||
const deletePath = await freshDbPath()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: deletePath, journalMode: 'delete' })
|
||||
await ctx.sessionPersistence.create(meta('jm-delete'))
|
||||
// Probe through a second connection: journal_mode=delete is a per-database
|
||||
// property only insofar as no WAL files exist — assert the world, not the
|
||||
// backend's self-report (no -wal sidecar after writes in delete mode).
|
||||
const db = openDatabase(deletePath, 'delete')
|
||||
expect((db.prepare('PRAGMA journal_mode').get() as { journal_mode: string }).journal_mode).toBe('delete')
|
||||
db.close()
|
||||
expect(existsSync(`${deletePath}-wal`)).toBe(false)
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('HMR: a DIFFERENT session colliding with a materialized on-disk id is rejected', async () => {
|
||||
const path = await freshDbPath()
|
||||
// Instance 1 materializes a session and disposes.
|
||||
const b1 = await backend(path)
|
||||
const s1 = b1.ctx.sessions.create(SessionId('hmr-collide'))
|
||||
appendLog(s1, oneTurnLog())
|
||||
await b1.ctx.sessions.flush(s1)
|
||||
await b1.dispose()
|
||||
|
||||
// A fresh context with an UNRELATED live session reusing the id meets a
|
||||
// materialized row that is NOT a prefix of its events → reject.
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
let session!: Session
|
||||
await ctx.plugin(Object.assign((inner: Context) => {
|
||||
session = inner.sessions.create(SessionId('hmr-collide'))
|
||||
}, { inject: ['sessions'] }))
|
||||
session.append('turn/start', { turn: 1 })
|
||||
await ctx.plugin(SessionPersistenceSqlite, { path })
|
||||
await expectFlushError(ctx.sessions.flush(session), /id collision/)
|
||||
await ctx.fiber.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('surface field round-trip', () => {
|
||||
it('rowToEvent parses surface fields from EventRow columns', () => {
|
||||
const row: EventRow = {
|
||||
seq: 0, type: 'assistant/message', time: 1,
|
||||
data: JSON.stringify({ turn: 1, step: 1, content: [] }),
|
||||
source_event_seqs: JSON.stringify([3, 5]),
|
||||
surface_op: JSON.stringify('append'),
|
||||
}
|
||||
const event = rowToEvent(row)
|
||||
expect((event as SurfaceEvent).sourceEventSeqs).toEqual([3, 5])
|
||||
expect((event as SurfaceEvent).surfaceOp).toBe('append')
|
||||
})
|
||||
|
||||
it('rowToEvent handles replace surfaceOp object', () => {
|
||||
const row: EventRow = {
|
||||
seq: 0, type: 'assistant/message', time: 1,
|
||||
data: JSON.stringify({ turn: 1, step: 1, content: [] }),
|
||||
source_event_seqs: JSON.stringify([0, 1]),
|
||||
surface_op: JSON.stringify({ op: 'replace', start: 0, end: 1 }),
|
||||
}
|
||||
const event = rowToEvent(row)
|
||||
expect((event as SurfaceEvent).sourceEventSeqs).toEqual([0, 1])
|
||||
expect((event as SurfaceEvent).surfaceOp).toEqual({ op: 'replace', start: 0, end: 1 })
|
||||
})
|
||||
|
||||
it('scanRows with surface columns reconstructs events with surface fields', () => {
|
||||
const rows: EventRow[] = [
|
||||
{ seq: 0, type: 'user/message', time: 1,
|
||||
data: JSON.stringify({ content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }),
|
||||
source_event_seqs: null, surface_op: '{"op":"replace","start":0,"end":0}' },
|
||||
{ seq: 1, type: 'turn/end', time: 2,
|
||||
data: JSON.stringify({ turn: 1, reason: { kind: 'completed' } }),
|
||||
source_event_seqs: null, surface_op: null },
|
||||
]
|
||||
const { preserved } = scanRows(rows)
|
||||
expect(preserved).toHaveLength(2)
|
||||
expect((preserved[0]! as SurfaceEvent).surfaceOp).toEqual({ op: 'replace', start: 0, end: 0 })
|
||||
expect((preserved[0]! as SurfaceEvent).sourceEventSeqs).toBeUndefined()
|
||||
expect((preserved[1] as SessionEvent<SurfaceEventType>).surfaceOp).toBeUndefined()
|
||||
})
|
||||
|
||||
it('append and load round-trips surface fields through SQLite', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const session = ctx.sessions.create(SessionId('roundtrip-surface'))
|
||||
session.append('turn/start', { turn: 1 })
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('user/message', createUserMessage({
|
||||
content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' })
|
||||
session.append('assistant/message', {
|
||||
turn: 1, step: 1,
|
||||
message: createMessage({
|
||||
role: 'assistant',
|
||||
content: [],
|
||||
source: {
|
||||
kind: 'model',
|
||||
...{ provider: 'mock', model: 'mock' },
|
||||
},
|
||||
}),
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [2] })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('roundtrip-surface'))
|
||||
expect(loaded.events).toHaveLength(6)
|
||||
const um = loaded.events[2]!
|
||||
expect((um as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((um as SurfaceEvent).sourceEventSeqs).toBeUndefined()
|
||||
const am = loaded.events[3]!
|
||||
expect((am as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((am as SurfaceEvent).sourceEventSeqs).toEqual([2])
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('persists events with surfaceOp but no sourceEventSeqs (covers null branch in surfaceBindings)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(SessionPersistenceSqlite, { path: ':memory:' })
|
||||
const session = ctx.sessions.create(SessionId('surface-noseq'))
|
||||
session.append('turn/start', { turn: 1 })
|
||||
session.append('user/message', createUserMessage({
|
||||
content: [],
|
||||
source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.sessions.flush(session)
|
||||
const loaded = await ctx.sessionPersistence.load(SessionId('surface-noseq'))
|
||||
expect((loaded.events[1]! as SurfaceEvent).surfaceOp).toBe('append')
|
||||
expect((loaded.events[1]! as SurfaceEvent).sourceEventSeqs).toBeUndefined()
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
30
packages/session/session-persistence-sqlite/tsconfig.json
Normal file
30
packages/session/session-persistence-sqlite/tsconfig.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../session-persistence"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user