refactor(session): fold the session family into packages/session/

git mv the 12 packages from session-persistence/, session-projection/,
session-title/, and telemetry/ into one session/ group per the
regrouping RFC; merge the four group READMEs into one bilingual
triplet; rewrite the group segment in tsconfig references (intra-group
references shorten to ../<pkg>), tsconfig.base.json paths/globs,
knip.json keys, vitest include, gate scripts, and authored doc/note
citations; regenerate module graph, doc graphs, catalogs, and the
lockfile importer keys. No npm names change.

Full unit suite: 8779 passed; the 18 reported failures reproduce as
env flakes (ambient-proxy IPv6 tunneling, watched-dir inotify
timeouts under parallel load) — each passes in isolation with
NO_PROXY set, matching their known pre-existing behavior on master.
This commit is contained in:
Tianyi Cui
2026-07-30 01:52:06 +08:00
parent 645fcf5713
commit 7e445c3a67
220 changed files with 258 additions and 286 deletions

View File

@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/session-projection/session-projection-cache/README.md
README.md: 5d4ad07fab6648acdb40c6aa86d32cc78b4c016e
README.zh.md: 827480658b8c69647380a782c1a983b390380108

View File

@@ -0,0 +1,62 @@
# @deepseek-ai/dsh-session-projection-cache
English | [中文](README.zh.md)
The persisted projection cache (`ctx.sessionProjectionCache`): durable checkpoints of every registered projection unit's state, one record per session on the domain data form (`session_projcache` domain — the shipped json backend lands it beside `workspace.json` under the configured storage root). Design authority: the [session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md) (persisted projection cache section).
A stored row `(key → {ver, seq, val})` is a fold shortcut, never an authority: possibly stale (`seq` says exactly how stale) but never wrong. Consequences the implementation commits to:
- **Every background write is fail-soft.** A failed durable write logs a warning and keeps the cache stale; the next write or cold read self-heals. A crash between writes costs a longer tail replay, never a wrong value.
- **A `ver` mismatch against the live unit's `stateVersion` discards, never migrates.** A unit bump invalidates its rows at read time; the key refolds from the log.
- **Whole-record writes.** Each write replaces the session's full checkpoint (the registry cut is always complete), snapshotted through the lossless-JSON boundary — a unit state violating the plain-JSON contract fails loud.
- **Records are bound to a log lifecycle, not just an id.** Each record stores the header identity (`createdAt`, `cwd`) it was folded from; every read validates it (the live or stored header is the witness) before accepting a row, so a deleted-then-recreated id or a persistence store swapped under a surviving cache discards the unrelated record instead of seeding phantom values.
- **The log leads, the cache follows.** A live checkpoint flushes the session's buffered events durably BEFORE the cache row lands, so a crash can leave the cache behind the log (a longer tail replay) but never ahead of it.
## Write policy
Two mandatory points, throttled in between:
| Trigger | Nature |
|---|---|
| `turn/end` | Mandatory — the turn-final value is what cold reads want. |
| Session disposal (detach) | Mandatory — the live-to-cold moment; after it the cold ladder serves this session. |
| `writeEveryEvents` committed events | Config throttle (count). |
| `writeIntervalMs` since the first dirty event | Config throttle (interval). |
Both `Config` fields are required (no defaults): flush cadence is a deployment choice with no universally correct value, stated in cordis.yml.
## Listing read (`cachedSnapshot(meta)`)
The zero-I/O rung: whole values viewed straight from the identity-matching stored record (version-matching keys only), returned as a `{asOfSeq, values}` cut — `asOfSeq` is the lowest served-row watermark, so a client seeding its per-session value store under higher-seq-wins can never let a stale list block overwrite a newer push frame. `undefined` when no usable record exists (unknown id, unrelated lifecycle, or no version-matching rows); the api-proxy list carrier turns that into an absent column.
## Cold read (`coldSnapshot(id, signal?)`)
The read ladder, zero full-log load on the happy path: cached rows → `sessionProjections.restoreFloor` (anchored one event below the lowest usable watermark) → persistence `readFrom(id, floor)``sessionProjections.restore` → fail-soft write-back of the refreshed rows. The anchor makes a shrunk log (crash-repair truncation) provable: an overreaching row triggers exactly one full re-read from seq 0 instead of serving a ghost value. No registered units serve `{asOfSeq: -1, values: {}}` without touching persistence; a session with no persisted log rejects with the seam's `not found`.
`write(session)` is the synchronous-cut checkpoint both mandatory points use; carriers may call it directly (not fail-soft — the fail-soft wrappers own containment).
## Composition
```yaml
- id: session-projection-cache
name: '@deepseek-ai/dsh-session-projection-cache'
config:
writeEveryEvents: 200
writeIntervalMs: 5000
```
Injects `storageDomain`, `sessionProjections`, `sessionPersistence`, `sessions`. Without this row the projection system runs live-only (watermark cache; cold reads fall back to full log loads wherever a carrier implements them).
## Model Experience
None, as the cache only persists and restores host-side read models of already-logged session state and touches no prompt, message, schema, stream, or tool result.
#### KV Cache effect
None; the cache never assembles or sends provider requests.
## Known Limitations and Deferred Work
- **No eviction or retention surface** — records accumulate per session; pruning stored checkpoints is out-of-band maintenance, same stance as session persistence itself.
- **Interval throttle is per-session coarse** — the timer arms at the first dirty event after a clean write; a steady sub-threshold trickle writes once per interval, not a sliding window.
- **`coldSnapshot` reads are not deduplicated** — two concurrent cold reads of one session each run the ladder; last write-back wins (rows are equivalent), acceptable for listing-scale call rates.

View File

@@ -0,0 +1,62 @@
# @deepseek-ai/dsh-session-projection-cache
[English](README.md) | 中文
持久投影缓存(`ctx.sessionProjectionCache`把每个已注册投影单元的状态持久化为检查点基于域数据形态domain data form每会话一条记录`session_projcache` 域——出厂 JSON 后端将其落在配置的存储根目录下、`workspace.json` 旁边)。设计权威:[session-projection RFC](../../../.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md)persisted projection cache 一节)。
一条存储行 `(key → {ver, seq, val})` 是折叠捷径,绝不是权威:可能陈旧(`seq` 精确说明陈旧到哪),但绝不会错。实现据此承诺:
- **每次后台写入都 fail-soft。** 持久写失败只记一条警告并保持缓存陈旧;下一次写入或冷读自愈。两次写之间崩溃的代价是更长的尾部回放,绝不是错误的值。
- **`ver` 与活单元 `stateVersion` 不匹配即丢弃,绝不迁移。** 单元递增版本会在读取时使其行失效;该 key 从日志重新折叠。
- **整记录写入。** 每次写入替换该会话的完整检查点(注册表切面始终是完整的),并经无损 JSON 边界快照——违反纯 JSON 契约的单元状态会大声失败。
- **记录绑定到日志生命周期,而不只是 id。** 每条记录存储其折叠来源的 header 身份(`createdAt``cwd`);每次读取先以活 header 或存储 header 为证验证它,再接受任何行——被删后重建的 id、或缓存幸存而持久化存储被换掉时无关记录被整体丢弃绝不播种幻影值。
- **日志领先,缓存跟随。** 活会话检查点先把缓冲事件持久 flush缓存行才落地因此崩溃只会让缓存落后于日志更长的尾部回放绝不领先于它。
## 写策略
两个必写点,其间节流:
| 触发 | 性质 |
|---|---|
| `turn/end` | 必写——冷读要的正是轮次终值。 |
| 会话销毁detach | 必写——live 转 cold 的时刻;此后冷读阶梯接管该会话。 |
| 累计 `writeEveryEvents` 个已提交事件 | 配置节流(条数)。 |
| 距首个脏事件 `writeIntervalMs` 毫秒 | 配置节流(间隔)。 |
两个 `Config` 字段均必填(无默认值):写入节奏是部署选择,没有普适正确值,由 cordis.yml 明示。
## 列表读(`cachedSnapshot(meta)`
零 I/O 一档:从身份匹配的存储记录直接 view 全量值(仅版本匹配的 key`{asOfSeq, values}` 切面返回——`asOfSeq` 取所服务行的最低水位,客户端在 higher-seq-wins 规则下播种值存储时,陈旧列表块永远压不过更新的推送帧。无可用记录(未知 id、无关生命周期、无版本匹配行时返回 `undefined`api-proxy 列表载体将其转为列缺席。
## 冷读(`coldSnapshot(id, signal?)`
读取阶梯,正常路径无需加载全量日志:缓存行 → `sessionProjections.restoreFloor`(锚定在最低可用水位之前一个事件的位置)→ 持久化 `readFrom(id, floor)``sessionProjections.restore` → 刷新行的 fail-soft 写回。这个锚使缩短的日志(崩溃修复截断)可被证明:越界的行恰好触发一次从 seq 0 的全量重读,而不是把幽灵值当现值服务。无已注册单元时直接服务 `{asOfSeq: -1, values: {}}`,不触碰持久化;无持久日志的会话以 seam 的 `not found` 拒绝。
`write(session)` 是两个必写点共用的同步切面检查点;载体可以直接调用(非 fail-soft——由 fail-soft 包装层负责遏制)。
## 组合
```yaml
- id: session-projection-cache
name: '@deepseek-ai/dsh-session-projection-cache'
config:
writeEveryEvents: 200
writeIntervalMs: 5000
```
注入 `storageDomain``sessionProjections``sessionPersistence``sessions`。没有这一行时,投影系统只跑 live水位缓存冷读在实现了它的载体处退回全量日志加载
## 模型体验
无,因为缓存只持久化并恢复 host 侧的、由已写入日志的会话状态派生的读模型不触碰任何提示词、消息、schema、流或工具结果。
#### KV Cache 影响
无;缓存从不组装或发送提供方请求。
## 已知局限与延后工作
- **没有淘汰或保留面**——记录按会话累积;清理存储的检查点是带外维护,与会话持久化本身同一立场。
- **间隔节流按会话粗粒度**——计时器在一次干净写入后的首个脏事件时武装;持续的低于阈值的涓流每个间隔写一次,不是滑动窗口。
- **`coldSnapshot` 读取不去重**——同一会话的两个并发冷读各跑一遍阶梯;写回最后者胜(行等价),对列表级调用频率可接受。

View File

@@ -0,0 +1,48 @@
{
"name": "@deepseek-ai/dsh-session-projection-cache",
"description": "Persisted projection cache (ctx.sessionProjectionCache): durable per-session projection checkpoints over the domain data form, throttled write-behind, and the cold-read ladder (cache row + persistence tail replay)",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./invariant": {
"types": "./lib/types/invariant.d.ts",
"default": "./lib/invariant.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/invariant.js",
"lib/types/**/*.d.ts"
],
"license": "BSD-3-Clause",
"dependencies": {
"schemastery": "^3.18.0",
"zod": "^4.4.3"
},
"peerDependencies": {
"@deepseek-ai/dsh-invariants": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
"@deepseek-ai/dsh-session-projection": "^0.0.1",
"@deepseek-ai/dsh-storage-domain": "^0.0.1",
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-session-persistence": "workspace:^",
"@deepseek-ai/dsh-session-projection": "workspace:^",
"@deepseek-ai/dsh-storage": "workspace:^",
"@deepseek-ai/dsh-storage-domain": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,300 @@
/**
* Persisted projection cache (`ctx.sessionProjectionCache`): durable
* checkpoints of every registered projection unit's state, one record per
* session on the domain data form (`session_projcache` domain — the shipped
* json backend lands it beside `workspace.json`). The cache is a fold
* shortcut, never an authority: a row is possibly stale (its `seq`
* says how stale) but never wrong, so every write path is fail-soft (a lost
* write costs a longer tail replay on the next cold read) and a
* `ver` mismatch discards the row instead of migrating it. Design
* authority: the session-projection RFC
* (.agents/notes/proposed/architecture/2026-07-27-session-projection-and-command-log.md).
* @module @deepseek-ai/dsh-session-projection-cache
*/
import { Context, Service } from 'cordis'
import z from 'schemastery'
import { snapshotJsonValue } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
// Empty type import: applies the package's cordis Context merge
// (`ctx.sessionPersistence`), which this service reads on the cold path.
import type {} from '@deepseek-ai/dsh-session-persistence'
import type { ProjectionCheckpoint, ProjectionSnapshot } from '@deepseek-ai/dsh-session-projection'
import type { KvTable } from '@deepseek-ai/dsh-storage-domain'
import { projectionCacheDomainSpec } from './spec.ts'
import type { CheckpointIdentity, CheckpointRecord } from './spec.ts'
export { checkpointIdentity, checkpointRecord, checkpointRow, projectionCacheDomainSpec } from './spec.ts'
export type { CheckpointIdentity, CheckpointRecord } from './spec.ts'
declare module 'cordis' {
interface Context {
sessionProjectionCache: SessionProjectionCache
}
}
/**
* Plugin config. Both throttle triggers are deployment choices with no
* universally correct value, so the composition states them explicitly
* (cordis.yml); the two mandatory write points (`turn/end` and session
* disposal) are policy, not tunables, and always fire.
*/
export interface Config {
/** Committed events per session that force a durable checkpoint write between mandatory points. */
writeEveryEvents: number
/** Longest time (milliseconds) a dirty checkpoint may stay unwritten between mandatory points. */
writeIntervalMs: number
}
export const Config: z<Config> = z.object({
writeEveryEvents: z.natural().min(1).required(),
writeIntervalMs: z.natural().min(1).required(),
})
/** Per-session write-behind bookkeeping (live sessions only; dropped at retire). */
interface DirtyState {
/** Committed events since the last durable write. */
pending: number
/** Interval trigger armed at the first dirty event after a clean write. */
timer: ReturnType<typeof setTimeout> | undefined
}
/**
* The persisted projection cache service. Opens the `session_projcache`
* domain at init, checkpoints live sessions on a throttled write-behind
* (count/interval triggers from {@link Config}) plus two mandatory points —
* `turn/end` and session disposal (the live-to-cold moment) — and serves the
* cold-read ladder: cached row, persistence `readFrom` tail, registry
* `restore`, durable write-back. Every durable write is fail-soft: failures
* log a warning and the cache self-heals on the next write or cold read.
*/
export class SessionProjectionCache extends Service {
static inject = ['storageDomain', 'sessionProjections', 'sessionPersistence', 'sessions']
static Config: z<Config> = Config
private table?: KvTable<SessionId, CheckpointRecord>
private readonly dirty = new Map<Session, DirtyState>()
constructor(ctx: Context, public config: Config) {
super(ctx, 'sessionProjectionCache')
}
/** Open the domain and install the write-behind listeners. */
protected async [Service.init](): Promise<void> {
const domain = await this.ctx.storageDomain.open(projectionCacheDomainSpec)
this.ctx.effect(() => () => domain.close(), 'sessionProjectionCache.domainClose')
this.table = domain.table('sessions')
this.installWritePath()
}
/**
* The stored record for one session, accepted only when its bound log
* identity matches `expected`. A session id names a slot, not a lifecycle:
* a recreated id or a persistence store swapped under a surviving cache
* must not let an old record seed state folded from an unrelated log.
* Synchronous from the domain's in-memory state.
* @param id - the session whose record is read.
* @param expected - the log identity the caller holds (live or stored header).
* @returns the identity-matching record, or `undefined` (absent or unrelated).
*/
private recordFor(id: SessionId, expected: CheckpointIdentity): CheckpointRecord | undefined {
const record = this.requireTable().get(id)
if (record === undefined) return undefined
return identityMatches(record.identity, expected) ? record : undefined
}
/**
* The zero-I/O listing read: whole values viewed straight from the stored
* rows (version-matching keys only), each cut carried with its watermark
* so a client value store can seed under its higher-seq-wins rule — as
* stale as the last durable checkpoint but never wrong, and never from an
* unrelated log (the caller's header is the identity witness). Fresher
* paths (the history tail baseline, {@link coldSnapshot}) supersede these
* values whenever a session is actually opened.
* @param meta - the listed session's header (identity witness; no log read).
* @returns the cut (`asOfSeq` = lowest served-row watermark), or
* `undefined` when no usable row exists for this lifecycle.
*/
cachedSnapshot(meta: SessionHeader): ProjectionSnapshot | undefined {
const record = this.recordFor(meta.id, identityOf(meta))
if (record === undefined) return undefined
const values = this.ctx.sessionProjections.viewCheckpoint(record.rows)
const keys = Object.keys(values)
if (keys.length === 0) return undefined
// The block carries ONE cut: the lowest served watermark is the seq every
// value is at least current as of (under-claiming is safe under
// higher-seq-wins; over-claiming would let a stale value outrank pushes).
const asOfSeq = Math.min(...keys.map(key => (record.rows[key] as { seq: number }).seq))
return { asOfSeq, values }
}
/**
* Durably checkpoint one live session NOW (both mandatory points call
* this; tests and carriers may too). The registry cut is snapshotted at
* this boundary (states are live references), then the whole record is
* replaced. NOT fail-soft — callers on the fail-soft paths contain it.
* @param session - the live session to checkpoint.
* @returns resolution after durability and event emission.
*/
async write(session: Session): Promise<void> {
const rows = this.ctx.sessionProjections.checkpoint(session)
this.markClean(session)
// Durability barrier: the checkpoint cut was taken above, so flushing
// AFTER it guarantees every event inside the cut is durably logged
// before the cache row lands — a crash can leave the cache behind the
// log (longer tail replay) but never ahead of it (phantom values folded
// from events no stored log contains). At detach the store entry is
// already gone; persistence's own retirement drain covers that path and
// any residual overreach is caught by the cold read's anchored floor.
if (this.ctx.sessions.get(session.id) === session) await this.ctx.sessions.flush(session)
await this.put(session.id, identityOf(session.header), rows)
}
/**
* Cold-read one persisted session's projections with zero full-log load:
* cached rows + a persistence `readFrom` tail from the registry's restore
* floor, refolded by the registry and written back (fail-soft) so the next
* cold read starts closer. A cache row invalidated by a shrunk log
* (crash-repair truncation) triggers one full re-read from seq 0 — the
* ladder's slow rung, still no crash. Rejects when the session has no
* persisted log (`not found` from the persistence seam).
* @param id - the persisted session to read.
* @param signal - optional cancellation for the persistence reads.
* @returns the snapshot cut at the stored log end.
*/
async coldSnapshot(id: SessionId, signal?: AbortSignal): Promise<ProjectionSnapshot> {
const record = this.requireTable().get(id)
const cached = record?.rows ?? {}
const floor = this.ctx.sessionProjections.restoreFloor(cached)
const persistence = this.ctx.sessionPersistence
if (floor === undefined) {
// No unit registered: nothing to fold, but the not-found contract must
// hold in this topology too — the probe read rejects for an absent log
// and dates the empty cut for a present one.
const probe = await persistence.readFrom(id, 0, signal)
return { asOfSeq: probe.events.at(-1)?.seq ?? -1, values: {} }
}
let restored: { snapshot: ProjectionSnapshot; checkpoint: ProjectionCheckpoint }
const tail = await persistence.readFrom(id, floor, signal)
// The tail's stored header is the identity witness: a record bound to a
// different lifecycle (recreated id, swapped store) is discarded whole
// before any of its rows can seed a fold.
const related = record === undefined || identityMatches(record.identity, identityOf(tail.meta))
try {
if (!related) throw new Error('unrelated log identity')
restored = this.ctx.sessionProjections.restore(cached, tail.events, floor)
} catch {
// The recoverable restore failures: an unrelated record, or a row
// overreaching the stored log end (or predating the floor). Both imply
// floor > 0 (baseSeq-0 restores never throw and an unrelated record
// still carried a usable watermark), so the full log is a fresh read.
const whole = await persistence.readFrom(id, 0, signal)
restored = this.ctx.sessionProjections.restore({}, whole.events, 0)
}
await this.putSoft(id, identityOf(tail.meta), restored.checkpoint, 'cold-read write-back')
return restored.snapshot
}
// --- write-behind (throttle + mandatory points) ---
private installWritePath(): void {
// Every committed event advances the dirty counter; turn/end is a
// mandatory point (the durable value most reads want is the turn-final
// one), count/interval throttle the in-turn stream.
this.ctx.on('session/event', (session: Session, event: SessionEvent) => {
if (event.type === 'turn/end') {
void this.flushSoft(session, 'turn/end')
return
}
const state = this.dirty.get(session) ?? { pending: 0, timer: undefined }
this.dirty.set(session, state)
state.pending += 1
if (state.pending >= this.config.writeEveryEvents) {
void this.flushSoft(session, 'count threshold')
return
}
state.timer ??= setTimeout(() => {
void this.flushSoft(session, 'interval')
}, this.config.writeIntervalMs)
})
// Detach (the live-to-cold moment): the second mandatory point. After
// this write the cold-read ladder serves the session from the cache.
// flushSoft's synchronous prefix reads and resets the dirty state, so
// dropping it (timer already cleared by markClean) right after is safe.
this.ctx.on('session/disposed', (session: Session) => {
void this.flushSoft(session, 'detach')
this.markClean(session)
this.dirty.delete(session)
})
// Clear pending timers with the plugin (their sessions outlive the cache).
this.ctx.effect(() => () => {
for (const state of this.dirty.values()) {
if (state.timer !== undefined) clearTimeout(state.timer)
}
this.dirty.clear()
}, 'sessionProjectionCache.timers')
}
/**
* One fail-soft durable checkpoint. Every caller has work by construction:
* the throttle triggers only fire dirty (markClean clears the timer with
* the counter) and the two mandatory points write unconditionally.
*/
private async flushSoft(session: Session, trigger: string): Promise<void> {
try {
await this.write(session)
} catch (error) {
this.ctx.logger.warn(`session projection cache: ${trigger} write for "${session.id}" failed (cache stays stale): ${String(error)}`)
}
}
/** Reset one session's dirty bookkeeping (its checkpoint is being written). */
private markClean(session: Session): void {
const state = this.dirty.get(session)
if (state === undefined) return
state.pending = 0
if (state.timer !== undefined) {
clearTimeout(state.timer)
state.timer = undefined
}
}
/** Replace one session's stored record with its log identity and a detached snapshot of `rows`. */
private async put(id: SessionId, identity: CheckpointIdentity, rows: ProjectionCheckpoint): Promise<void> {
const detached = snapshotJsonValue(rows)
if (detached === undefined) {
throw new TypeError('projection checkpoint is not losslessly JSON-serializable (a unit state violates the plain-JSON contract)')
}
await this.requireTable().put(id, { identity, rows: detached as CheckpointRecord['rows'] })
}
/** Fail-soft {@link put}: cache writes must never fail their caller's read or event path. */
private async putSoft(id: SessionId, identity: CheckpointIdentity, rows: ProjectionCheckpoint, what: string): Promise<void> {
try {
await this.put(id, identity, rows)
} catch (error) {
this.ctx.logger.warn(`session projection cache: ${what} for "${id}" failed (cache stays stale): ${String(error)}`)
}
}
private requireTable(): KvTable<SessionId, CheckpointRecord> {
/* v8 ignore next -- Service.init assigns the table before the service becomes injectable */
if (this.table === undefined) throw new Error('session projection cache is not initialized')
return this.table
}
}
/** Project a header onto the identity fields a record is bound to. */
function identityOf(header: SessionHeader): CheckpointIdentity {
return { createdAt: header.createdAt, ...header.cwd === undefined ? {} : { cwd: header.cwd } }
}
/** Whether a stored record's bound identity names the caller's lifecycle. */
function identityMatches(stored: CheckpointIdentity, expected: CheckpointIdentity): boolean {
return stored.createdAt === expected.createdAt && stored.cwd === expected.cwd
}
export default SessionProjectionCache

View File

@@ -0,0 +1,35 @@
/**
* Package-owned invariant companion for `@deepseek-ai/dsh-session-projection-cache`.
* @module @deepseek-ai/dsh-session-projection-cache/invariant
*/
/* jscpd:ignore-start */
import type { Context } from 'cordis'
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
const PACKAGE_NAME = '@deepseek-ai/dsh-session-projection-cache'
/** Cordis companion plugin name. */
export const name = 'session-projection-cache-invariant'
/** Service required before the companion can reserve package ownership. */
export const inject = ['invariants']
/**
* No runtime invariant: the cache's correctness relation (a stored row equals
* the registry fold at its `seq` watermark) is only checkable by re-running the
* fold over the persisted log — duplicating the implementation rather than
* detecting drift — and its staleness is by design (fail-soft writes). The
* durable boundary is already schema-validated by the storage-domain layer
* on every reopen, and the read ladder's version/watermark guards are proven
* by the package spec.
*/
const install: InvariantInstaller = () => {}
/**
* Register this package's invariant companion.
* @param ctx - Cordis context carrying the invariant service.
* @returns the installed registration's disposer after setup succeeds.
*/
export const apply = (ctx: Context): Promise<() => void> =>
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
/* jscpd:ignore-end */

View File

@@ -0,0 +1,71 @@
/**
* The session-projcache domain declaration: one `sessions` table keyed by
* {@link SessionId}, each record the full projection checkpoint for one
* session (`key → {ver, seq, val}` rows). The spec object
* is the single source of the domain's identity, version, and record schema;
* the storage-domain routing decides the medium (the shipped composition's
* json backend lands it at `<root>/session_projcache.json`, beside
* `workspace.json`).
* @module @deepseek-ai/dsh-session-projection-cache/src/spec
*/
import { z } from 'zod'
import { SessionId } from '@deepseek-ai/dsh-session'
import { defineDomain, domainTable } from '@deepseek-ai/dsh-storage-domain'
/**
* One persisted checkpoint row (the RFC's `(sessionId, key, ver, seq, val)`
* minus the two record keys). `val` is the unit's internal state — plain
* JSON by the unit contract; `z.json()` enforces that at the durable
* boundary. A row is never wrong, only possibly stale: `seq` says exactly
* how stale, and a `ver` mismatch against the live unit's `stateVersion`
* discards it at read time (never a migration).
*/
export const checkpointRow = z.object({
ver: z.number().int().nonnegative(),
seq: z.number().int().gte(-1),
val: z.json(),
})
/**
* The stored-log identity a record is bound to: the immutable header fields
* that distinguish one session lifecycle from another under the same id. A
* session id names a slot, not a lifecycle — a deleted-then-recreated id, or
* a persistence root swapped under a surviving cache, would otherwise let an
* old row pass every watermark check and seed state folded from an unrelated
* log. Reads validate this against the live header (listing) or the stored
* header (cold read) before accepting any row.
*/
export const checkpointIdentity = z.object({
createdAt: z.number().int().nonnegative(),
cwd: z.string().optional(),
})
/** The identity fields a record is bound to, inferred from {@link checkpointIdentity}. */
export type CheckpointIdentity = z.infer<typeof checkpointIdentity>
/**
* One session's stored record: the log identity it was folded from plus its
* checkpoint rows keyed by projection key. The whole record is replaced on
* every write (whole-value discipline — the registry checkpoint is always
* the complete per-session cut).
*/
export const checkpointRecord = z.object({
identity: checkpointIdentity,
rows: z.record(z.string(), checkpointRow),
})
/** One stored per-session checkpoint record, inferred from {@link checkpointRecord}. */
export type CheckpointRecord = z.infer<typeof checkpointRecord>
/**
* The session-projcache domain spec. Version bumps discard the whole medium
* (cache semantics: a stale or unreadable cache costs a longer tail replay,
* never a wrong value). v2 added the record's log-identity binding; v3
* renamed the row fields to `ver`/`seq`/`val`.
*/
export const projectionCacheDomainSpec = defineDomain({
name: 'session_projcache',
version: 3,
tables: { sessions: domainTable<SessionId, CheckpointRecord>(checkpointRecord) },
})

View File

@@ -0,0 +1,387 @@
/**
* SessionProjectionCache behavior: mandatory-point writes (turn/end, detach),
* count/interval throttling between them, fail-soft durability (a failed
* write logs and stays stale, never throws into the event path), and the
* cold-read ladder (cached row + readFrom tail + registry restore +
* write-back; version bump and shrunk-log rows degrade to a full re-read).
*/
import { afterEach, describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import { z } from 'zod'
import Storage from '@deepseek-ai/dsh-storage'
import { DomainFacility } from '@deepseek-ai/dsh-storage-domain'
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
import type { Session, SessionEvent } from '@deepseek-ai/dsh-session'
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection'
import { MemoryMediaPool, MemoryStorageBackend } from '../../../storage/storage-domain/tests/helpers/memory-backend.ts'
import SessionProjectionCache from '../src/index.ts'
declare module '@deepseek-ai/dsh-session-projection/types' {
interface SessionProjectionMap {
'cache-test/marks': { marks: string[] }
}
}
declare module '@deepseek-ai/dsh-session' {
interface SessionEventMap {
'cache-test/mark': { marks: string[] }
}
interface OutOfBandSessionEventMap {
'cache-test/mark': true
}
}
type MarksState = { marks: string[] } | null
const marksUnit = (stateVersion = 1): ProjectionDefinition<'cache-test/marks', MarksState> => ({
key: 'cache-test/marks',
schema: z.object({ marks: z.array(z.string()) }),
init: () => null,
apply: (state, event) => (event.type === 'cache-test/mark' ? (event).data : state),
view: state => state ?? { marks: [] },
stateVersion,
})
/** A persistence double serving readFrom over a fixed per-id stored log (headers stamp createdAt 0). */
function fakePersistence(logs: Map<string, SessionEvent[]>) {
const readFrom = vi.fn(async (id: SessionId, fromSeq: number) => {
const events = logs.get(String(id))
if (events === undefined) throw new Error(`session "${id}" not found`)
return {
meta: { version: 0, id, createdAt: 0 },
events: events.filter(event => event.seq >= fromSeq),
}
})
return { readFrom }
}
/** Header shape for cachedSnapshot calls (fake logs stamp createdAt 0, no cwd). */
const headerOf = (id: SessionId, createdAt = 0, cwd?: string) =>
({ version: 0, id, createdAt, ...cwd === undefined ? {} : { cwd } })
interface HarnessOptions {
pool?: MemoryMediaPool
config?: { writeEveryEvents: number; writeIntervalMs: number }
stateVersion?: number
logs?: Map<string, SessionEvent[]>
}
const contexts: Context[] = []
async function harness(options: HarnessOptions = {}) {
const pool = options.pool ?? new MemoryMediaPool()
const logs = options.logs ?? new Map<string, SessionEvent[]>()
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(Storage)
ctx.storage.backend.register('memory', new MemoryStorageBackend(pool))
const facility = new DomainFacility(ctx, { backend: 'memory', routes: {} })
ctx.storage.mount('domain', facility)
ctx.provide('storageDomain', facility)
await ctx.plugin(SessionStore)
await ctx.plugin(SessionProjectionRegistry)
ctx.sessionProjections.register(marksUnit(options.stateVersion))
const persistence = fakePersistence(logs)
ctx.provide('sessionPersistence', persistence as never)
const fiber = await ctx.plugin(SessionProjectionCache, options.config ?? { writeEveryEvents: 100, writeIntervalMs: 60_000 })
return { ctx, pool, logs, fiber, persistence, cache: ctx.sessionProjectionCache }
}
const mark = (session: Session, marks: string[]): SessionEvent =>
session.append('cache-test/mark', { marks })
const endTurn = (session: Session): SessionEvent =>
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
/** The stored medium record for one session id (undefined = never written). */
function storedRecord(pool: MemoryMediaPool, id: Session['id']) {
return pool.media.get('session_projcache')?.tables.get('sessions')?.get(String(id)) as
{
identity: { createdAt: number; cwd?: string }
rows: Record<string, { ver: number; seq: number; val: unknown }>
} | undefined
}
/** The stored medium rows for one session id (undefined = never written). */
function storedRows(pool: MemoryMediaPool, id: Session['id']) {
return storedRecord(pool, id)?.rows
}
/** Wait until queued fail-soft writes (event-listener fire-and-forget) drain. */
const settle = () => new Promise(resolve => setTimeout(resolve, 0))
afterEach(async () => {
vi.useRealTimers()
await Promise.all(contexts.splice(0).map(ctx => ctx.fiber.dispose()))
})
describe('SessionProjectionCache write policy', () => {
it('writes a durable checkpoint at turn/end (mandatory point)', async () => {
const { ctx, pool } = await harness()
const session = ctx.sessions.create(SessionId('turn-end'))
mark(session, ['a'])
expect(storedRows(pool, session.id)).toBeUndefined() // throttled: no write yet
const end = endTurn(session)
await settle()
const rows = storedRows(pool, session.id)
expect(rows?.['cache-test/marks']).toEqual({ ver: 1, seq: end.seq, val: { marks: ['a'] } })
})
it('writes at session disposal (detach, the live-to-cold moment)', async () => {
const { ctx, pool } = await harness()
// Sessions dispose with their owning fiber: create in a child plugin.
let session: Session | undefined
const owner = await ctx.plugin(Object.assign((inner: Context) => {
session = inner.sessions.create(SessionId('detach'))
}, { inject: ['sessions'] }))
if (session === undefined) throw new Error('session was not created')
mark(session, ['live'])
await owner.dispose()
await settle()
expect(storedRows(pool, session.id)?.['cache-test/marks']?.val).toEqual({ marks: ['live'] })
})
it('flushes when the in-turn event count reaches the configured threshold', async () => {
const { ctx, pool } = await harness({ config: { writeEveryEvents: 3, writeIntervalMs: 60_000 } })
const session = ctx.sessions.create(SessionId('count'))
mark(session, ['1'])
mark(session, ['2'])
await settle()
expect(storedRows(pool, session.id)).toBeUndefined()
mark(session, ['3'])
await settle()
expect(storedRows(pool, session.id)?.['cache-test/marks']?.val).toEqual({ marks: ['3'] })
})
it('flushes on the configured interval when the count threshold is not reached', async () => {
vi.useFakeTimers()
const { ctx, pool } = await harness({ config: { writeEveryEvents: 100, writeIntervalMs: 250 } })
const session = ctx.sessions.create(SessionId('interval'))
mark(session, ['slow'])
await vi.advanceTimersByTimeAsync(249)
expect(storedRows(pool, session.id)).toBeUndefined()
await vi.advanceTimersByTimeAsync(1)
await vi.advanceTimersByTimeAsync(0)
expect(storedRows(pool, session.id)?.['cache-test/marks']?.val).toEqual({ marks: ['slow'] })
})
it('write() on a never-dirty session checkpoints directly and rejects a non-JSON unit state', async () => {
const { ctx, pool } = await harness()
// Never dirtied: no events — write() still lands the init-derived cut.
const clean = ctx.sessions.create(SessionId('clean-write'))
await ctx.sessionProjectionCache.write(clean)
expect(storedRows(pool, clean.id)?.['cache-test/marks']).toEqual({ ver: 1, seq: -1, val: null })
// A unit whose state violates the plain-JSON contract fails the write loud.
ctx.sessionProjections.register({
key: 'cache-test/marks2' as never,
schema: { parse: (value: unknown) => value } as never,
init: () => new Map<string, string>(),
apply: (state: unknown) => state,
view: () => null as never,
stateVersion: 1,
})
await expect(ctx.sessionProjectionCache.write(clean)).rejects.toThrow('not losslessly JSON-serializable')
})
it('plugin disposal clears armed interval timers and leaves cleaned sessions alone', async () => {
vi.useFakeTimers()
const { ctx, pool, fiber } = await harness({ config: { writeEveryEvents: 100, writeIntervalMs: 5000 } })
const armed = ctx.sessions.create(SessionId('armed'))
const cleaned = ctx.sessions.create(SessionId('cleaned'))
mark(armed, ['pending']) // timer armed, no write yet
mark(cleaned, ['done'])
endTurn(cleaned) // mandatory write; markClean leaves {pending: 0, timer: undefined} in the map
await vi.advanceTimersByTimeAsync(0)
await fiber.dispose()
// The armed timer died with the plugin: advancing time writes nothing.
await vi.advanceTimersByTimeAsync(10_000)
expect(storedRows(pool, armed.id)).toBeUndefined()
})
it('contains a durable write failure: logs a warning, event path unharmed, next write self-heals', async () => {
const { ctx, pool } = await harness()
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
const session = ctx.sessions.create(SessionId('fail-soft'))
mark(session, ['x'])
pool.failNextWrites = 1
endTurn(session)
await settle()
expect(storedRows(pool, session.id)).toBeUndefined()
expect(warn).toHaveBeenCalledWith(expect.stringContaining('turn/end write for "fail-soft" failed'))
// Self-heal: the next mandatory point writes the current cut.
mark(session, ['y'])
endTurn(session)
await settle()
expect(storedRows(pool, session.id)?.['cache-test/marks']?.val).toEqual({ marks: ['y'] })
})
})
describe('SessionProjectionCache cold read', () => {
const storedLog = (marks: string[][]): SessionEvent[] => {
const events: SessionEvent[] = [
{ type: 'turn/start', seq: 0, time: 0, data: { turn: 1 } },
]
for (const m of marks) {
events.push({ type: 'cache-test/mark', seq: events.length, time: events.length, data: { marks: m } })
}
events.push({ type: 'turn/end', seq: events.length, time: events.length, data: { turn: 1, reason: { kind: 'completed' } } })
return events
}
/** Pre-seed the medium with one stored checkpoint record (before the domain opens). */
function seedRow(
pool: MemoryMediaPool,
id: string,
row: { ver: number; seq: number; val: unknown },
identity: { createdAt: number; cwd?: string } = { createdAt: 0 },
): void {
pool.versions.set('session_projcache', 3)
pool.media.set('session_projcache', {
tables: new Map([['sessions', new Map([[id, { identity, rows: { 'cache-test/marks': row } }]])]]),
global: null,
})
}
it('serves a cold session from the cache row plus a bounded tail read, and writes the refresh back', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['cold', storedLog([['a'], ['a', 'b']])]])
// A warm-era checkpoint at watermark 1 (only ['a'] folded).
seedRow(pool, 'cold', { ver: 1, seq: 1, val: { marks: ['a'] } })
const { cache, persistence, pool: samePool } = await harness({ pool, logs })
const id = SessionId('cold')
const snapshot = await cache.coldSnapshot(id)
expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['a', 'b'] })
expect(snapshot.asOfSeq).toBe(3)
// The tail read was bounded by the anchored floor (watermark 1 -> floor 1), not 0.
expect(persistence.readFrom).toHaveBeenCalledWith(id, 1, undefined)
// Write-back: the stored row advanced to the served cut.
expect(storedRows(samePool, id)?.['cache-test/marks'])
.toEqual({ ver: 1, seq: 3, val: { marks: ['a', 'b'] } })
})
it('discards a version-mismatched row and refolds the full log', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['bumped', storedLog([['a']])]])
seedRow(pool, 'bumped', { ver: 1, seq: 2, val: { marks: ['stale'] } })
const { cache, persistence } = await harness({ pool, logs, stateVersion: 2 })
const snapshot = await cache.coldSnapshot(SessionId('bumped'))
expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['a'] })
// Mismatch pulls the floor to 0: one full read, no second pass needed.
expect(persistence.readFrom).toHaveBeenCalledTimes(1)
expect(persistence.readFrom).toHaveBeenCalledWith(SessionId('bumped'), 0, undefined)
})
it('detects a log shrunk below the row watermark and degrades to one full re-read', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['shrunk', storedLog([['a']])]]) // seqs 0..2
seedRow(pool, 'shrunk', { ver: 1, seq: 9, val: { marks: ['ghost'] } })
const { cache, persistence } = await harness({ pool, logs })
const snapshot = await cache.coldSnapshot(SessionId('shrunk'))
expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['a'] })
expect(snapshot.asOfSeq).toBe(2)
// Anchored tail read (floor 9) came back empty -> full re-read from 0.
expect(persistence.readFrom).toHaveBeenNthCalledWith(1, SessionId('shrunk'), 9, undefined)
expect(persistence.readFrom).toHaveBeenNthCalledWith(2, SessionId('shrunk'), 0, undefined)
})
it('write-back failure is contained: the snapshot is still served', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['soft', storedLog([['a']])]])
const { ctx, cache } = await harness({ pool, logs })
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
pool.failNextWrites = 1
const snapshot = await cache.coldSnapshot(SessionId('soft'))
expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['a'] })
expect(warn).toHaveBeenCalledWith(expect.stringContaining('cold-read write-back for "soft" failed'))
})
it('rejects for a session with no persisted log', async () => {
const { cache } = await harness()
await expect(cache.coldSnapshot(SessionId('absent'))).rejects.toThrow('not found')
})
it('discards a record bound to a different log lifecycle and refolds from the actual log', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['reborn', storedLog([['real']])]]) // stored header stamps createdAt 0
// A checkpoint from a PRIOR lifecycle of the same id (different createdAt):
// its rows pass every watermark check, but the identity does not match.
seedRow(pool, 'reborn', { ver: 1, seq: 2, val: { marks: ['phantom'] } }, { createdAt: 999 })
const { cache, pool: samePool } = await harness({ pool, logs })
const snapshot = await cache.coldSnapshot(SessionId('reborn'))
expect(snapshot.values['cache-test/marks']).toEqual({ marks: ['real'] })
// The write-back rebinds the record to the actual log's identity.
expect(storedRecord(samePool, SessionId('reborn'))?.identity).toEqual({ createdAt: 0 })
})
it('cachedSnapshot returns undefined when every stored row is version-mismatched', async () => {
const pool = new MemoryMediaPool()
seedRow(pool, 'all-stale', { ver: 99, seq: 4, val: { marks: ['old'] } })
const { cache } = await harness({ pool })
expect(cache.cachedSnapshot(headerOf(SessionId('all-stale')))).toBeUndefined()
})
it('binds identity on cwd too: a matching cwd serves, a moved session does not', async () => {
const pool = new MemoryMediaPool()
seedRow(pool, 'homed', { ver: 1, seq: 2, val: { marks: ['w'] } }, { createdAt: 0, cwd: '/work' })
const { cache } = await harness({ pool })
const id = SessionId('homed')
expect(cache.cachedSnapshot(headerOf(id, 0, '/work'))?.values['cache-test/marks']).toEqual({ marks: ['w'] })
expect(cache.cachedSnapshot(headerOf(id, 0, '/elsewhere'))).toBeUndefined()
expect(cache.cachedSnapshot(headerOf(id, 0))).toBeUndefined()
})
it('dates an empty stored log at -1 in the zero-units topology', async () => {
const pool = new MemoryMediaPool()
const logs = new Map([['empty', [] as SessionEvent[]]])
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(Storage)
ctx.storage.backend.register('memory', new MemoryStorageBackend(pool))
const facility = new DomainFacility(ctx, { backend: 'memory', routes: {} })
ctx.storage.mount('domain', facility)
ctx.provide('storageDomain', facility)
await ctx.plugin(SessionStore)
await ctx.plugin(SessionProjectionRegistry)
ctx.provide('sessionPersistence', fakePersistence(logs) as never)
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
await expect(ctx.sessionProjectionCache.coldSnapshot(SessionId('empty')))
.resolves.toEqual({ asOfSeq: -1, values: {} })
})
it('cachedSnapshot serves identity-matching rows with the cut watermark and refuses unrelated ones', async () => {
const pool = new MemoryMediaPool()
seedRow(pool, 'listed', { ver: 1, seq: 4, val: { marks: ['t'] } })
const { cache } = await harness({ pool })
const id = SessionId('listed')
// Matching header: values plus the watermark the client seeds under.
expect(cache.cachedSnapshot(headerOf(id))).toEqual({ asOfSeq: 4, values: { 'cache-test/marks': { marks: ['t'] } } })
// A recreated id (different createdAt): the record is unrelated — no block.
expect(cache.cachedSnapshot(headerOf(id, 777))).toBeUndefined()
// Unknown id: no block.
expect(cache.cachedSnapshot(headerOf(SessionId('never-cached')))).toBeUndefined()
})
it('holds the not-found contract with zero registered units, and dates the empty cut for a present log', async () => {
// Same composition minus any registered unit: restoreFloor is undefined,
// yet coldSnapshot must still reject for an absent log (probe read) and
// serve an empty cut at the stored end for a present one.
const pool = new MemoryMediaPool()
const logs = new Map([['bare', storedLog([['a']])]]) // seqs 0..2
const ctx = new Context()
contexts.push(ctx)
await ctx.plugin(Storage)
ctx.storage.backend.register('memory', new MemoryStorageBackend(pool))
const facility = new DomainFacility(ctx, { backend: 'memory', routes: {} })
ctx.storage.mount('domain', facility)
ctx.provide('storageDomain', facility)
await ctx.plugin(SessionStore)
await ctx.plugin(SessionProjectionRegistry)
ctx.provide('sessionPersistence', fakePersistence(logs) as never)
await ctx.plugin(SessionProjectionCache, { writeEveryEvents: 100, writeIntervalMs: 60_000 })
await expect(ctx.sessionProjectionCache.coldSnapshot(SessionId('absent'))).rejects.toThrow('not found')
await expect(ctx.sessionProjectionCache.coldSnapshot(SessionId('bare')))
.resolves.toEqual({ asOfSeq: 2, values: {} })
})
})

View File

@@ -0,0 +1,39 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": [
{
"path": "../../../vendor/cosmokit"
},
{
"path": "../../../vendor/cordis"
},
{
"path": "../../../vendor/schemastery"
},
{
"path": "../../core/session"
},
{
"path": "../session-persistence"
},
{
"path": "../session-projection"
},
{
"path": "../../storage/storage"
},
{
"path": "../../storage/storage-domain"
},
{
"path": "../../support/invariants"
}
]
}