fix(session-query): harden SQLite search reconciliation
This commit is contained in:
Hypatia May
@@ -1,22 +1,22 @@
|
||||
# @deepseek-ai/dsh-session-query-sqlite
|
||||
|
||||
SQLite FTS5 implementation of `ctx.sessionSearch`. The service searches the live-preferred logical session corpus, groups cross-session results by their strongest event, and keeps provider-specific BM25 scores private.
|
||||
SQLite FTS5 implementation of `ctx.sessionSearch`. The service searches the live-preferred logical session corpus and groups cross-session results by their strongest event.
|
||||
|
||||
## Search contract
|
||||
|
||||
`searchSessions(request, exec?)` returns `SessionSearchHit` pages across the corpus; `searchEvents(request, exec?)` returns `SessionEventSearchHit` pages within one session. Queries are required, trimmed, whitespace-normalized literal phrases. FTS5 syntax such as quotes, `OR`, `NEAR`, and `*` is treated as data rather than executable MATCH syntax. Metadata filters are parameterized SQL predicates applied before ranking.
|
||||
|
||||
Ordering is deterministic: relevance first, then event time, session id where applicable, and seq. Cross-session results expose the selected event as `bestMatch`; both scopes return plain-text snippets bounded in Unicode code points. Cursors are opaque, bind to the normalized request and service instance, and fail when the relevant generation changes. A within-session cursor survives unrelated-session changes; a cross-session cursor does not.
|
||||
Relevance is source-comparable across persistent and TEMP tables: actual FTS5 highlighted-match span count descending, then stored document code-point length ascending. Event time, session id where applicable, and seq break remaining ties. Cross-session results expose the selected event as `bestMatch`; both scopes derive whitespace-normalized plain text from FTS5 highlight positions and bound it in Unicode code points. Cursors are opaque branded values, bind to the normalized request and service instance, and fail when the relevant generation changes. A within-session cursor survives unrelated-session changes; a cross-session cursor does not.
|
||||
|
||||
All three surfaces (`current`, `shadowed`, and `log-only`) are searchable by default. Pass a surface filter to narrow them.
|
||||
|
||||
## Source and index lifecycle
|
||||
|
||||
The service requires `ctx.sessions` and observes optional `ctx.sessionPersistence` dynamically. One serialized state machine observes complete sources, extracts shared semantic documents, reconciles changes transactionally, and runs the query. Stable fingerprints preserve unchanged persisted rows and generations; new, changed, and deleted durable sessions reconcile on the next search. Source or transaction failure commits nothing, and the next search retries.
|
||||
The service requires `ctx.sessions` and observes optional `ctx.sessionPersistence` dynamically. One serialized state machine compares lightweight durable snapshot revisions, loads only new or changed logs, extracts shared semantic documents, reconciles changes transactionally, and runs the query. Repeated queries and unchanged reopen load no full durable logs; new, changed, deleted, or load-repaired sources reconcile on the next stable observation. Source or transaction failure commits nothing, and the next search retries.
|
||||
|
||||
Persisted FTS rows live in a dedicated derived database. Connection-local TEMP tables hold live rows, which shadow the durable base for the same session and reveal it when the live owner disappears. Unmounting persistence hides durable rows without discarding the cache; remounting reconciles it. Closing or reopening the database drops every live overlay while retaining persisted rows.
|
||||
|
||||
The database is disposable but reset is guarded: a recognized incompatible search schema rebuilds in place, while an unrelated or canonical database is refused. Never point `path` at the session-persistence database.
|
||||
The database is disposable but reset is guarded: a recognized incompatible search schema rebuilds in place, while an unrelated or canonical database is refused before mutating journal mode. Never point `path` at the session-persistence database. Exactly one service in one process owns a derived-index path; external writers or a second process are unsupported because generations and TEMP shadow state are connection-owned.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -30,6 +30,6 @@ The database is disposable but reset is guarded: a recognized incompatible searc
|
||||
|
||||
## Tokenizer and limits
|
||||
|
||||
The index uses FTS5 `unicode61`. In the implementation experiment it supported the two-character query `AI` and produced an index about 2.1× smaller than the trigram alternative. The trade-off is token/phrase recall rather than arbitrary substring recall: `AI` does not match the token `BRAID`. Use `ctx.sessionQuery.filterEvents()` with a `text` clause when a literal whitespace-flexible substring scan is required.
|
||||
The index uses FTS5 `unicode61`. In the implementation experiment it supported the two-character query `AI` and produced an index about 2.1× smaller than the trigram alternative. The trade-off is token/phrase recall rather than arbitrary substring recall: `AI` does not match the token `BRAID`. Use `ctx.sessionQuery.filterEvents()` with a `text` clause when a literal whitespace-flexible substring scan is required. NUL is rejected in queries; reserved highlight markers and NUL in documents are normalized before indexing so presentation markers cannot collide with source text.
|
||||
|
||||
Abort signals stop queued work and caller waits around asynchronous source observation. Node's synchronous `DatabaseSync` API cannot interrupt a MATCH statement already executing on the JavaScript thread; the signal is checked immediately before and after the serialized observation/reconciliation boundary.
|
||||
|
||||
@@ -6,12 +6,17 @@
|
||||
|
||||
import { createHash, randomUUID } from 'node:crypto'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { Context } from 'cordis'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type SessionPersistence from '@deepseek-ai/dsh-session-persistence'
|
||||
import type {
|
||||
SessionPersistenceRevision,
|
||||
SessionPersistenceSnapshot,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import {
|
||||
SessionQueryError,
|
||||
SessionSearchCursor,
|
||||
SessionSearchService,
|
||||
assertSessionHeadersCompatible,
|
||||
buildSessionEventSearchDocuments,
|
||||
@@ -22,6 +27,7 @@ import type {
|
||||
SessionEventSearchRequest,
|
||||
SessionSearchExecContext,
|
||||
SessionSearchHit,
|
||||
SessionSearchCursor as SessionSearchCursorValue,
|
||||
SessionSearchPage,
|
||||
SessionSearchRequest,
|
||||
} from '@deepseek-ai/dsh-session-query'
|
||||
@@ -32,6 +38,8 @@ import {
|
||||
import {
|
||||
type NormalizedEventRequest,
|
||||
type NormalizedSessionRequest,
|
||||
FTS_HIGHLIGHT_END,
|
||||
FTS_HIGHLIGHT_START,
|
||||
buildEventWhere,
|
||||
buildSessionWhere,
|
||||
makeSnippet,
|
||||
@@ -39,6 +47,7 @@ import {
|
||||
normalizeSessionRequest,
|
||||
quoteFtsData,
|
||||
requestFingerprint,
|
||||
sanitizeFtsText,
|
||||
} from './query.ts'
|
||||
|
||||
export {
|
||||
@@ -78,19 +87,30 @@ interface ResolvedConfig {
|
||||
|
||||
interface ObservedSession {
|
||||
header: SessionHeader
|
||||
events: SessionEvent[]
|
||||
documents: SessionEventSearchDocument[]
|
||||
fingerprint: string
|
||||
}
|
||||
|
||||
interface ObservedPersistedSession {
|
||||
header: SessionHeader
|
||||
revision: SessionPersistenceRevision
|
||||
loaded?: ObservedSession
|
||||
}
|
||||
|
||||
interface Observation {
|
||||
persistence: SessionPersistence | undefined
|
||||
persistenceRevision: number
|
||||
persisted: Map<SessionId, ObservedSession>
|
||||
persisted: Map<SessionId, ObservedPersistedSession>
|
||||
live: Map<SessionId, ObservedSession>
|
||||
}
|
||||
|
||||
interface IndexedRow {
|
||||
interface IndexedPersistedRow {
|
||||
id: string
|
||||
revision: string
|
||||
generation: number
|
||||
}
|
||||
|
||||
interface IndexedLiveRow {
|
||||
id: string
|
||||
fingerprint: string
|
||||
generation: number
|
||||
@@ -109,8 +129,9 @@ interface SearchRow {
|
||||
type: string
|
||||
time: number
|
||||
surface: string
|
||||
text: string
|
||||
score: number
|
||||
marked_text: string
|
||||
match_count: number
|
||||
document_length: number
|
||||
}
|
||||
|
||||
interface CursorPayload {
|
||||
@@ -149,27 +170,32 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
private _localGeneration = 0
|
||||
private _tail: Promise<void> = Promise.resolve()
|
||||
private _closed = false
|
||||
private _closePromise: Promise<void> | undefined
|
||||
private readonly _optionalPersistenceFiber: Fiber
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
this.config = resolveConfig(config)
|
||||
this._ready = this._open()
|
||||
ctx.effect(() => {
|
||||
const fiber = ctx.inject(['sessionPersistence'], (childCtx: Context) => {
|
||||
const service = childCtx.sessionPersistence
|
||||
const binding = {}
|
||||
this._persistenceBinding = binding
|
||||
this._persistence = service
|
||||
// Attach a rejection observer immediately; callers still receive the same
|
||||
// rejection from `_ready`, including when no search is ever attempted.
|
||||
void this._ready.catch(() => undefined)
|
||||
this._optionalPersistenceFiber = ctx.inject(['sessionPersistence'], (childCtx: Context) => {
|
||||
const service = childCtx.sessionPersistence
|
||||
const binding = {}
|
||||
this._persistenceBinding = binding
|
||||
this._persistence = service
|
||||
this._persistenceRevision += 1
|
||||
childCtx.effect(() => () => {
|
||||
/* v8 ignore next -- a stale optional-service disposer cannot clear a replacement */
|
||||
if (this._persistenceBinding !== binding) return
|
||||
this._persistenceBinding = undefined
|
||||
this._persistence = undefined
|
||||
this._persistenceRevision += 1
|
||||
childCtx.effect(() => () => {
|
||||
/* v8 ignore next -- a stale optional-service disposer cannot clear a replacement */
|
||||
if (this._persistenceBinding !== binding) return
|
||||
this._persistenceBinding = undefined
|
||||
this._persistence = undefined
|
||||
this._persistenceRevision += 1
|
||||
}, 'sessionSearchSqlite.persistenceBinding')
|
||||
})
|
||||
return () => void fiber.dispose()
|
||||
}, 'sessionSearchSqlite.persistenceBinding')
|
||||
})
|
||||
ctx.effect(() => {
|
||||
return () => this._optionalPersistenceFiber.dispose()
|
||||
}, 'sessionSearchSqlite.optionalPersistence')
|
||||
ctx.effect(() => async () => this.close(), 'sessionSearchSqlite.close')
|
||||
}
|
||||
@@ -179,17 +205,18 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
exec?: SessionSearchExecContext,
|
||||
): Promise<SessionSearchPage<SessionSearchHit>> {
|
||||
const normalized = normalizeSessionRequest(request, this.config)
|
||||
return this._serialized(exec?.signal, async () => {
|
||||
await this._ensureReady(exec?.signal)
|
||||
await this._reconcile(exec?.signal)
|
||||
assertNotAborted(exec?.signal)
|
||||
const signal = exec?.signal
|
||||
return this._serialized(signal, async () => {
|
||||
await this._ensureReady(signal)
|
||||
await this._reconcile(signal)
|
||||
assertNotAborted(signal)
|
||||
const generation = String(this._globalGeneration)
|
||||
const fingerprint = requestFingerprint(normalized)
|
||||
const offset = normalized.cursor === undefined
|
||||
? 0
|
||||
: decodeCursor(normalized.cursor, this._instance, 'sessions', fingerprint, generation)
|
||||
const rows = this._querySessions(normalized, offset)
|
||||
return page(rows, normalized.limit, row => this._sessionHit(row, normalized.query), cursorOffset => encodeCursor({
|
||||
return page(rows, normalized.limit, row => this._sessionHit(row), cursorOffset => encodeCursor({
|
||||
version: 1,
|
||||
instance: this._instance,
|
||||
scope: 'sessions',
|
||||
@@ -205,17 +232,18 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
exec?: SessionSearchExecContext,
|
||||
): Promise<SessionSearchPage<SessionEventSearchHit>> {
|
||||
const normalized = normalizeEventRequest(request, this.config)
|
||||
return this._serialized(exec?.signal, async () => {
|
||||
await this._ensureReady(exec?.signal)
|
||||
await this._reconcile(exec?.signal)
|
||||
assertNotAborted(exec?.signal)
|
||||
const signal = exec?.signal
|
||||
return this._serialized(signal, async () => {
|
||||
await this._ensureReady(signal)
|
||||
await this._reconcile(signal)
|
||||
assertNotAborted(signal)
|
||||
const generation = this._targetGeneration(normalized.sessionId)
|
||||
const fingerprint = requestFingerprint(normalized)
|
||||
const offset = normalized.cursor === undefined
|
||||
? 0
|
||||
: decodeCursor(normalized.cursor, this._instance, 'events', fingerprint, generation)
|
||||
const rows = this._queryEvents(normalized, offset)
|
||||
return page(rows, normalized.limit, row => this._eventHit(row, normalized.query), cursorOffset => encodeCursor({
|
||||
return page(rows, normalized.limit, row => this._eventHit(row), cursorOffset => encodeCursor({
|
||||
version: 1,
|
||||
instance: this._instance,
|
||||
scope: 'events',
|
||||
@@ -227,8 +255,12 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
}
|
||||
|
||||
/** Close the database after every accepted operation reaches quiescence. */
|
||||
async close(): Promise<void> {
|
||||
if (this._closed) return
|
||||
close(): Promise<void> {
|
||||
this._closePromise ??= this._close()
|
||||
return this._closePromise
|
||||
}
|
||||
|
||||
private async _close(): Promise<void> {
|
||||
this._closed = true
|
||||
await this._tail
|
||||
try {
|
||||
@@ -287,20 +319,20 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
}
|
||||
|
||||
private async _reconcile(signal: AbortSignal | undefined): Promise<void> {
|
||||
const observation = await this._observeStable(signal)
|
||||
assertNotAborted(signal)
|
||||
const db = this._requireDb()
|
||||
const persistedRows = db.prepare(
|
||||
'SELECT id, fingerprint, generation FROM persisted_sessions',
|
||||
).all() as unknown as IndexedRow[]
|
||||
'SELECT id, revision, generation FROM persisted_sessions',
|
||||
).all() as unknown as IndexedPersistedRow[]
|
||||
const liveRows = db.prepare(
|
||||
'SELECT id, fingerprint, generation FROM temp.live_sessions',
|
||||
).all() as unknown as IndexedRow[]
|
||||
).all() as unknown as IndexedLiveRow[]
|
||||
const persistedById = new Map(persistedRows.map(row => [row.id as SessionId, row]))
|
||||
const liveById = new Map(liveRows.map(row => [row.id as SessionId, row]))
|
||||
const observation = await this._observeStable(persistedById, signal)
|
||||
assertNotAborted(signal)
|
||||
const persistentChanges = observation.persistence === undefined
|
||||
? []
|
||||
: [...observation.persisted.values()].filter(entry => persistedById.get(entry.header.id)?.fingerprint !== entry.fingerprint)
|
||||
: [...observation.persisted.values()].filter(entry => entry.loaded !== undefined)
|
||||
const persistentDeletes = observation.persistence === undefined
|
||||
? []
|
||||
: persistedRows.filter(row => !observation.persisted.has(row.id as SessionId))
|
||||
@@ -327,13 +359,17 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
db.exec('BEGIN IMMEDIATE')
|
||||
began = true
|
||||
for (const row of persistentDeletes) this._deleteSession('persisted', row.id as SessionId)
|
||||
for (const entry of persistentChanges) this._replaceSession('persisted', entry, nextMainGeneration)
|
||||
for (const entry of persistentChanges) {
|
||||
/* v8 ignore next -- observation loads every entry whose revision differs */
|
||||
if (entry.loaded === undefined) throw new Error(`missing loaded revision for session "${entry.header.id}"`)
|
||||
this._replacePersistedSession(entry.loaded, entry.revision, nextMainGeneration)
|
||||
}
|
||||
if (persistentChanges.length > 0 || persistentDeletes.length > 0) {
|
||||
db.prepare('UPDATE search_state SET global_generation = ? WHERE singleton = 1').run(nextMainGeneration)
|
||||
}
|
||||
for (const row of liveDeletes) this._deleteSession('live', row.id as SessionId)
|
||||
for (const { entry, generation } of liveReplacements) {
|
||||
this._replaceSession('live', entry, generation)
|
||||
this._replaceLiveSession(entry, generation)
|
||||
}
|
||||
db.exec('COMMIT')
|
||||
} catch (error: unknown) {
|
||||
@@ -360,21 +396,39 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
this._lastPersistenceRevision = observation.persistenceRevision
|
||||
}
|
||||
|
||||
private async _observeStable(signal: AbortSignal | undefined): Promise<Observation> {
|
||||
private async _observeStable(
|
||||
indexed: ReadonlyMap<SessionId, IndexedPersistedRow>,
|
||||
signal: AbortSignal | undefined,
|
||||
): Promise<Observation> {
|
||||
for (;;) {
|
||||
assertNotAborted(signal)
|
||||
const persistence = this._persistence
|
||||
const persistenceRevision = this._persistenceRevision
|
||||
const persisted = new Map<SessionId, ObservedSession>()
|
||||
let persisted = new Map<SessionId, ObservedPersistedSession>()
|
||||
if (persistence !== undefined) {
|
||||
try {
|
||||
const headers = await waitWithAbort(persistence.list(), signal)
|
||||
for (const listed of headers) {
|
||||
const loaded = await waitWithAbort(persistence.load(listed.id), signal)
|
||||
assertSessionHeadersCompatible(listed, loaded.meta)
|
||||
persisted.set(listed.id, observeSession(loaded.meta, loaded.events))
|
||||
const canReuseIndexed = this._lastPersistenceRevision === undefined
|
||||
|| this._lastPersistenceRevision === persistenceRevision
|
||||
const before = await waitWithAbort(persistence.listSnapshots(), signal)
|
||||
persisted = materializePersistenceSnapshots(before)
|
||||
for (const entry of persisted.values()) {
|
||||
if (canReuseIndexed && indexed.get(entry.header.id)?.revision === entry.revision) continue
|
||||
const loaded = await waitWithAbort(persistence.load(entry.header.id), signal)
|
||||
assertSessionHeadersCompatible(entry.header, loaded.meta)
|
||||
entry.loaded = observeSession(loaded.meta, loaded.events)
|
||||
}
|
||||
const after = materializePersistenceSnapshots(
|
||||
await waitWithAbort(persistence.listSnapshots(), signal),
|
||||
)
|
||||
if (!samePersistenceSnapshots(persisted, after)) continue
|
||||
if (this._persistenceRevision !== persistenceRevision) continue
|
||||
} catch (error: unknown) {
|
||||
if (isAbort(error) || signal?.aborted) {
|
||||
throw new SessionQueryError('session-search aborted', 'SESSION_QUERY_ABORTED', {
|
||||
cause: error,
|
||||
})
|
||||
}
|
||||
if (this._persistenceRevision !== persistenceRevision) continue
|
||||
if (error instanceof SessionQueryError) throw error
|
||||
throw new SessionQueryError(
|
||||
`session-search persistence observation failed: ${errorMessage(error)}`,
|
||||
@@ -414,13 +468,50 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
}
|
||||
}
|
||||
|
||||
private _replaceSession(source: 'persisted' | 'live', entry: ObservedSession, generation: number): void {
|
||||
this._deleteSession(source, entry.header.id)
|
||||
private _replacePersistedSession(
|
||||
entry: ObservedSession,
|
||||
revision: SessionPersistenceRevision,
|
||||
generation: number,
|
||||
): void {
|
||||
this._deleteSession('persisted', entry.header.id)
|
||||
const db = this._requireDb()
|
||||
const sessionTable = source === 'persisted' ? 'persisted_sessions' : 'temp.live_sessions'
|
||||
const docsTable = source === 'persisted' ? 'persisted_docs' : 'temp.live_docs'
|
||||
db.prepare(`
|
||||
INSERT INTO ${sessionTable}
|
||||
INSERT INTO persisted_sessions
|
||||
(id, version, created_at, cwd, parent_session, seed_length, revision, generation)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(
|
||||
entry.header.id,
|
||||
entry.header.version,
|
||||
entry.header.createdAt,
|
||||
entry.header.cwd ?? null,
|
||||
entry.header.parentSession ?? null,
|
||||
entry.header.seedLength ?? null,
|
||||
revision,
|
||||
generation,
|
||||
)
|
||||
const insert = db.prepare(`
|
||||
INSERT INTO persisted_docs (text, session_id, seq, type, time, surface, codepoint_length)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`)
|
||||
for (const document of entry.documents) {
|
||||
const text = sanitizeFtsText(document.text)
|
||||
insert.run(
|
||||
text,
|
||||
document.sessionId,
|
||||
document.seq,
|
||||
document.type,
|
||||
document.time,
|
||||
document.surface,
|
||||
Array.from(text).length,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
private _replaceLiveSession(entry: ObservedSession, generation: number): void {
|
||||
this._deleteSession('live', entry.header.id)
|
||||
const db = this._requireDb()
|
||||
db.prepare(`
|
||||
INSERT INTO temp.live_sessions
|
||||
(id, version, created_at, cwd, parent_session, seed_length, fingerprint, generation)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`).run(
|
||||
@@ -434,11 +525,20 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
generation,
|
||||
)
|
||||
const insert = db.prepare(`
|
||||
INSERT INTO ${docsTable} (text, session_id, seq, type, time, surface)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
INSERT INTO temp.live_docs (text, session_id, seq, type, time, surface, codepoint_length)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`)
|
||||
for (const document of entry.documents) {
|
||||
insert.run(document.text, document.sessionId, document.seq, document.type, document.time, document.surface)
|
||||
const text = sanitizeFtsText(document.text)
|
||||
insert.run(
|
||||
text,
|
||||
document.sessionId,
|
||||
document.seq,
|
||||
document.type,
|
||||
document.time,
|
||||
document.surface,
|
||||
Array.from(text).length,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -455,19 +555,16 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
ranked AS (
|
||||
SELECT *, ROW_NUMBER() OVER (
|
||||
PARTITION BY session_id
|
||||
ORDER BY score ASC, time DESC, seq DESC
|
||||
ORDER BY match_count DESC, document_length ASC, time DESC, seq DESC
|
||||
) AS event_rank
|
||||
FROM filtered
|
||||
)
|
||||
SELECT * FROM ranked
|
||||
WHERE event_rank = 1
|
||||
ORDER BY score ASC, time DESC, session_id ASC, seq DESC
|
||||
ORDER BY match_count DESC, document_length ASC, time DESC, session_id ASC, seq DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`).all(
|
||||
quoteFtsData(request.query),
|
||||
this._persistence === undefined ? 0 : 1,
|
||||
this._persistence === undefined ? 0 : 1,
|
||||
quoteFtsData(request.query),
|
||||
...selectedDocumentsParams(request.query, this._persistence !== undefined),
|
||||
...sessionWhere.params,
|
||||
...eventWhere.params,
|
||||
request.limit + 1,
|
||||
@@ -483,13 +580,10 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
${selected.sql}
|
||||
SELECT * FROM matched
|
||||
WHERE ${where}
|
||||
ORDER BY score ASC, time DESC, seq DESC
|
||||
ORDER BY match_count DESC, document_length ASC, time DESC, seq DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`).all(
|
||||
quoteFtsData(request.query),
|
||||
this._persistence === undefined ? 0 : 1,
|
||||
this._persistence === undefined ? 0 : 1,
|
||||
quoteFtsData(request.query),
|
||||
...selectedDocumentsParams(request.query, this._persistence !== undefined),
|
||||
request.sessionId,
|
||||
...eventWhere.params,
|
||||
request.limit + 1,
|
||||
@@ -515,23 +609,23 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
)
|
||||
}
|
||||
|
||||
private _sessionHit(row: SearchRow, query: string): SessionSearchHit {
|
||||
private _sessionHit(row: SearchRow): SessionSearchHit {
|
||||
return {
|
||||
header: rowHeader(row),
|
||||
live: row.live === 1,
|
||||
persisted: row.persisted === 1,
|
||||
bestMatch: this._eventHit(row, query),
|
||||
bestMatch: this._eventHit(row),
|
||||
}
|
||||
}
|
||||
|
||||
private _eventHit(row: SearchRow, query: string): SessionEventSearchHit {
|
||||
private _eventHit(row: SearchRow): SessionEventSearchHit {
|
||||
return {
|
||||
sessionId: row.session_id as SessionId,
|
||||
seq: row.seq,
|
||||
type: row.type as SessionEventSearchHit['type'],
|
||||
time: row.time,
|
||||
surface: row.surface as SessionEventSearchHit['surface'],
|
||||
snippet: makeSnippet(row.text, query, this.config.snippetChars),
|
||||
snippet: makeSnippet(row.marked_text, this.config.snippetChars),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -548,7 +642,7 @@ export class SessionSearchSqlite extends SessionSearchService {
|
||||
|
||||
function selectedDocumentsSql(): { sql: string } {
|
||||
return {
|
||||
sql: `WITH matched AS (
|
||||
sql: `WITH candidates AS (
|
||||
SELECT
|
||||
pd.session_id AS session_id,
|
||||
ps.version AS version,
|
||||
@@ -562,8 +656,8 @@ function selectedDocumentsSql(): { sql: string } {
|
||||
pd.type AS type,
|
||||
CAST(pd.time AS INTEGER) AS time,
|
||||
pd.surface AS surface,
|
||||
pd.text AS text,
|
||||
bm25(persisted_docs) AS score
|
||||
highlight(persisted_docs, 0, ?, ?) AS marked_text,
|
||||
CAST(pd.codepoint_length AS INTEGER) AS document_length
|
||||
FROM persisted_docs AS pd
|
||||
JOIN persisted_sessions AS ps ON ps.id = pd.session_id
|
||||
WHERE persisted_docs MATCH ?
|
||||
@@ -585,15 +679,39 @@ function selectedDocumentsSql(): { sql: string } {
|
||||
ld.type AS type,
|
||||
CAST(ld.time AS INTEGER) AS time,
|
||||
ld.surface AS surface,
|
||||
ld.text AS text,
|
||||
bm25(live_docs) AS score
|
||||
highlight(live_docs, 0, ?, ?) AS marked_text,
|
||||
CAST(ld.codepoint_length AS INTEGER) AS document_length
|
||||
FROM temp.live_docs AS ld
|
||||
JOIN temp.live_sessions AS ls ON ls.id = ld.session_id
|
||||
WHERE live_docs MATCH ?
|
||||
), matched AS (
|
||||
SELECT *,
|
||||
(
|
||||
length(CAST(marked_text AS BLOB))
|
||||
- length(CAST(replace(marked_text, ?, '') AS BLOB))
|
||||
) / ? AS match_count
|
||||
FROM candidates
|
||||
)`,
|
||||
}
|
||||
}
|
||||
|
||||
function selectedDocumentsParams(query: string, persistenceVisible: boolean): Array<string | number> {
|
||||
const expression = quoteFtsData(query)
|
||||
const visible = persistenceVisible ? 1 : 0
|
||||
return [
|
||||
FTS_HIGHLIGHT_START,
|
||||
FTS_HIGHLIGHT_END,
|
||||
expression,
|
||||
visible,
|
||||
visible,
|
||||
FTS_HIGHLIGHT_START,
|
||||
FTS_HIGHLIGHT_END,
|
||||
expression,
|
||||
FTS_HIGHLIGHT_START,
|
||||
Buffer.byteLength(FTS_HIGHLIGHT_START, 'utf8'),
|
||||
]
|
||||
}
|
||||
|
||||
function observeLive(session: Session): ObservedSession {
|
||||
return observeSession(
|
||||
structuredClone(session.header),
|
||||
@@ -606,7 +724,6 @@ function observeSession(header: SessionHeader, events: readonly SessionEvent[]):
|
||||
const detachedEvents = events.map(event => structuredClone(event))
|
||||
return {
|
||||
header: detachedHeader,
|
||||
events: detachedEvents,
|
||||
documents: buildSessionEventSearchDocuments(detachedHeader.id, detachedEvents),
|
||||
fingerprint: createHash('sha256')
|
||||
.update(JSON.stringify({ header: detachedHeader, events: detachedEvents }))
|
||||
@@ -614,6 +731,49 @@ function observeSession(header: SessionHeader, events: readonly SessionEvent[]):
|
||||
}
|
||||
}
|
||||
|
||||
function materializePersistenceSnapshots(
|
||||
snapshots: readonly SessionPersistenceSnapshot[],
|
||||
): Map<SessionId, ObservedPersistedSession> {
|
||||
if (!isRuntimeArray(snapshots)) throw new Error('persistence snapshots must be an array')
|
||||
const result = new Map<SessionId, ObservedPersistedSession>()
|
||||
for (const snapshot of snapshots) {
|
||||
if (typeof snapshot.revision !== 'string') {
|
||||
throw new Error('persistence snapshot revision must be a string')
|
||||
}
|
||||
const header = structuredClone(snapshot.header)
|
||||
if (result.has(header.id)) {
|
||||
throw new Error(`persistence listed duplicate session "${header.id}"`)
|
||||
}
|
||||
result.set(header.id, { header, revision: snapshot.revision })
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
function samePersistenceSnapshots(
|
||||
before: ReadonlyMap<SessionId, ObservedPersistedSession>,
|
||||
after: ReadonlyMap<SessionId, ObservedPersistedSession>,
|
||||
): boolean {
|
||||
if (before.size !== after.size) return false
|
||||
for (const [id, first] of before) {
|
||||
const second = after.get(id)
|
||||
if (
|
||||
second === undefined
|
||||
|| first.revision !== second.revision
|
||||
|| !sameHeader(first.header, second.header)
|
||||
) return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
function sameHeader(a: SessionHeader, b: SessionHeader): boolean {
|
||||
return a.version === b.version
|
||||
&& a.id === b.id
|
||||
&& a.createdAt === b.createdAt
|
||||
&& a.cwd === b.cwd
|
||||
&& a.parentSession === b.parentSession
|
||||
&& a.seedLength === b.seedLength
|
||||
}
|
||||
|
||||
function rowHeader(row: SearchRow): SessionHeader {
|
||||
return {
|
||||
version: row.version,
|
||||
@@ -629,7 +789,7 @@ function page<Row, Item>(
|
||||
rows: readonly Row[],
|
||||
limit: number,
|
||||
convert: (row: Row) => Item,
|
||||
nextCursor: (offset: number) => string,
|
||||
nextCursor: (offset: number) => SessionSearchCursorValue,
|
||||
offset: number,
|
||||
): SessionSearchPage<Item> {
|
||||
const hasMore = rows.length > limit
|
||||
@@ -639,12 +799,12 @@ function page<Row, Item>(
|
||||
}
|
||||
}
|
||||
|
||||
function encodeCursor(payload: CursorPayload): string {
|
||||
return Buffer.from(JSON.stringify(payload), 'utf8').toString('base64url')
|
||||
function encodeCursor(payload: CursorPayload): SessionSearchCursorValue {
|
||||
return SessionSearchCursor(Buffer.from(JSON.stringify(payload), 'utf8').toString('base64url'))
|
||||
}
|
||||
|
||||
function decodeCursor(
|
||||
cursor: string,
|
||||
cursor: SessionSearchCursorValue,
|
||||
instance: string,
|
||||
scope: CursorPayload['scope'],
|
||||
fingerprint: string,
|
||||
@@ -762,4 +922,8 @@ function errorMessage(error: unknown): string {
|
||||
return error instanceof Error ? error.message : 'unknown error'
|
||||
}
|
||||
|
||||
function isRuntimeArray(value: unknown): boolean {
|
||||
return Array.isArray(value)
|
||||
}
|
||||
|
||||
export default SessionSearchSqlite
|
||||
|
||||
@@ -2,16 +2,24 @@
|
||||
|
||||
import {
|
||||
SessionQueryError,
|
||||
filterSessionEventDocuments,
|
||||
filterSessionResults,
|
||||
materializeSessionEventResultFilters,
|
||||
materializeSessionResultFilters,
|
||||
} from '@deepseek-ai/dsh-session-query'
|
||||
import type {
|
||||
SessionAvailability,
|
||||
SessionEventMetadataFilter,
|
||||
SessionEventResultFilter,
|
||||
SessionEventSearchRequest,
|
||||
SessionResultFilter,
|
||||
SessionSearchCursor,
|
||||
SessionSearchRequest,
|
||||
} from '@deepseek-ai/dsh-session-query'
|
||||
|
||||
/** Collision-free marker inserted before an FTS5 match by `highlight()`. */
|
||||
export const FTS_HIGHLIGHT_START = '\uFDD0'
|
||||
/** Collision-free marker inserted after an FTS5 match by `highlight()`. */
|
||||
export const FTS_HIGHLIGHT_END = '\uFDD1'
|
||||
|
||||
/** Limit defaults needed to normalize a search request. */
|
||||
export interface QueryLimits {
|
||||
/** Page size used when the request omits one. */
|
||||
@@ -26,7 +34,7 @@ export interface NormalizedSessionRequest {
|
||||
sessionFilters: readonly SessionResultFilter[]
|
||||
eventFilters: readonly SessionEventMetadataFilter[]
|
||||
limit: number
|
||||
cursor?: string
|
||||
cursor?: SessionSearchCursor
|
||||
}
|
||||
|
||||
/** Normalized within-session request. */
|
||||
@@ -35,7 +43,7 @@ export interface NormalizedEventRequest {
|
||||
query: string
|
||||
filters: readonly SessionEventMetadataFilter[]
|
||||
limit: number
|
||||
cursor?: string
|
||||
cursor?: SessionSearchCursor
|
||||
}
|
||||
|
||||
/** Parameterized SQL predicate fragment. */
|
||||
@@ -56,16 +64,15 @@ export function normalizeSessionRequest(
|
||||
request: SessionSearchRequest,
|
||||
limits: QueryLimits,
|
||||
): NormalizedSessionRequest {
|
||||
const sessionFilters = request.sessionFilters ?? []
|
||||
const eventFilters = request.eventFilters ?? []
|
||||
filterSessionResults([], sessionFilters)
|
||||
filterSessionEventDocuments([], eventFilters)
|
||||
const sessionFilters = materializeSessionResultFilters(request.sessionFilters ?? [])
|
||||
const eventFilters = materializeMetadataFilters(request.eventFilters ?? [])
|
||||
const cursor = materializeCursor(request.cursor)
|
||||
return {
|
||||
query: normalizeQuery(request.query),
|
||||
sessionFilters,
|
||||
eventFilters,
|
||||
limit: normalizeLimit(request.limit, limits),
|
||||
...request.cursor === undefined ? {} : { cursor: request.cursor },
|
||||
...cursor === undefined ? {} : { cursor },
|
||||
}
|
||||
}
|
||||
|
||||
@@ -79,14 +86,17 @@ export function normalizeEventRequest(
|
||||
request: SessionEventSearchRequest,
|
||||
limits: QueryLimits,
|
||||
): NormalizedEventRequest {
|
||||
const filters = request.filters ?? []
|
||||
filterSessionEventDocuments([], filters)
|
||||
if (typeof request.sessionId !== 'string') {
|
||||
throw new SessionQueryError('session-search session id must be text', 'SESSION_QUERY_INVALID_FILTER')
|
||||
}
|
||||
const filters = materializeMetadataFilters(request.filters ?? [])
|
||||
const cursor = materializeCursor(request.cursor)
|
||||
return {
|
||||
sessionId: request.sessionId,
|
||||
query: normalizeQuery(request.query),
|
||||
filters,
|
||||
limit: normalizeLimit(request.limit, limits),
|
||||
...request.cursor === undefined ? {} : { cursor: request.cursor },
|
||||
...cursor === undefined ? {} : { cursor },
|
||||
}
|
||||
}
|
||||
|
||||
@@ -115,9 +125,23 @@ export function buildSessionWhere(filters: readonly SessionResultFilter[]): SqlW
|
||||
case 'availability': {
|
||||
const availability = [...new Set(filter.values)]
|
||||
if (availability.length === 0) clauses.push('0')
|
||||
else if (availability.length === 1) clauses.push(`${availability[0]} = 1`)
|
||||
else if (availability.length === 1) {
|
||||
const value = availability[0] as SessionAvailability
|
||||
switch (value) {
|
||||
case 'live':
|
||||
clauses.push('live = 1')
|
||||
break
|
||||
case 'persisted':
|
||||
clauses.push('persisted = 1')
|
||||
break
|
||||
default:
|
||||
unknownAvailability(value)
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
default:
|
||||
unknownFilter(filter)
|
||||
}
|
||||
}
|
||||
return { sql: clauses.join(' AND '), params }
|
||||
@@ -145,6 +169,8 @@ export function buildEventWhere(filters: readonly SessionEventMetadataFilter[]):
|
||||
case 'surface':
|
||||
addList(clauses, params, 'surface', filter.values)
|
||||
break
|
||||
default:
|
||||
unknownFilter(filter)
|
||||
}
|
||||
}
|
||||
return { sql: clauses.join(' AND '), params }
|
||||
@@ -159,6 +185,18 @@ export function quoteFtsData(query: string): string {
|
||||
return `"${query.replaceAll('"', '""')}"`
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove reserved marker collisions before text enters FTS5 or MATCH.
|
||||
* @param text - extracted document text or normalized caller query.
|
||||
* @returns text with reserved noncharacters mapped to replacement characters.
|
||||
*/
|
||||
export function sanitizeFtsText(text: string): string {
|
||||
return text
|
||||
.replaceAll('\0', '\uFFFD')
|
||||
.replaceAll(FTS_HIGHLIGHT_START, '\uFFFD')
|
||||
.replaceAll(FTS_HIGHLIGHT_END, '\uFFFD')
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the stable normalized request identity stored in opaque cursors.
|
||||
* @param request - normalized request whose filter ordering is canonicalized.
|
||||
@@ -185,19 +223,16 @@ export function requestFingerprint(request: NormalizedSessionRequest | Normalize
|
||||
|
||||
/**
|
||||
* Build a whitespace-normalized excerpt no longer than `maxChars`.
|
||||
* @param text - complete extracted semantic document.
|
||||
* @param query - normalized literal query used to position the excerpt.
|
||||
* @param markedText - complete document with FTS5 `highlight()` markers.
|
||||
* @param maxChars - maximum result length in Unicode code points.
|
||||
* @returns bounded plain-text snippet.
|
||||
*/
|
||||
export function makeSnippet(text: string, query: string, maxChars: number): string {
|
||||
const clean = text.replace(/\s+/gu, ' ').trim()
|
||||
export function makeSnippet(markedText: string, maxChars: number): string {
|
||||
const { text: clean, matchStart } = normalizeMarkedText(markedText)
|
||||
const characters = Array.from(clean)
|
||||
if (characters.length <= maxChars) return clean
|
||||
if (maxChars === 1) return '…'
|
||||
const foundUnits = clean.toLowerCase().indexOf(query.toLowerCase())
|
||||
const found = foundUnits < 0 ? -1 : Array.from(clean.slice(0, foundUnits)).length
|
||||
let start = found < 0 ? 0 : Math.max(0, found - Math.floor(maxChars / 3))
|
||||
let start = Math.max(0, matchStart - Math.floor(maxChars / 3))
|
||||
let prefix = start > 0 ? '…' : ''
|
||||
let suffix = '…'
|
||||
let contentLength = maxChars - prefix.length - suffix.length
|
||||
@@ -216,6 +251,28 @@ export function makeSnippet(text: string, query: string, maxChars: number): stri
|
||||
return `${prefix}${characters.slice(start, end).join('')}${suffix}`
|
||||
}
|
||||
|
||||
function normalizeMarkedText(markedText: string): { text: string; matchStart: number } {
|
||||
const characters: string[] = []
|
||||
let matchStart: number | undefined
|
||||
for (const character of markedText) {
|
||||
if (character === FTS_HIGHLIGHT_START) {
|
||||
matchStart ??= characters.length
|
||||
continue
|
||||
}
|
||||
if (character === FTS_HIGHLIGHT_END) continue
|
||||
if (/\s/u.test(character)) {
|
||||
if (characters.length > 0 && characters.at(-1) !== ' ') characters.push(' ')
|
||||
} else {
|
||||
characters.push(character)
|
||||
}
|
||||
}
|
||||
if (characters.at(-1) === ' ') characters.pop()
|
||||
return {
|
||||
text: characters.join(''),
|
||||
matchStart: matchStart ?? 0,
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeQuery(value: string): string {
|
||||
if (typeof value !== 'string') {
|
||||
throw new SessionQueryError('session-search query must be text', 'SESSION_QUERY_INVALID_QUERY')
|
||||
@@ -227,7 +284,44 @@ function normalizeQuery(value: string): string {
|
||||
'SESSION_QUERY_INVALID_QUERY',
|
||||
)
|
||||
}
|
||||
return query
|
||||
if (query.includes('\0')) {
|
||||
throw new SessionQueryError(
|
||||
'session-search query must not contain NUL',
|
||||
'SESSION_QUERY_INVALID_QUERY',
|
||||
)
|
||||
}
|
||||
return sanitizeFtsText(query)
|
||||
}
|
||||
|
||||
function materializeCursor(cursor: SessionSearchCursor | undefined): SessionSearchCursor | undefined {
|
||||
if (cursor === undefined) return undefined
|
||||
if (typeof cursor !== 'string') {
|
||||
throw new SessionQueryError('session-search cursor must be text', 'SESSION_QUERY_INVALID_CURSOR')
|
||||
}
|
||||
return cursor
|
||||
}
|
||||
|
||||
function materializeMetadataFilters(
|
||||
filters: readonly SessionEventMetadataFilter[],
|
||||
): SessionEventMetadataFilter[] {
|
||||
const candidates: readonly SessionEventResultFilter[] = filters
|
||||
for (const filter of candidates) {
|
||||
switch (filter.kind) {
|
||||
case 'seq':
|
||||
case 'time':
|
||||
case 'type':
|
||||
case 'surface':
|
||||
break
|
||||
case 'text':
|
||||
throw new SessionQueryError(
|
||||
'session-search metadata filters do not accept text clauses',
|
||||
'SESSION_QUERY_INVALID_FILTER',
|
||||
)
|
||||
default:
|
||||
unknownFilter(filter)
|
||||
}
|
||||
}
|
||||
return materializeSessionEventResultFilters(filters) as SessionEventMetadataFilter[]
|
||||
}
|
||||
|
||||
function normalizeLimit(value: number | undefined, limits: QueryLimits): number {
|
||||
@@ -310,3 +404,18 @@ function compareNullable(a: string | null, b: string | null): number {
|
||||
if (b === null) return 1
|
||||
return a.localeCompare(b)
|
||||
}
|
||||
|
||||
function unknownAvailability(value: never): never {
|
||||
throw new SessionQueryError(
|
||||
`session availability filter contains unknown value "${String(value)}"`,
|
||||
'SESSION_QUERY_INVALID_FILTER',
|
||||
)
|
||||
}
|
||||
|
||||
function unknownFilter(filter: never): never {
|
||||
const kind = (filter as { kind?: unknown }).kind
|
||||
throw new SessionQueryError(
|
||||
`session filter contains unknown kind ${typeof kind === 'string' ? `"${kind}"` : '(missing)'}`,
|
||||
'SESSION_QUERY_INVALID_FILTER',
|
||||
)
|
||||
}
|
||||
|
||||
@@ -5,7 +5,7 @@ import { mkdir } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
|
||||
/** Current derived-index schema version. Incompatible versions reset in place. */
|
||||
export const SESSION_QUERY_SQLITE_SCHEMA_VERSION = 1
|
||||
export const SESSION_QUERY_SQLITE_SCHEMA_VERSION = 2
|
||||
|
||||
/** SQLite application id protecting unrelated databases from derived resets. */
|
||||
export const SESSION_QUERY_SQLITE_APPLICATION_ID = 0x44534851
|
||||
@@ -24,8 +24,6 @@ export async function openSearchDatabase(path: string, journalMode: JournalMode)
|
||||
if (actual !== ':memory:') await mkdir(dirname(actual), { recursive: true, mode: 0o700 })
|
||||
const db = new DatabaseSync(actual)
|
||||
try {
|
||||
// journalMode is a validated closed union, not caller-controlled SQL.
|
||||
db.exec(`PRAGMA journal_mode = ${journalMode.toUpperCase()}`)
|
||||
const { application_id: applicationId } = db.prepare('PRAGMA application_id').get() as { application_id: number }
|
||||
const { user_version: version } = db.prepare('PRAGMA user_version').get() as { user_version: number }
|
||||
const userTables = listUserTables(db)
|
||||
@@ -38,6 +36,9 @@ export async function openSearchDatabase(path: string, journalMode: JournalMode)
|
||||
if (applicationId === SESSION_QUERY_SQLITE_APPLICATION_ID && version !== SESSION_QUERY_SQLITE_SCHEMA_VERSION) {
|
||||
resetDerivedSchema(db)
|
||||
}
|
||||
// Apply mutating pragmas only after refusing foreign or canonical files.
|
||||
// journalMode is a validated closed union, not caller-controlled SQL.
|
||||
db.exec(`PRAGMA journal_mode = ${journalMode.toUpperCase()}`)
|
||||
ensurePersistentSchema(db)
|
||||
ensureTemporarySchema(db)
|
||||
return db
|
||||
@@ -78,7 +79,7 @@ function ensurePersistentSchema(db: DatabaseSync): void {
|
||||
cwd TEXT,
|
||||
parent_session TEXT,
|
||||
seed_length INTEGER,
|
||||
fingerprint TEXT NOT NULL,
|
||||
revision TEXT NOT NULL,
|
||||
generation INTEGER NOT NULL
|
||||
) STRICT
|
||||
`)
|
||||
@@ -90,6 +91,7 @@ function ensurePersistentSchema(db: DatabaseSync): void {
|
||||
type UNINDEXED,
|
||||
time UNINDEXED,
|
||||
surface UNINDEXED,
|
||||
codepoint_length UNINDEXED,
|
||||
tokenize = 'unicode61'
|
||||
)
|
||||
`)
|
||||
@@ -117,6 +119,7 @@ function ensureTemporarySchema(db: DatabaseSync): void {
|
||||
type UNINDEXED,
|
||||
time UNINDEXED,
|
||||
surface UNINDEXED,
|
||||
codepoint_length UNINDEXED,
|
||||
tokenize = 'unicode61'
|
||||
)
|
||||
`)
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionQueryErrorCode } from '@deepseek-ai/dsh-session-query'
|
||||
import { SessionSearchCursor, type SessionQueryErrorCode } from '@deepseek-ai/dsh-session-query'
|
||||
import {
|
||||
buildEventWhere,
|
||||
buildSessionWhere,
|
||||
FTS_HIGHLIGHT_END,
|
||||
FTS_HIGHLIGHT_START,
|
||||
makeSnippet,
|
||||
normalizeEventRequest,
|
||||
normalizeSessionRequest,
|
||||
@@ -32,13 +34,13 @@ describe('SQLite search request normalization', () => {
|
||||
sessionFilters: [{ kind: 'availability', values: ['live'] }],
|
||||
eventFilters: [{ kind: 'surface', values: ['current'] }],
|
||||
limit: 3,
|
||||
cursor: 'next',
|
||||
cursor: SessionSearchCursor('next'),
|
||||
}, limits)).toEqual({
|
||||
query: 'needle',
|
||||
sessionFilters: [{ kind: 'availability', values: ['live'] }],
|
||||
eventFilters: [{ kind: 'surface', values: ['current'] }],
|
||||
limit: 3,
|
||||
cursor: 'next',
|
||||
cursor: SessionSearchCursor('next'),
|
||||
})
|
||||
expect(normalizeEventRequest({ sessionId: SessionId('s'), query: 'needle' }, limits)).toEqual({
|
||||
sessionId: SessionId('s'),
|
||||
@@ -50,13 +52,13 @@ describe('SQLite search request normalization', () => {
|
||||
sessionId: SessionId('s'),
|
||||
query: 'needle',
|
||||
filters: [{ kind: 'seq', from: 1 }],
|
||||
cursor: 'next',
|
||||
cursor: SessionSearchCursor('next'),
|
||||
}, limits)).toEqual({
|
||||
sessionId: SessionId('s'),
|
||||
query: 'needle',
|
||||
filters: [{ kind: 'seq', from: 1 }],
|
||||
limit: 2,
|
||||
cursor: 'next',
|
||||
cursor: SessionSearchCursor('next'),
|
||||
})
|
||||
})
|
||||
|
||||
@@ -65,11 +67,39 @@ describe('SQLite search request normalization', () => {
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_QUERY'))
|
||||
expect(() => normalizeSessionRequest({ query: ' \n ' }, limits))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_QUERY'))
|
||||
expect(() => normalizeSessionRequest({ query: 'bad\0query' }, limits))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_QUERY'))
|
||||
expect(() => normalizeEventRequest({ sessionId: 1 as never, query: 'x' }, limits))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => normalizeEventRequest({
|
||||
sessionId: SessionId('s'),
|
||||
query: 'x',
|
||||
cursor: 1 as never,
|
||||
}, limits)).toThrow(expectCode('SESSION_QUERY_INVALID_CURSOR'))
|
||||
expect(() => normalizeSessionRequest({
|
||||
query: 'x',
|
||||
eventFilters: [{ kind: 'text', text: 'x' } as never],
|
||||
}, limits)).toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => normalizeSessionRequest({
|
||||
query: 'x',
|
||||
eventFilters: [{} as never],
|
||||
}, limits)).toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
for (const limit of [1.5, 0, 4]) {
|
||||
expect(() => normalizeEventRequest({ sessionId: SessionId('s'), query: 'x', limit }, limits))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_LIMIT'))
|
||||
}
|
||||
})
|
||||
|
||||
it('materializes owned filter values during normalization', () => {
|
||||
const values = ['live'] as Array<'live' | 'persisted'>
|
||||
const filter = { kind: 'availability' as const, values }
|
||||
const request = { query: 'needle', sessionFilters: [filter] }
|
||||
const normalized = normalizeSessionRequest(request, limits)
|
||||
|
||||
values[0] = 'persisted'
|
||||
request.sessionFilters.push({ kind: 'availability', values: ['persisted'] })
|
||||
expect(normalized.sessionFilters).toEqual([{ kind: 'availability', values: ['live'] }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('SQLite search predicate compilation', () => {
|
||||
@@ -120,6 +150,17 @@ describe('SQLite search predicate compilation', () => {
|
||||
{ kind: 'surface', values: [] },
|
||||
])).toEqual({ sql: '0 AND 0', params: [] })
|
||||
})
|
||||
|
||||
it('rejects runtime-unknown filter discriminants in both SQL builders', () => {
|
||||
expect(() => buildSessionWhere([{ kind: 'future' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => buildEventWhere([{ kind: 'future' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => buildSessionWhere([{ kind: 'availability', values: ['future'] } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => buildSessionWhere([{} as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
})
|
||||
})
|
||||
|
||||
describe('SQLite query identity and presentation', () => {
|
||||
@@ -169,11 +210,13 @@ describe('SQLite query identity and presentation', () => {
|
||||
})
|
||||
|
||||
it('normalizes, bounds, and positions snippets by Unicode code point', () => {
|
||||
expect(makeSnippet(' short\ntext ', 'absent', 20)).toBe('short text')
|
||||
expect(makeSnippet('abcdef', 'f', 1)).toBe('…')
|
||||
expect(makeSnippet('abcdefghij', 'absent', 5)).toBe('abcd…')
|
||||
expect(makeSnippet('abcdefghij', 'c', 5)).toBe('…bcd…')
|
||||
expect(makeSnippet('abcdef', 'f', 2)).toBe('a…')
|
||||
expect(makeSnippet('abcdef', 'f', 5)).toBe('…cdef')
|
||||
expect(makeSnippet(' short\ntext ', 20)).toBe('short text')
|
||||
expect(makeSnippet(`abcde${FTS_HIGHLIGHT_START}f${FTS_HIGHLIGHT_END}`, 1)).toBe('…')
|
||||
expect(makeSnippet('abcdefghij', 5)).toBe('abcd…')
|
||||
expect(makeSnippet(`ab${FTS_HIGHLIGHT_START}c${FTS_HIGHLIGHT_END}defghij`, 5)).toBe('…bcd…')
|
||||
expect(makeSnippet(`abcde${FTS_HIGHLIGHT_START}f${FTS_HIGHLIGHT_END}`, 2)).toBe('a…')
|
||||
expect(makeSnippet(`abcde${FTS_HIGHLIGHT_START}f${FTS_HIGHLIGHT_END}`, 5)).toBe('…cdef')
|
||||
expect(makeSnippet(` x—${FTS_HIGHLIGHT_START}café${FTS_HIGHLIGHT_END}\n y `, 20))
|
||||
.toBe('x—café y')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,18 +1,25 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader, SessionId as SessionIdType } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistence from '@deepseek-ai/dsh-session-persistence'
|
||||
import SessionPersistence, { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { SessionPersistenceSnapshot } from '@deepseek-ai/dsh-session-persistence'
|
||||
import SessionPersistenceSqlite from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import SessionSearchSqlite, {
|
||||
SESSION_QUERY_SQLITE_APPLICATION_ID,
|
||||
SESSION_QUERY_SQLITE_SCHEMA_VERSION,
|
||||
} from '@deepseek-ai/dsh-session-query-sqlite'
|
||||
import type { SessionQueryErrorCode } from '@deepseek-ai/dsh-session-query'
|
||||
import {
|
||||
SessionQueryError,
|
||||
SessionSearchCursor,
|
||||
type SessionAvailability,
|
||||
type SessionQueryErrorCode,
|
||||
type SessionSearchRequest,
|
||||
} from '@deepseek-ai/dsh-session-query'
|
||||
|
||||
const temporaryDirectories: string[] = []
|
||||
|
||||
@@ -48,19 +55,36 @@ function expectCode(code: SessionQueryErrorCode): Error {
|
||||
|
||||
class TestPersistence extends SessionPersistence {
|
||||
static entries = new Map<SessionIdType, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
static revisions = new Map<SessionIdType, number>()
|
||||
static nextRevision = 0
|
||||
static loads = new Map<SessionIdType, number>()
|
||||
static loadEffect: ((entry: { meta: SessionHeader; events: SessionEvent[] }) => void) | undefined
|
||||
static listGate: Promise<void> | undefined
|
||||
static listStarted: (() => void) | undefined
|
||||
static snapshotEffect: (() => void | Promise<void>) | undefined
|
||||
static snapshotOverride: (() => SessionPersistenceSnapshot[]) | undefined
|
||||
static failure: unknown
|
||||
|
||||
static reset(entries: readonly { meta: SessionHeader; events: SessionEvent[] }[] = []): void {
|
||||
this.entries = new Map(entries.map(entry => [entry.meta.id, structuredClone(entry)]))
|
||||
this.entries = new Map()
|
||||
this.revisions = new Map()
|
||||
this.loads = new Map()
|
||||
this.loadEffect = undefined
|
||||
for (const entry of entries) this.set(entry)
|
||||
this.listGate = undefined
|
||||
this.listStarted = undefined
|
||||
this.snapshotEffect = undefined
|
||||
this.snapshotOverride = undefined
|
||||
this.failure = undefined
|
||||
}
|
||||
|
||||
static set(entry: { meta: SessionHeader; events: SessionEvent[] }): void {
|
||||
this.entries.set(entry.meta.id, structuredClone(entry))
|
||||
this.revisions.set(entry.meta.id, ++this.nextRevision)
|
||||
}
|
||||
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
TestPersistence.entries.set(meta.id, { meta: structuredClone(meta), events: [] })
|
||||
TestPersistence.set({ meta, events: [] })
|
||||
return Promise.resolve()
|
||||
}
|
||||
|
||||
@@ -68,13 +92,21 @@ class TestPersistence extends SessionPersistence {
|
||||
const entry = TestPersistence.entries.get(id)
|
||||
if (entry === undefined) return Promise.reject(new Error('missing test session'))
|
||||
entry.events.push(...structuredClone(events))
|
||||
TestPersistence.revisions.set(id, ++TestPersistence.nextRevision)
|
||||
return Promise.resolve()
|
||||
}
|
||||
|
||||
async load(id: SessionIdType): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
TestPersistence.loads.set(id, (TestPersistence.loads.get(id) ?? 0) + 1)
|
||||
if (TestPersistence.failure !== undefined) throw TestPersistence.failure
|
||||
const entry = TestPersistence.entries.get(id)
|
||||
if (entry === undefined) throw new Error('missing test session')
|
||||
if (TestPersistence.loadEffect !== undefined) {
|
||||
const effect = TestPersistence.loadEffect
|
||||
TestPersistence.loadEffect = undefined
|
||||
effect(entry)
|
||||
TestPersistence.revisions.set(id, ++TestPersistence.nextRevision)
|
||||
}
|
||||
return structuredClone(entry)
|
||||
}
|
||||
|
||||
@@ -84,6 +116,20 @@ class TestPersistence extends SessionPersistence {
|
||||
if (TestPersistence.failure !== undefined) throw TestPersistence.failure
|
||||
return [...TestPersistence.entries.values()].map(entry => structuredClone(entry.meta))
|
||||
}
|
||||
|
||||
|
||||
async listSnapshots(): Promise<SessionPersistenceSnapshot[]> {
|
||||
TestPersistence.listStarted?.()
|
||||
await TestPersistence.listGate
|
||||
if (TestPersistence.failure !== undefined) throw TestPersistence.failure
|
||||
const snapshots = TestPersistence.snapshotOverride?.()
|
||||
?? [...TestPersistence.entries.values()].map(entry => ({
|
||||
header: structuredClone(entry.meta),
|
||||
revision: SessionPersistenceRevision(`test:${TestPersistence.revisions.get(entry.meta.id)}`),
|
||||
}))
|
||||
await TestPersistence.snapshotEffect?.()
|
||||
return snapshots
|
||||
}
|
||||
}
|
||||
|
||||
async function liveContext(config: ConstructorParameters<typeof SessionSearchSqlite>[1] = { path: ':memory:' }): Promise<Context> {
|
||||
@@ -175,6 +221,44 @@ describe('SQLite session search', () => {
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: '*' })).resolves.toEqual({ items: [] })
|
||||
})
|
||||
|
||||
it('ranks live and persisted matches on one source-comparable contract', async () => {
|
||||
const persisted = header('z-persisted')
|
||||
TestPersistence.reset([
|
||||
{ meta: persisted, events: messageEvents('needle needle', 10) },
|
||||
...Array.from({ length: 12 }, (_, index) => ({
|
||||
meta: header(`filler-${index}`),
|
||||
events: messageEvents('needle', 10),
|
||||
})),
|
||||
])
|
||||
const ctx = await liveContext()
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
ctx.sessions.create(SessionId('a-live'), {
|
||||
seed: messageEvents('needle needle', 10),
|
||||
meta: { createdAt: persisted.createdAt },
|
||||
})
|
||||
|
||||
const result = await ctx.sessionSearch.searchSessions({
|
||||
query: 'needle',
|
||||
sessionFilters: [{ kind: 'id', values: [SessionId('a-live'), persisted.id] }],
|
||||
})
|
||||
expect(result.items.map(item => item.header.id)).toEqual([SessionId('a-live'), persisted.id])
|
||||
await persistence.dispose()
|
||||
})
|
||||
|
||||
it('positions snippets from FTS5 matches across diacritics and punctuation', async () => {
|
||||
const ctx = await liveContext({ path: ':memory:', snippetChars: 14 })
|
||||
const session = ctx.sessions.create(SessionId('snippet'), {
|
||||
seed: messageEvents('long long long—café,\nnext value', 10),
|
||||
})
|
||||
|
||||
const page = await ctx.sessionSearch.searchEvents({ sessionId: session.id, query: 'CAFE' })
|
||||
expect(page.items).toHaveLength(1)
|
||||
expect(page.items[0]!.snippet).toContain('café')
|
||||
expect(page.items[0]!.snippet).toContain('—')
|
||||
expect(page.items[0]!.snippet).not.toContain('\n')
|
||||
expect(Array.from(page.items[0]!.snippet).length).toBeLessThanOrEqual(14)
|
||||
})
|
||||
|
||||
it('binds cursors to requests and only invalidates within-session pages for target changes', async () => {
|
||||
const ctx = await liveContext({ path: ':memory:', defaultLimit: 1, maxLimit: 5 })
|
||||
const target = ctx.sessions.create(SessionId('target'), {
|
||||
@@ -193,7 +277,7 @@ describe('SQLite session search', () => {
|
||||
if (eventPage.nextCursor === undefined || sessionPage.nextCursor === undefined) throw new Error('expected cursors')
|
||||
|
||||
const eventKeys = eventPage.items.map(item => `${item.sessionId}:${item.seq}`)
|
||||
let eventCursor: string | undefined = eventPage.nextCursor
|
||||
let eventCursor: ReturnType<typeof SessionSearchCursor> | undefined = eventPage.nextCursor
|
||||
while (eventCursor !== undefined) {
|
||||
const next = await ctx.sessionSearch.searchEvents({
|
||||
sessionId: target.id,
|
||||
@@ -208,7 +292,7 @@ describe('SQLite session search', () => {
|
||||
expect(new Set(eventKeys).size).toBe(eventKeys.length)
|
||||
|
||||
const sessionIds = sessionPage.items.map(item => item.header.id)
|
||||
let sessionCursor: string | undefined = sessionPage.nextCursor
|
||||
let sessionCursor: ReturnType<typeof SessionSearchCursor> | undefined = sessionPage.nextCursor
|
||||
while (sessionCursor !== undefined) {
|
||||
const next = await ctx.sessionSearch.searchSessions({ query: 'needle', limit: 1, cursor: sessionCursor })
|
||||
sessionIds.push(...next.items.map(item => item.header.id))
|
||||
@@ -251,6 +335,7 @@ describe('SQLite session search', () => {
|
||||
{ sessionId: session.id, query: 'needle', limit: 4 },
|
||||
{ sessionId: session.id, query: 'needle', filters: [{ kind: 'seq', from: 2, to: 1 }] },
|
||||
{ sessionId: session.id, query: 'needle', filters: [{ kind: 'surface', values: ['future'] }] },
|
||||
{ sessionId: session.id, query: 'bad\0query' },
|
||||
] as const) {
|
||||
await expect(ctx.sessionSearch.searchEvents(request as never)).rejects.toBeInstanceOf(Error)
|
||||
}
|
||||
@@ -258,7 +343,24 @@ describe('SQLite session search', () => {
|
||||
query: 'needle',
|
||||
sessionFilters: [{ kind: 'availability', values: ['remote' as never] }],
|
||||
})).rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await expect(ctx.sessionSearch.searchEvents({ sessionId: session.id, query: 'needle', cursor: 'not-json' }))
|
||||
await expect(ctx.sessionSearch.searchSessions({
|
||||
query: 'needle',
|
||||
sessionFilters: [{ kind: 'future' } as never],
|
||||
})).rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await expect(ctx.sessionSearch.searchSessions({
|
||||
query: 'needle',
|
||||
eventFilters: [{ kind: 'future' } as never],
|
||||
})).rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await expect(ctx.sessionSearch.searchEvents({
|
||||
sessionId: session.id,
|
||||
query: 'needle',
|
||||
filters: [{ kind: 'future' } as never],
|
||||
})).rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await expect(ctx.sessionSearch.searchEvents({
|
||||
sessionId: session.id,
|
||||
query: 'needle',
|
||||
cursor: SessionSearchCursor('not-json'),
|
||||
}))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INVALID_CURSOR'))
|
||||
await expect(ctx.sessionSearch.searchEvents({ sessionId: SessionId('absent'), query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_SESSION_NOT_FOUND'))
|
||||
@@ -280,6 +382,37 @@ describe('SQLite session search', () => {
|
||||
})
|
||||
|
||||
describe('SQLite reconciliation and source lifecycle', () => {
|
||||
it('owns queued request and filter values before waiting for the serializer', async () => {
|
||||
const durable = header('owned')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
let release!: () => void
|
||||
TestPersistence.listGate = new Promise<void>((resolve) => { release = resolve })
|
||||
let markStarted!: () => void
|
||||
const started = new Promise<void>((resolve) => { markStarted = resolve })
|
||||
TestPersistence.listStarted = () => {
|
||||
TestPersistence.listStarted = undefined
|
||||
markStarted()
|
||||
}
|
||||
const blocking = ctx.sessionSearch.searchSessions({ query: 'needle' })
|
||||
await started
|
||||
|
||||
const availability: SessionAvailability[] = ['persisted']
|
||||
const request: SessionSearchRequest = {
|
||||
query: 'needle',
|
||||
sessionFilters: [{ kind: 'availability', values: availability }],
|
||||
}
|
||||
const queued = ctx.sessionSearch.searchSessions(request)
|
||||
request.query = 'absent'
|
||||
availability[0] = 'live'
|
||||
release()
|
||||
|
||||
await expect(blocking).resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
await expect(queued).resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
await persistence.dispose()
|
||||
})
|
||||
|
||||
it('mounts persistence dynamically, shadows with TEMP live rows, reveals, and hides on unmount', async () => {
|
||||
const shared = header('shared', 10, { cwd: '/work' })
|
||||
const durable = header('durable', 5)
|
||||
@@ -311,7 +444,7 @@ describe('SQLite reconciliation and source lifecycle', () => {
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_SESSION_NOT_FOUND'))
|
||||
})
|
||||
|
||||
it('restarts observation when persistence unmounts during an asynchronous list', async () => {
|
||||
it('discards a stale list rejection when persistence unmounts during observation', async () => {
|
||||
const durable = header('racing')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
@@ -328,10 +461,140 @@ describe('SQLite reconciliation and source lifecycle', () => {
|
||||
const search = ctx.sessionSearch.searchSessions({ query: 'needle' })
|
||||
await started
|
||||
await persistenceFiber.dispose()
|
||||
TestPersistence.failure = new Error('stale backend rejection')
|
||||
release()
|
||||
await expect(search).resolves.toEqual({ items: [] })
|
||||
})
|
||||
|
||||
it('retries against a replacement after the prior binding rejects', async () => {
|
||||
const durable = header('replacement')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
const prior = await ctx.plugin(TestPersistence)
|
||||
let rejectPrior!: (reason: unknown) => void
|
||||
TestPersistence.listGate = new Promise<void>((_resolve, reject) => { rejectPrior = reject })
|
||||
let markStarted!: () => void
|
||||
const started = new Promise<void>((resolve) => { markStarted = resolve })
|
||||
TestPersistence.listStarted = () => {
|
||||
TestPersistence.listStarted = undefined
|
||||
markStarted()
|
||||
}
|
||||
|
||||
const search = ctx.sessionSearch.searchSessions({ query: 'needle' })
|
||||
await started
|
||||
await prior.dispose()
|
||||
TestPersistence.listGate = undefined
|
||||
const replacement = await ctx.plugin(TestPersistence)
|
||||
rejectPrior(new Error('stale prior binding'))
|
||||
await expect(search).resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
await replacement.dispose()
|
||||
})
|
||||
|
||||
it('reloads a replacement source even when its opaque revisions collide', async () => {
|
||||
const durable = header('colliding-replacement')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('old content') }])
|
||||
const revision = TestPersistence.revisions.get(durable.id)!
|
||||
const ctx = await liveContext()
|
||||
const prior = await ctx.plugin(TestPersistence)
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'old' }))
|
||||
.resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
await prior.dispose()
|
||||
|
||||
TestPersistence.set({ meta: durable, events: messageEvents('new needle') })
|
||||
TestPersistence.revisions.set(durable.id, revision)
|
||||
const replacement = await ctx.plugin(TestPersistence)
|
||||
const internals = ctx.sessionSearch as unknown as {
|
||||
_lastPersistenceRevision: number
|
||||
_persistenceRevision: number
|
||||
}
|
||||
expect(internals._persistenceRevision).not.toBe(internals._lastPersistenceRevision)
|
||||
const page = await ctx.sessionSearch.searchSessions({ query: 'new needle' })
|
||||
expect(TestPersistence.loads.get(durable.id)).toBe(2)
|
||||
expect(page).toMatchObject({ items: [{ header: durable }] })
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'old' })).resolves.toEqual({ items: [] })
|
||||
expect(TestPersistence.loads.get(durable.id)).toBe(2)
|
||||
await replacement.dispose()
|
||||
})
|
||||
|
||||
it('retries when a successful observation belongs to a source unmounted during listing', async () => {
|
||||
const durable = header('successful-unmount')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
let lists = 0
|
||||
TestPersistence.snapshotEffect = async () => {
|
||||
lists += 1
|
||||
if (lists === 2) await persistence.dispose()
|
||||
}
|
||||
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' })).resolves.toEqual({ items: [] })
|
||||
expect(lists).toBe(2)
|
||||
})
|
||||
|
||||
it('retries when the snapshot population changes during observation', async () => {
|
||||
const first = header('first')
|
||||
const added = header('added-during-list')
|
||||
TestPersistence.reset([{ meta: first, events: messageEvents('first needle') }])
|
||||
const ctx = await liveContext()
|
||||
await ctx.plugin(TestPersistence)
|
||||
TestPersistence.snapshotEffect = () => {
|
||||
TestPersistence.snapshotEffect = undefined
|
||||
TestPersistence.set({ meta: added, events: messageEvents('added needle') })
|
||||
}
|
||||
|
||||
const page = await ctx.sessionSearch.searchSessions({ query: 'needle' })
|
||||
expect(page.items.map(item => item.header.id).sort()).toEqual([added.id, first.id].sort())
|
||||
expect(TestPersistence.loads.get(first.id)).toBe(2)
|
||||
expect(TestPersistence.loads.get(added.id)).toBe(1)
|
||||
})
|
||||
|
||||
it('retries if the source revision changes while live sessions are observed', async () => {
|
||||
const durable = header('live-boundary-retry')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
await ctx.plugin(TestPersistence)
|
||||
const internals = ctx.sessionSearch as unknown as { _persistenceRevision: number }
|
||||
const originalList = ctx.sessions.list.bind(ctx.sessions)
|
||||
let bumped = false
|
||||
const list = vi.spyOn(ctx.sessions, 'list').mockImplementation(() => {
|
||||
if (!bumped) {
|
||||
bumped = true
|
||||
internals._persistenceRevision += 1
|
||||
}
|
||||
return originalList()
|
||||
})
|
||||
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
expect(TestPersistence.loads.get(durable.id)).toBe(2)
|
||||
list.mockRestore()
|
||||
})
|
||||
|
||||
it('rejects malformed snapshots and preserves typed persistence failures', async () => {
|
||||
const durable = header('invalid-snapshot')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
await ctx.plugin(TestPersistence)
|
||||
|
||||
TestPersistence.snapshotOverride = () => 'not-an-array' as never
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED'))
|
||||
TestPersistence.snapshotOverride = () => [{ header: durable, revision: 1 as never }]
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED'))
|
||||
TestPersistence.snapshotOverride = () => [
|
||||
{ header: durable, revision: SessionPersistenceRevision('duplicate:1') },
|
||||
{ header: durable, revision: SessionPersistenceRevision('duplicate:2') },
|
||||
]
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED'))
|
||||
|
||||
TestPersistence.snapshotOverride = undefined
|
||||
const typed = new SessionQueryError('typed persistence failure', 'SESSION_QUERY_PERSISTENCE_FAILED')
|
||||
TestPersistence.failure = typed
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'needle' })).rejects.toBe(typed)
|
||||
})
|
||||
|
||||
it('rejects immutable header conflicts between live and persisted sources', async () => {
|
||||
const shared = header('conflict', 10)
|
||||
TestPersistence.reset([{ meta: shared, events: messageEvents('persisted needle') }])
|
||||
@@ -358,6 +621,9 @@ describe('SQLite reconciliation and source lifecycle', () => {
|
||||
const firstPersistence = await first.plugin(TestPersistence)
|
||||
const firstSearch = await first.plugin(SessionSearchSqlite, { path })
|
||||
await first.sessionSearch.searchSessions({ query: 'needle' })
|
||||
expect(Object.fromEntries(TestPersistence.loads)).toEqual({ unchanged: 1, changed: 1, deleted: 1 })
|
||||
await first.sessionSearch.searchSessions({ query: 'needle' })
|
||||
expect(Object.fromEntries(TestPersistence.loads)).toEqual({ unchanged: 1, changed: 1, deleted: 1 })
|
||||
await firstSearch.dispose()
|
||||
await firstPersistence.dispose()
|
||||
|
||||
@@ -368,14 +634,20 @@ describe('SQLite reconciliation and source lifecycle', () => {
|
||||
|
||||
const added = header('added')
|
||||
TestPersistence.entries.delete(deleted.id)
|
||||
TestPersistence.entries.set(changed.id, { meta: changed, events: messageEvents('changed needle') })
|
||||
TestPersistence.entries.set(added.id, { meta: added, events: messageEvents('added needle') })
|
||||
TestPersistence.set({ meta: changed, events: messageEvents('changed needle') })
|
||||
TestPersistence.set({ meta: added, events: messageEvents('added needle') })
|
||||
const second = new Context()
|
||||
await second.plugin(SessionStore)
|
||||
const secondPersistence = await second.plugin(TestPersistence)
|
||||
const secondSearch = await second.plugin(SessionSearchSqlite, { path })
|
||||
const result = await second.sessionSearch.searchSessions({ query: 'needle' })
|
||||
expect(result.items.map(item => item.header.id).sort()).toEqual([added.id, changed.id, unchanged.id].sort())
|
||||
expect(Object.fromEntries(TestPersistence.loads)).toEqual({
|
||||
unchanged: 1,
|
||||
changed: 2,
|
||||
deleted: 1,
|
||||
added: 1,
|
||||
})
|
||||
await secondSearch.dispose()
|
||||
await secondPersistence.dispose()
|
||||
|
||||
@@ -409,10 +681,27 @@ describe('SQLite reconciliation and source lifecycle', () => {
|
||||
await expect(second.sessionSearch.searchSessions({ query: 'live' })).resolves.toEqual({ items: [] })
|
||||
await expect(second.sessionSearch.searchSessions({ query: 'persisted' }))
|
||||
.resolves.toMatchObject({ items: [{ header: shared, live: false, persisted: true }] })
|
||||
expect(TestPersistence.loads.get(shared.id)).toBe(1)
|
||||
await searchAgain.dispose()
|
||||
await persistenceAgain.dispose()
|
||||
})
|
||||
|
||||
it('refreshes the stored revision after a mutating load repair', async () => {
|
||||
const durable = header('repair')
|
||||
TestPersistence.reset([{ meta: durable, events: messageEvents('before repair') }])
|
||||
TestPersistence.loadEffect = (entry) => {
|
||||
entry.events = messageEvents('repaired needle')
|
||||
}
|
||||
const ctx = await liveContext()
|
||||
await ctx.plugin(TestPersistence)
|
||||
|
||||
await expect(ctx.sessionSearch.searchSessions({ query: 'repaired' }))
|
||||
.resolves.toMatchObject({ items: [{ header: durable }] })
|
||||
expect(TestPersistence.loads.get(durable.id)).toBe(2)
|
||||
await ctx.sessionSearch.searchSessions({ query: 'repaired' })
|
||||
expect(TestPersistence.loads.get(durable.id)).toBe(2)
|
||||
})
|
||||
|
||||
it('recovers on the next search after source and SQLite transaction failures', async () => {
|
||||
TestPersistence.reset([{ meta: header('durable'), events: messageEvents('durable needle') }])
|
||||
const ctx = await liveContext()
|
||||
@@ -462,15 +751,18 @@ describe('SQLite schema, cancellation, and real persistence integration', () =>
|
||||
|
||||
const foreignPath = await temporaryPath('foreign.db')
|
||||
const foreign = new DatabaseSync(foreignPath)
|
||||
foreign.exec('PRAGMA journal_mode = WAL')
|
||||
foreign.exec('CREATE TABLE canonical(value TEXT)')
|
||||
foreign.exec("INSERT INTO canonical VALUES ('safe')")
|
||||
foreign.close()
|
||||
const foreignCtx = await liveContext({ path: foreignPath })
|
||||
const foreignCtx = await liveContext({ path: foreignPath, journalMode: 'delete' })
|
||||
await expect(foreignCtx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INDEX_FAILED'))
|
||||
const stillForeign = new DatabaseSync(foreignPath)
|
||||
expect(stillForeign.prepare('SELECT value FROM canonical').get()).toEqual({ value: 'safe' })
|
||||
expect(stillForeign.prepare('PRAGMA journal_mode').get()).toEqual({ journal_mode: 'wal' })
|
||||
stillForeign.close()
|
||||
await (foreignCtx.sessionSearch as SessionSearchSqlite).close()
|
||||
|
||||
const otherAppPath = await temporaryPath('other-app.db')
|
||||
const otherApp = new DatabaseSync(otherAppPath)
|
||||
@@ -479,6 +771,25 @@ describe('SQLite schema, cancellation, and real persistence integration', () =>
|
||||
const otherAppCtx = await liveContext({ path: otherAppPath })
|
||||
await expect(otherAppCtx.sessionSearch.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INDEX_FAILED'))
|
||||
await (otherAppCtx.sessionSearch as SessionSearchSqlite).close()
|
||||
})
|
||||
|
||||
it('observes asynchronous open rejection even when no query is made', async () => {
|
||||
const path = await temporaryPath('never-queried.db')
|
||||
const foreign = new DatabaseSync(path)
|
||||
foreign.exec('CREATE TABLE canonical(value TEXT)')
|
||||
foreign.close()
|
||||
const unhandled: unknown[] = []
|
||||
const onUnhandled = (reason: unknown) => { unhandled.push(reason) }
|
||||
process.on('unhandledRejection', onUnhandled)
|
||||
try {
|
||||
const ctx = await liveContext({ path })
|
||||
await new Promise<void>((resolve) => { setImmediate(resolve) })
|
||||
expect(unhandled).toEqual([])
|
||||
await (ctx.sessionSearch as SessionSearchSqlite).close()
|
||||
} finally {
|
||||
process.off('unhandledRejection', onUnhandled)
|
||||
}
|
||||
})
|
||||
|
||||
it('cancels both queued and in-flight source waits without committing them', async () => {
|
||||
@@ -518,7 +829,7 @@ describe('SQLite schema, cancellation, and real persistence integration', () =>
|
||||
releaseBlocking()
|
||||
await expect(blocking).resolves.toEqual({ items: [] })
|
||||
|
||||
TestPersistence.entries.set(SessionId('uncommitted'), {
|
||||
TestPersistence.set({
|
||||
meta: header('uncommitted'),
|
||||
events: messageEvents('durable needle'),
|
||||
})
|
||||
@@ -560,14 +871,38 @@ describe('SQLite schema, cancellation, and real persistence integration', () =>
|
||||
await started
|
||||
const queued = search.searchSessions({ query: 'needle' })
|
||||
const closing = search.close()
|
||||
const repeatedClose = search.close()
|
||||
expect(repeatedClose).toBe(closing)
|
||||
release()
|
||||
|
||||
await expect(accepted).resolves.toEqual({ items: [] })
|
||||
await expect(queued).rejects.toThrow(expectCode('SESSION_QUERY_INDEX_FAILED'))
|
||||
await closing
|
||||
await Promise.all([closing, repeatedClose])
|
||||
await expect(search.searchSessions({ query: 'needle' }))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INDEX_FAILED'))
|
||||
await search.close()
|
||||
expect(search.close()).toBe(closing)
|
||||
})
|
||||
|
||||
it('awaits optional-persistence child-fiber quiescence on disposal', async () => {
|
||||
TestPersistence.reset()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const search = await ctx.plugin(SessionSearchSqlite, { path: ':memory:' })
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
const optional = (ctx.sessionSearch as unknown as {
|
||||
_optionalPersistenceFiber: Fiber
|
||||
})._optionalPersistenceFiber
|
||||
let release!: () => void
|
||||
const cleanup = new Promise<void>((resolve) => { release = resolve })
|
||||
optional.ctx.effect(() => () => cleanup)
|
||||
|
||||
let settled = false
|
||||
const disposing = search.dispose().then(() => { settled = true })
|
||||
await Promise.resolve()
|
||||
expect(settled).toBe(false)
|
||||
release()
|
||||
await disposing
|
||||
await persistence.dispose()
|
||||
})
|
||||
|
||||
it('combines the real SQLite persistence backend with the real search service keylessly', async () => {
|
||||
|
||||
@@ -7,6 +7,7 @@ This is trusted context-wide infrastructure. It performs no caller authorization
|
||||
## Reads
|
||||
|
||||
- `listSessions()` reads current persistence metadata, merges live records with live precedence, and returns cloned records in deterministic newest-first order.
|
||||
- `filterSessions(filters)` applies provider-independent session metadata and availability predicates to that same cloned logical corpus.
|
||||
- `listEvents(sessionId)` loads the live-preferred raw log and classifies each event as `current`, `shadowed`, or `log-only` with the shared `dsh-session` surface fold.
|
||||
- `filterEvents(sessionId, filters)` extracts first-party semantic documents and applies provider-independent metadata and literal-text predicates in ascending seq order.
|
||||
- `readEvent(request)` returns a cloned header, the full target event, and a bounded raw-seq window. `before` and `after` default to zero and may not exceed `readWindowMax`.
|
||||
@@ -21,7 +22,7 @@ The text clause is deliberately independent of FTS providers: caller text is esc
|
||||
|
||||
## Full-text seam
|
||||
|
||||
`SessionSearchService` owns the independent `ctx.sessionSearch` key. `searchSessions(request, exec?)` groups the logical corpus by strongest matching event; `searchEvents(request, exec?)` searches one logical session. Both return opaque cursor pages, accept optional cancellation, and expose snippets without provider-specific numeric scores. Search requests accept only metadata event filters, because literal-text filtering is the scan path described above.
|
||||
`SessionSearchService` owns the independent `ctx.sessionSearch` key. `searchSessions(request, exec?)` groups the logical corpus by strongest matching event; `searchEvents(request, exec?)` searches one logical session. Both return pages whose continuation is an owned branded `SessionSearchCursor`, accept optional cancellation, and expose snippets without provider-specific numeric scores. Search requests accept only metadata event filters, because literal-text filtering is the scan path described above.
|
||||
|
||||
The package has no provider coordinator or registration protocol. A concrete backend owns observation, reconciliation, ranking, cursor generations, and query execution as one lifecycle; the first implementation is [`@deepseek-ai/dsh-session-query-sqlite`](../session-query-sqlite/README.md).
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence": "^0.0.1",
|
||||
@@ -36,6 +37,7 @@
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
/** Live/persisted logical-corpus resolution for session-query. */
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import type { Context, Fiber } from 'cordis'
|
||||
import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type SessionPersistence from '@deepseek-ai/dsh-session-persistence'
|
||||
import type { SessionRecord } from './types.ts'
|
||||
@@ -18,18 +18,19 @@ export interface LogicalSession {
|
||||
/** Resolves a live-preferred corpus against the persistence service mounted now. */
|
||||
export class SessionCorpus {
|
||||
private _persistence: SessionPersistence | undefined
|
||||
private readonly _optionalPersistenceFiber: Fiber
|
||||
|
||||
constructor(private readonly _ctx: Context) {
|
||||
this._optionalPersistenceFiber = _ctx.inject(['sessionPersistence'], (childCtx: Context) => {
|
||||
const service = childCtx.sessionPersistence
|
||||
this._persistence = service
|
||||
childCtx.effect(() => () => {
|
||||
/* v8 ignore next -- a stale optional-service disposer cannot clear a replacement */
|
||||
if (this._persistence === service) this._persistence = undefined
|
||||
}, 'sessionQuery.persistenceBinding')
|
||||
})
|
||||
_ctx.effect(() => {
|
||||
const fiber = _ctx.inject(['sessionPersistence'], (childCtx: Context) => {
|
||||
const service = childCtx.sessionPersistence
|
||||
this._persistence = service
|
||||
childCtx.effect(() => () => {
|
||||
/* v8 ignore next -- a stale optional-service disposer cannot clear a replacement */
|
||||
if (this._persistence === service) this._persistence = undefined
|
||||
}, 'sessionQuery.persistenceBinding')
|
||||
})
|
||||
return () => void fiber.dispose()
|
||||
return () => this._optionalPersistenceFiber.dispose()
|
||||
}, 'sessionQuery.optionalPersistence')
|
||||
}
|
||||
|
||||
|
||||
15
packages/session-query/session-query/src/cursor.ts
Normal file
15
packages/session-query/session-query/src/cursor.ts
Normal file
@@ -0,0 +1,15 @@
|
||||
/** Opaque cursor identity for session-search pagination. */
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
/** Provider-owned opaque continuation token returned by session search. */
|
||||
export type SessionSearchCursor = Branded<'SessionSearchCursor'>
|
||||
|
||||
/**
|
||||
* Brand an encoded provider cursor for the public search contract.
|
||||
* @param value - opaque encoded cursor value.
|
||||
* @returns the same runtime string with session-search cursor identity.
|
||||
*/
|
||||
export function SessionSearchCursor(value: string): SessionSearchCursor {
|
||||
return value as SessionSearchCursor
|
||||
}
|
||||
@@ -1,6 +1,12 @@
|
||||
/** Pure provider-independent predicates for logical sessions and event text. */
|
||||
|
||||
import type { SessionRecord, SessionEventSearchDocument, SessionEventResultFilter, SessionResultFilter, SessionResultRange } from './types.ts'
|
||||
import type {
|
||||
SessionEventResultFilter,
|
||||
SessionEventSearchDocument,
|
||||
SessionRecord,
|
||||
SessionResultFilter,
|
||||
SessionResultRange,
|
||||
} from './types.ts'
|
||||
import { SessionQueryError } from './config.ts'
|
||||
|
||||
/**
|
||||
@@ -31,6 +37,66 @@ export function filterSessionEventDocuments<T extends SessionEventSearchDocument
|
||||
return documents.filter(document => predicates.every(predicate => predicate(document)))
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy and validate logical-session filters before an asynchronous boundary.
|
||||
* @param filters - caller-owned clauses to materialize.
|
||||
* @returns detached validated clauses.
|
||||
*/
|
||||
export function materializeSessionResultFilters(
|
||||
filters: readonly SessionResultFilter[],
|
||||
): SessionResultFilter[] {
|
||||
assertArray(filters)
|
||||
return filters.map((filter) => {
|
||||
switch (filter.kind) {
|
||||
case 'id':
|
||||
return { kind: filter.kind, values: copyStrings(filter.kind, filter.values) }
|
||||
case 'cwd':
|
||||
return { kind: filter.kind, values: copyNullableStrings(filter.kind, filter.values) }
|
||||
case 'created-at':
|
||||
return copyRange(filter.kind, filter)
|
||||
case 'parent':
|
||||
return { kind: filter.kind, values: copyNullableStrings(filter.kind, filter.values) }
|
||||
case 'availability': {
|
||||
const values = copyStrings(filter.kind, filter.values)
|
||||
assertAllowedValues(filter.kind, values, ['live', 'persisted'])
|
||||
return { kind: filter.kind, values }
|
||||
}
|
||||
default:
|
||||
return unknownFilter(filter)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy and validate event filters before an asynchronous boundary.
|
||||
* @param filters - caller-owned clauses to materialize.
|
||||
* @returns detached validated clauses.
|
||||
*/
|
||||
export function materializeSessionEventResultFilters(
|
||||
filters: readonly SessionEventResultFilter[],
|
||||
): SessionEventResultFilter[] {
|
||||
assertArray(filters)
|
||||
return filters.map((filter) => {
|
||||
switch (filter.kind) {
|
||||
case 'seq':
|
||||
case 'time':
|
||||
return copyRange(filter.kind, filter)
|
||||
case 'type':
|
||||
return { kind: filter.kind, values: copyStrings(filter.kind, filter.values) }
|
||||
case 'surface': {
|
||||
const values = copyStrings(filter.kind, filter.values)
|
||||
assertAllowedValues(filter.kind, values, ['current', 'shadowed', 'log-only'])
|
||||
return { kind: filter.kind, values }
|
||||
}
|
||||
case 'text':
|
||||
if (typeof filter.text !== 'string') throw invalidFilter('text filter text must be a string')
|
||||
return { kind: filter.kind, text: filter.text }
|
||||
default:
|
||||
return unknownFilter(filter)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Compile a literal case-insensitive, whitespace-flexible semantic-text match.
|
||||
* @param text - caller-provided literal text.
|
||||
@@ -66,6 +132,8 @@ function sessionPredicate(filter: SessionResultFilter): (record: SessionRecord)
|
||||
case 'availability':
|
||||
assertAllowedValues(filter.kind, filter.values, ['live', 'persisted'])
|
||||
return record => filter.values.some(value => value === 'live' ? record.live : record.persisted)
|
||||
default:
|
||||
return unknownFilter(filter)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -88,9 +156,47 @@ function eventPredicate(filter: SessionEventResultFilter): (document: SessionEve
|
||||
const pattern = compileSessionTextFilter(filter.text)
|
||||
return document => pattern.test(document.text)
|
||||
}
|
||||
default:
|
||||
return unknownFilter(filter)
|
||||
}
|
||||
}
|
||||
|
||||
function copyStrings<T extends string>(name: string, values: readonly T[]): T[] {
|
||||
if (!isRuntimeArray(values) || values.some(value => typeof value !== 'string')) {
|
||||
throw invalidFilter(`${name} filter values must be an array of strings`)
|
||||
}
|
||||
return [...values]
|
||||
}
|
||||
|
||||
function assertArray(value: unknown): void {
|
||||
if (!Array.isArray(value)) throw invalidFilter('filters must be an array')
|
||||
}
|
||||
|
||||
function copyNullableStrings<T extends string>(name: string, values: readonly (T | null)[]): Array<T | null> {
|
||||
if (!isRuntimeArray(values) || values.some(value => value !== null && typeof value !== 'string')) {
|
||||
throw invalidFilter(`${name} filter values must be an array of strings or null`)
|
||||
}
|
||||
return [...values]
|
||||
}
|
||||
|
||||
function copyRange<K extends 'created-at' | 'seq' | 'time'>(
|
||||
kind: K,
|
||||
range: SessionResultRange,
|
||||
): { kind: K } & SessionResultRange {
|
||||
const copy = {
|
||||
kind,
|
||||
...range.from === undefined ? {} : { from: range.from },
|
||||
...range.to === undefined ? {} : { to: range.to },
|
||||
}
|
||||
validateRange(kind, copy)
|
||||
return copy
|
||||
}
|
||||
|
||||
function unknownFilter(filter: never): never {
|
||||
const kind = (filter as { kind?: unknown }).kind
|
||||
throw invalidFilter(`unknown filter kind ${typeof kind === 'string' ? `"${kind}"` : '(missing)'}`)
|
||||
}
|
||||
|
||||
function assertAllowedValues(
|
||||
name: string,
|
||||
values: readonly string[],
|
||||
@@ -125,8 +231,13 @@ function matchesRange(value: number, range: SessionResultRange): boolean {
|
||||
}
|
||||
|
||||
function invalidRange(name: string, detail: string): SessionQueryError {
|
||||
return new SessionQueryError(
|
||||
`session ${name} filter ${detail}`,
|
||||
'SESSION_QUERY_INVALID_FILTER',
|
||||
)
|
||||
return invalidFilter(`${name} filter ${detail}`)
|
||||
}
|
||||
|
||||
function invalidFilter(detail: string): SessionQueryError {
|
||||
return new SessionQueryError(`session ${detail}`, 'SESSION_QUERY_INVALID_FILTER')
|
||||
}
|
||||
|
||||
function isRuntimeArray(value: unknown): boolean {
|
||||
return Array.isArray(value)
|
||||
}
|
||||
|
||||
@@ -16,6 +16,7 @@ import type {
|
||||
SessionEventSearchRequest,
|
||||
SessionEventWindow,
|
||||
SessionRecord,
|
||||
SessionResultFilter,
|
||||
SessionSearchExecContext,
|
||||
SessionSearchHit,
|
||||
SessionSearchPage,
|
||||
@@ -28,14 +29,26 @@ import {
|
||||
} from './config.ts'
|
||||
import { SessionCorpus } from './corpus.ts'
|
||||
import { buildSessionEventRecords, buildSessionEventSearchDocuments } from './documents.ts'
|
||||
import { filterSessionEventDocuments } from './filters.ts'
|
||||
import {
|
||||
filterSessionEventDocuments,
|
||||
filterSessionResults,
|
||||
materializeSessionEventResultFilters,
|
||||
materializeSessionResultFilters,
|
||||
} from './filters.ts'
|
||||
|
||||
export type * from './types.ts'
|
||||
export { SessionSearchCursor } from './cursor.ts'
|
||||
export type { Config, SessionQueryErrorCode } from './config.ts'
|
||||
export { SESSION_QUERY_READ_WINDOW_MAX, SessionQueryError } from './config.ts'
|
||||
export { extractSessionEventText } from './extraction.ts'
|
||||
export { buildSessionEventRecords, buildSessionEventSearchDocuments } from './documents.ts'
|
||||
export { compileSessionTextFilter, filterSessionEventDocuments, filterSessionResults } from './filters.ts'
|
||||
export {
|
||||
compileSessionTextFilter,
|
||||
filterSessionEventDocuments,
|
||||
filterSessionResults,
|
||||
materializeSessionEventResultFilters,
|
||||
materializeSessionResultFilters,
|
||||
} from './filters.ts'
|
||||
export { assertSessionHeadersCompatible } from './sources.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
@@ -109,6 +122,16 @@ export class SessionQueryService extends Service {
|
||||
return this._corpus.listSessions()
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter the complete logical corpus with provider-independent predicates.
|
||||
* @param filters - ANDed session metadata and availability clauses.
|
||||
* @returns matching cloned records in deterministic newest-first order.
|
||||
*/
|
||||
async filterSessions(filters: readonly SessionResultFilter[]): Promise<SessionRecord[]> {
|
||||
const ownedFilters = materializeSessionResultFilters(filters)
|
||||
return this._filterSessions(ownedFilters)
|
||||
}
|
||||
|
||||
/**
|
||||
* List lightweight raw-log event records for one logical session.
|
||||
* @param sessionId - live-preferred session id to read.
|
||||
@@ -128,6 +151,18 @@ export class SessionQueryService extends Service {
|
||||
async filterEvents(
|
||||
sessionId: SessionId,
|
||||
filters: readonly SessionEventResultFilter[],
|
||||
): Promise<SessionEventSearchDocument[]> {
|
||||
const ownedFilters = materializeSessionEventResultFilters(filters)
|
||||
return this._filterEvents(sessionId, ownedFilters)
|
||||
}
|
||||
|
||||
private async _filterSessions(filters: readonly SessionResultFilter[]): Promise<SessionRecord[]> {
|
||||
return filterSessionResults(await this._corpus.listSessions(), filters)
|
||||
}
|
||||
|
||||
private async _filterEvents(
|
||||
sessionId: SessionId,
|
||||
filters: readonly SessionEventResultFilter[],
|
||||
): Promise<SessionEventSearchDocument[]> {
|
||||
const loaded = await this._corpus.load(sessionId)
|
||||
const documents = buildSessionEventSearchDocuments(sessionId, loaded.events)
|
||||
@@ -142,16 +177,27 @@ export class SessionQueryService extends Service {
|
||||
async readEvent(request: SessionEventReadRequest): Promise<SessionEventWindow> {
|
||||
const before = this._readWindow('before', request.before)
|
||||
const after = this._readWindow('after', request.after)
|
||||
const loaded = await this._corpus.load(request.sessionId)
|
||||
const target = loaded.events[request.seq]
|
||||
if (target === undefined || target.seq !== request.seq) {
|
||||
const sessionId = request.sessionId
|
||||
const seq = request.seq
|
||||
return this._readEvent(sessionId, seq, before, after)
|
||||
}
|
||||
|
||||
private async _readEvent(
|
||||
sessionId: SessionId,
|
||||
seq: number,
|
||||
before: number,
|
||||
after: number,
|
||||
): Promise<SessionEventWindow> {
|
||||
const loaded = await this._corpus.load(sessionId)
|
||||
const target = loaded.events[seq]
|
||||
if (target === undefined || target.seq !== seq) {
|
||||
throw new SessionQueryError(
|
||||
`session "${request.sessionId}" has no event at seq ${request.seq}`,
|
||||
`session "${sessionId}" has no event at seq ${seq}`,
|
||||
'SESSION_QUERY_EVENT_NOT_FOUND',
|
||||
)
|
||||
}
|
||||
const startSeq = Math.max(0, request.seq - before)
|
||||
const endSeq = Math.min(loaded.events.length - 1, request.seq + after)
|
||||
const startSeq = Math.max(0, seq - before)
|
||||
const endSeq = Math.min(loaded.events.length - 1, seq + after)
|
||||
return {
|
||||
session: loaded.header,
|
||||
target,
|
||||
|
||||
@@ -5,6 +5,9 @@
|
||||
*/
|
||||
|
||||
import type { SessionEvent, SessionEventType, SessionHeader, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionSearchCursor } from './cursor.ts'
|
||||
|
||||
export type { SessionSearchCursor } from './cursor.ts'
|
||||
|
||||
/** Whether an event is current model context, replaced context, or raw-log-only. */
|
||||
export type SessionEventSurface = 'current' | 'shadowed' | 'log-only'
|
||||
@@ -106,7 +109,7 @@ export interface SessionSearchPage<T> {
|
||||
/** Results for this page in contract-defined order. */
|
||||
items: readonly T[]
|
||||
/** Opaque continuation cursor, absent on the final page. */
|
||||
nextCursor?: string
|
||||
nextCursor?: SessionSearchCursor
|
||||
}
|
||||
|
||||
/** Controls shared by cross-session and within-session search calls. */
|
||||
@@ -126,7 +129,7 @@ export interface SessionSearchRequest {
|
||||
/** Maximum sessions in this page. */
|
||||
limit?: number
|
||||
/** Opaque cursor returned for the identical normalized request. */
|
||||
cursor?: string
|
||||
cursor?: SessionSearchCursor
|
||||
}
|
||||
|
||||
/** Within-session full-text search request. */
|
||||
@@ -140,7 +143,7 @@ export interface SessionEventSearchRequest {
|
||||
/** Maximum events in this page. */
|
||||
limit?: number
|
||||
/** Opaque cursor returned for the identical normalized request. */
|
||||
cursor?: string
|
||||
cursor?: SessionSearchCursor
|
||||
}
|
||||
|
||||
/** One event full-text search hit with a bounded plain-text excerpt. */
|
||||
|
||||
@@ -10,6 +10,8 @@ import SessionQueryService, {
|
||||
extractSessionEventText,
|
||||
filterSessionEventDocuments,
|
||||
filterSessionResults,
|
||||
materializeSessionEventResultFilters,
|
||||
materializeSessionResultFilters,
|
||||
SessionSearchService,
|
||||
type SessionEventSearchHit,
|
||||
type SessionEventSearchRequest,
|
||||
@@ -177,6 +179,31 @@ describe('session-query document and filter helpers', () => {
|
||||
expect(() => buildSessionEventRecords(id, malformed)).toThrow(expectCode('SESSION_QUERY_INVALID_SURFACE'))
|
||||
})
|
||||
|
||||
it('owns filters and rejects malformed runtime filter shapes deterministically', () => {
|
||||
expect(materializeSessionResultFilters([{ kind: 'created-at', to: 2 }]))
|
||||
.toEqual([{ kind: 'created-at', to: 2 }])
|
||||
expect(() => materializeSessionResultFilters('not-an-array' as never))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionResultFilters([{ kind: 'id', values: 'bad' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionResultFilters([{ kind: 'id', values: [1] } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionResultFilters([{ kind: 'cwd', values: 'bad' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionResultFilters([{ kind: 'parent', values: [1] } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionResultFilters([{} as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionEventResultFilters([{ kind: 'text', text: 1 } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => materializeSessionEventResultFilters([{ kind: 'future' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => filterSessionResults([], [{ kind: 'future' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
expect(() => filterSessionEventDocuments([], [{ kind: 'future' } as never]))
|
||||
.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
})
|
||||
|
||||
it('exposes the scan path on the concrete exact-read service', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { Context, type Fiber } from 'cordis'
|
||||
import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader, SessionId as SessionIdType } from '@deepseek-ai/dsh-session'
|
||||
import SessionPersistence from '@deepseek-ai/dsh-session-persistence'
|
||||
import SessionPersistence, { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence'
|
||||
import SessionQueryService, {
|
||||
type SessionEventSurface,
|
||||
type SessionQueryErrorCode,
|
||||
} from '@deepseek-ai/dsh-session-query'
|
||||
|
||||
@@ -59,6 +60,14 @@ class TestPersistence extends SessionPersistence {
|
||||
TestPersistence.afterList?.()
|
||||
return Promise.resolve(headers)
|
||||
}
|
||||
|
||||
|
||||
async listSnapshots() {
|
||||
return [...TestPersistence.entries.values()].map(entry => ({
|
||||
header: structuredClone(entry.meta),
|
||||
revision: SessionPersistenceRevision(`events:${entry.events.length}`),
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
async function liveContext(config: ConstructorParameters<typeof SessionQueryService>[1] = {}): Promise<Context> {
|
||||
@@ -94,6 +103,38 @@ describe('session-query exact reads', () => {
|
||||
expect(older.header.createdAt).toBe(1)
|
||||
})
|
||||
|
||||
it('filters sessions symmetrically and owns mutable filter values immediately', async () => {
|
||||
const durable = header('durable-filter', 1)
|
||||
TestPersistence.reset([{ meta: durable, events: eventLog('durable') }])
|
||||
const ctx = await liveContext()
|
||||
const live = ctx.sessions.create(SessionId('live-filter'), { meta: { createdAt: 2 } })
|
||||
live.append(
|
||||
'user/message',
|
||||
{ content: [{ type: 'text', text: 'live' }], source: { kind: 'user' } },
|
||||
{ surfaceOp: 'append' },
|
||||
)
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
|
||||
const ids = [durable.id]
|
||||
const filtered = ctx.sessionQuery.filterSessions([{ kind: 'id', values: ids }])
|
||||
ids[0] = live.id
|
||||
await expect(filtered).resolves.toEqual([{
|
||||
header: durable,
|
||||
live: false,
|
||||
persisted: true,
|
||||
}])
|
||||
|
||||
const surfaces: SessionEventSurface[] = ['current']
|
||||
const events = ctx.sessionQuery.filterEvents(live.id, [{ kind: 'surface', values: surfaces }])
|
||||
surfaces[0] = 'shadowed'
|
||||
await expect(events).resolves.toMatchObject([{ sessionId: live.id, surface: 'current', text: 'live' }])
|
||||
await expect(ctx.sessionQuery.filterSessions([{ kind: 'future' } as never]))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await expect(ctx.sessionQuery.filterEvents(live.id, [{ kind: 'future' } as never]))
|
||||
.rejects.toThrow(expectCode('SESSION_QUERY_INVALID_FILTER'))
|
||||
await persistence.dispose()
|
||||
})
|
||||
|
||||
it('classifies current, shadowed, and raw-log-only events through foldSurface', async () => {
|
||||
const ctx = await liveContext()
|
||||
const session = ctx.sessions.create(SessionId('surface'))
|
||||
@@ -267,4 +308,26 @@ describe('session-query exact reads', () => {
|
||||
await fiber.dispose()
|
||||
expect(ctx.sessionQuery).toBeUndefined()
|
||||
})
|
||||
|
||||
it('awaits optional-persistence child-fiber quiescence on disposal', async () => {
|
||||
TestPersistence.reset()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const query = await ctx.plugin(SessionQueryService)
|
||||
const persistence = await ctx.plugin(TestPersistence)
|
||||
const optional = (ctx.sessionQuery as unknown as {
|
||||
_corpus: { _optionalPersistenceFiber: Fiber }
|
||||
})._corpus._optionalPersistenceFiber
|
||||
let release!: () => void
|
||||
const cleanup = new Promise<void>((resolve) => { release = resolve })
|
||||
optional.ctx.effect(() => () => cleanup)
|
||||
|
||||
let settled = false
|
||||
const disposing = query.dispose().then(() => { settled = true })
|
||||
await Promise.resolve()
|
||||
expect(settled).toBe(false)
|
||||
release()
|
||||
await disposing
|
||||
await persistence.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -17,6 +17,9 @@
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../util/brand"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user