Merge remote-tracking branch 'origin/master' into codex/truncated-design

# Conflicts:
#	docs/rfc/INDEX.md
This commit is contained in:
Dudu-0223
2026-07-14 11:06:41 +08:00
35 changed files with 1230 additions and 73 deletions

View File

@@ -34,6 +34,7 @@ A harness is one [Cordis](cordis-primer.md) context. Packages contribute service
| `ctx.subagents` | [`subagent/`](../packages/subagent/README.md) | named delegation providers |
| `ctx.workflows` | [`workflow/`](../packages/workflow/README.md) | script-driven multi-agent orchestration |
| `ctx.sessionPersistence` | [`session-persistence/`](../packages/session-persistence/README.md) | durable storage for session logs |
| `ctx.sessionQuery` | [`session-query/`](../packages/session-query/README.md) | live-preferred logical-corpus and exact-event reads |
## Event

View File

@@ -18,12 +18,14 @@ flowchart LR
svc_sessions["ctx.sessions<br/>In-memory session store"]
pkg_agent["agent"]
pkg_session_persistence["session-persistence"]
pkg_session_query["session-query"]
pkg_subagent_inprocess["subagent-inprocess"]
pkg_invariants["invariants"]
svc_sessionPersistence["ctx.sessionPersistence<br/>Durable session persistence seam"]
pkg_session_persistence_jsonl["session-persistence-jsonl"]
pkg_session_persistence_sqlite["session-persistence-sqlite"]
pkg_acp["acp"]
svc_sessionQuery["ctx.sessionQuery<br/>Exact session-history reads"]
pkg_system_prompt["system-prompt"]
svc_systemPrompt["ctx.systemPrompt<br/>System prompt assembly registry"]
pkg_tools["tools"]
@@ -112,6 +114,7 @@ flowchart LR
pkg_session_persistence --> svc_sessionPersistence
pkg_session_persistence_jsonl --> svc_sessionPersistence
pkg_session_persistence_sqlite --> svc_sessionPersistence
pkg_session_query --> svc_sessionQuery
pkg_skill --> svc_skills
pkg_skill_local --> svc_skills
pkg_spill --> svc_spillStore
@@ -152,10 +155,12 @@ flowchart LR
svc_sandbox --> pkg_bash_sandbox
svc_sessionPersistence --> pkg_acp
svc_sessionPersistence --> pkg_agent_loop
svc_sessionPersistence --> pkg_session_query
svc_sessions --> pkg_agent
svc_sessions --> pkg_agent_loop
svc_sessions --> pkg_invariants
svc_sessions --> pkg_session_persistence
svc_sessions --> pkg_session_query
svc_sessions --> pkg_subagent_inprocess
svc_skills --> pkg_tool_skill
svc_spillStore --> pkg_spill_policy
@@ -185,8 +190,9 @@ flowchart LR
| ctx key | Role | Owner | Implementations | Direct consumers | Companion plugins | Note |
| --- | --- | --- | --- | --- | --- | --- |
| `ctx.llm` | `seam` | [`llm`](../packages/llm/llm) | [`llm-deepseek`](../packages/llm/llm-deepseek), [`llm-pi-ai`](../packages/llm/llm-pi-ai), [`llm-replay`](../packages/support/llm-replay) | [`agent-loop`](../packages/core/agent-loop), [`compact-basic`](../packages/compact/compact-basic) | - | Adapters register provider implementations; the loop and compaction call the provider-neutral stream service. |
| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`session-persistence`](../packages/session-persistence/session-persistence), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. |
| `ctx.sessionPersistence` | `seam` | [`session-persistence`](../packages/session-persistence/session-persistence) | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl), [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/ui/acp) | - | Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time. |
| `ctx.sessions` | `core` | [`session`](../packages/core/session) | - | [`agent-loop`](../packages/core/agent-loop), [`agent`](../packages/core/agent), [`session-persistence`](../packages/session-persistence/session-persistence), [`session-query`](../packages/session-query/session-query), [`subagent-inprocess`](../packages/subagent/subagent-inprocess), [`invariants`](../packages/support/invariants) | - | Owns append-only Session instances and emits the durable session event feed. |
| `ctx.sessionPersistence` | `seam` | [`session-persistence`](../packages/session-persistence/session-persistence) | [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl), [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | [`agent-loop`](../packages/core/agent-loop), [`acp`](../packages/ui/acp), [`session-query`](../packages/session-query/session-query) | - | Backends persist the same SessionEvent vocabulary; apps choose a backend at composition time. |
| `ctx.sessionQuery` | `seam` | [`session-query`](../packages/session-query/session-query) | - | - | - | Resolves live and optional persisted logs into one logical corpus for exact reads. |
| `ctx.systemPrompt` | `core` | [`system-prompt`](../packages/core/system-prompt) | - | [`agent-loop`](../packages/core/agent-loop), [`tools`](../packages/core/tools), [`tool-fs`](../packages/fs/tool-fs), [`tool-web`](../packages/web/tool-web) | - | Collects prompt sections and model-facing tool schemas for each step. |
| `ctx.tools` | `core` | [`tools`](../packages/core/tools) | - | [`agent-loop`](../packages/core/agent-loop), [`tool-ask-user`](../packages/ui/tool-ask-user), [`tool-bash`](../packages/bash/tool-bash), [`tool-cordis`](../packages/cordis/tool-cordis), [`tool-fs`](../packages/fs/tool-fs), [`tool-skill`](../packages/skill/tool-skill), [`tool-subagent`](../packages/subagent/tool-subagent), [`tool-todo`](../packages/todo/tool-todo), [`tool-web`](../packages/web/tool-web), [`acp`](../packages/ui/acp) | - | Registers capabilities, owns Code Mode transport, and routes calls through pre-policy, monotonic guards, around dispatch, post-policy, and final-result observation. |
| `ctx.userInteraction` | `seam` | [`user-interaction`](../packages/ui/user-interaction) | [`stdio-agent`](../packages/ui/stdio-agent), [`acp`](../packages/ui/acp) | [`tool-ask-user`](../packages/ui/tool-ask-user), [`stdio-agent`](../packages/ui/stdio-agent), [`acp`](../packages/ui/acp) | - | UI front doors provide the active human-answer provider; tool-ask-user pauses a tool call on the provider-neutral ask() promise. |

View File

@@ -611,6 +611,20 @@ export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist'
Source: [`packages/session-persistence/session-persistence-sqlite/src/index.ts:50`](../packages/session-persistence/session-persistence-sqlite/src/index.ts)
## `@deepseek-ai/dsh-session-query`
Requires: `sessions`
```ts config-catalog
/** Configuration for exact session-query reads. */
export interface Config {
/** Maximum accepted raw read context on either side. Defaults to 50. */
readWindowMax?: number
}
```
Source: [`packages/session-query/session-query/src/config.ts:9`](../packages/session-query/session-query/src/config.ts)
## `@deepseek-ai/dsh-skill`
```ts config-catalog

View File

@@ -216,6 +216,18 @@ Types: [SessionEvent](../core-data-structures/core.md)
Source: [`packages/session-persistence/session-persistence/src/index.ts:102`](../../packages/session-persistence/session-persistence/src/index.ts)
## `ctx.sessionQuery` — `SessionQueryService`
Live-preferred logical-corpus and exact-event read service.
```ts cordis-catalog
listSessions(): Promise<SessionRecord[]>
async listEvents(sessionId: SessionId): Promise<SessionEventRecord[]>
async readEvent(request: SessionEventReadRequest): Promise<SessionEventWindow>
```
Source: [`packages/session-query/session-query/src/index.ts:35`](../../packages/session-query/session-query/src/index.ts)
## `ctx.sessions` — `SessionStore`
In-memory session store (`ctx.sessions`).

View File

@@ -19,6 +19,7 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
| [scope.md](scope.md) | scoped registration identity, dispatch carriers, and the owned `Scope` context |
| [session.md](session.md) | the full `SessionEventMap` variant catalog, `TurnTrigger`/`TurnEndReason`, `deriveMessages()`, the turn-enclosure invariant |
| [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` |
| [session-query.md](session-query.md) | logical session/event records and bounded exact-event reads |
| [system-prompt.md](system-prompt.md) | per-assembly context, tool-provider results, prompt sections, and cooperative assembly |
| [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, and the guarded execution pipeline |
| [user-interaction.md](user-interaction.md) | the UI-backed human question/answer seam: `AskUserQuestionRequest`, answer/options vocabulary, provider API, error taxonomy |

View File

@@ -0,0 +1,69 @@
# Session Query
Exact reads over the live-preferred logical session corpus. The [package contract](../../packages/session-query/session-query) owns source precedence, dynamic optional persistence, cloning, surface classification, bounded windows, and typed failures. Full-text search is a separate proposed SQLite phase.
Source: [`packages/session-query/session-query/src/types.ts`](../../packages/session-query/session-query/src/types.ts)
## Logical records
`SessionRecord` is returned by the cross-corpus list. It exposes source availability independently from the cloned live-preferred header. `SessionEventRecord` is a lightweight raw-log projection; classification uses the same `foldSurface()` transitions as model-history derivation.
```ts type-equiv
export type SessionEventSurface = 'current' | 'shadowed' | 'log-only'
```
```ts type-equiv
export interface SessionRecord {
header: SessionHeader
live: boolean
persisted: boolean
}
```
```ts type-equiv
export interface SessionEventRecord {
sessionId: SessionId
seq: number
type: SessionEventType
time: number
surface: SessionEventSurface
}
```
## Bounded event reads
The request addresses one raw seq and optional neighboring counts. The result carries a `SessionHeader` rather than availability flags so a known live target can remain independent of persistence health.
```ts type-equiv
export interface SessionEventReadRequest {
sessionId: SessionId
seq: number
before?: number
after?: number
}
```
```ts type-equiv
export interface SessionEventWindow {
session: SessionHeader
target: SessionEvent
events: SessionEvent[]
startSeq: number
endSeq: number
}
```
## Errors
The closed code union distinguishes request validation, missing targets, malformed surface logs, optional-backend failure, and contradictory source metadata.
```ts type-equiv
export type SessionQueryErrorCode =
| 'SESSION_QUERY_EVENT_NOT_FOUND'
| 'SESSION_QUERY_INVALID_CONFIG'
| 'SESSION_QUERY_INVALID_SURFACE'
| 'SESSION_QUERY_INVALID_WINDOW'
| 'SESSION_QUERY_PERSISTENCE_FAILED'
| 'SESSION_QUERY_SESSION_NOT_FOUND'
| 'SESSION_QUERY_SOURCE_CONFLICT'
```

View File

@@ -196,6 +196,26 @@ export interface SurfaceNode {
}
```
### `SurfaceFoldReplacement` and `SurfaceFoldResult` — a complete surface replay
`foldSurface(events)` returns detached current nodes together with the actual node seqs shadowed by each declared replacement range. `SurfaceManager` uses the same transition functions for its incremental cache.
```ts type-equiv
export interface SurfaceFoldReplacement {
seq: number
start: number
end: number
shadowedSeqs: number[]
}
```
```ts type-equiv
export interface SurfaceFoldResult {
nodes: SurfaceNode[]
replacements: SurfaceFoldReplacement[]
}
```
## Derived history: `deriveMessages()` and `deriveEventMessage()`
`Session.deriveMessages()` projects the event log into the `Message[]` the model sees — cached (each surface node projected once, when first seen; a surface rewrite rebuilds) and frozen (a fresh array per call over shared, deep-frozen messages, so mutating logged history through a projection is unrepresentable). `deriveEventMessage(event)` is the per-node pure function the fold applies — public so external reconstructors and the dev invariant project a log prefix with exactly the same rules and cannot disagree with the cache. The projection rules:

View File

@@ -89,6 +89,9 @@ flowchart TD
pkg_session_persistence_jsonl["session-persistence-jsonl"]
pkg_session_persistence_sqlite["session-persistence-sqlite"]
end
subgraph group_session_query["packages/session-query"]
pkg_session_query["session-query"]
end
subgraph group_support["packages/support"]
pkg_acp_snapshot["acp-snapshot"]
pkg_invariants["invariants"]
@@ -176,6 +179,9 @@ flowchart TD
pkg_session_persistence_jsonl --> pkg_session_persistence
pkg_session_persistence_sqlite --> pkg_session
pkg_session_persistence_sqlite --> pkg_session_persistence
pkg_session_query --> pkg_llm
pkg_session_query --> pkg_session
pkg_session_query --> pkg_session_persistence
pkg_invariants --> pkg_agent
pkg_invariants --> pkg_llm
pkg_invariants --> pkg_scope
@@ -391,6 +397,7 @@ flowchart TD
| [`hook-protocol`](../packages/hooks/hook-protocol) | `hooks` | [`bash`](../packages/bash/bash), [`session`](../packages/core/session) |
| [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |
| [`session-persistence-sqlite`](../packages/session-persistence/session-persistence-sqlite) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |
| [`session-query`](../packages/session-query/session-query) | `session-query` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |
| [`invariants`](../packages/support/invariants) | `support` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session) |
| [`user-approval`](../packages/ui/user-approval) | `ui` | [`agent`](../packages/core/agent), [`brand`](../packages/util/brand), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`session`](../packages/core/session), [`system-prompt`](../packages/core/system-prompt) |
| [`user-interaction`](../packages/ui/user-interaction) | `ui` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm) |

View File

@@ -11,6 +11,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
| [Pre-tool input rewrite — a consistent design](proposed/feature/2026-06-30-pre-tool-input-rewrite.md) | 2026-06-30 |
| [Claude Code and Codex subagent backends (out-of-process delegation to external coding agents)](proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md) | 2026-07-07 |
| [Interactive side sessions and merge-back](proposed/feature/2026-07-08-interactive-side-sessions.md) | 2026-07-08 |
| [SQLite FTS5 session search](proposed/feature/2026-07-10-sqlite-session-query-provider.md) | 2026-07-10 |
| [Stream workflow progress through tool calls](proposed/feature/2026-07-13-stream-workflow-progress-through-tool-calls.md) | 2026-07-13 |
### Simplification
@@ -76,6 +77,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
| [Repeat-tool-call guard plugin](implemented/feature/2026-07-08-repeat-tool-guard.md) | 2026-07-08 |
| [The self-referential cordis toolset](implemented/feature/2026-07-08-self-referential-cordis-toolset.md) | 2026-07-08 |
| [Bash-backed grep and glob discovery tools](implemented/feature/2026-07-09-bash-backed-grep-glob-discovery.md) | 2026-07-09 |
| [Exact session query service](implemented/feature/2026-07-10-session-query-service.md) | 2026-07-10 |
| [Configure subagent persona, tool visibility, and depth](implemented/feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) | 2026-07-12 |
### Simplification

View File

@@ -0,0 +1,41 @@
# RFC: Exact session query service
Status: implemented
## Problem
Session history exists in two places: current `SessionStore` objects and an optional persistence backend. Consumers that need exact inspection would otherwise duplicate live-versus-persisted precedence, persistence lifecycle handling, raw-event surface classification, and defensive cloning. Durable state can lag the live log between checkpoints, so persistence alone is not a truthful current source.
Full-text search is related but materially larger. Designing provider registration, extraction, synchronization, invalidation, ranking, and cursor contracts before a real backend exists creates two speculative state machines: one in the interface service and another in the eventual database package.
## Decision
`@deepseek-ai/dsh-session-query` owns `ctx.sessionQuery`, a small trusted exact-read service over one logical corpus. It exposes `listSessions()`, `listEvents(sessionId)`, and bounded `readEvent(request)`. It does not expose filters, lineage or provenance traversals, text extractors, search requests, provider registration, or derived-index synchronization.
The service observes the optional `ctx.sessionPersistence` binding dynamically but retains no persisted cache or invalidation listener. Each cross-corpus list asks the active backend for authoritative metadata, then overlays a fresh live-store list. Matching ids become one `SessionRecord`: the live header wins and `live`/`persisted` independently report source availability. Immutable header disagreement is `SESSION_QUERY_SOURCE_CONFLICT`.
An exact target read first checks the live store and snapshots the live header and event log. This path never consults persistence, so a failing durable backend cannot make known live history unreadable. With no live target, the service lists current persistence metadata, proves the id exists, loads it, and rejects a list/load header mismatch. All returned headers and events cross one structured-clone boundary.
## Surface semantics
`dsh-session` exports `foldSurface(events)`, and `SurfaceManager` uses the same transition functions for its incremental cache. The fold returns detached current nodes and each replacement's actual removed seqs. `listEvents()` uses that result to classify every raw event as `current`, `shadowed`, or `log-only`, so inspection cannot disagree with model-history derivation about positional replacement semantics.
`readEvent()` returns the complete target plus raw neighbors by contiguous seq. `before` and `after` default to zero and are independently bounded by `readWindowMax`, default 50. The result carries a cloned `SessionHeader`, not a source-availability record, because determining a live target's persisted flag would violate the guarantee that live exact reads do not depend on persistence health.
## Security boundary
The service is context-wide trusted infrastructure, not an authorization layer. A future model-facing history tool or human UI applies explicit caller/session scope. This phase adds no model-facing tool and changes no transcript or snapshot surface.
## Alternatives considered
- **Put logical-corpus resolution directly in every consumer** — rejected because source precedence, conflicts, optional-service lifecycle, cloning, and surface classification are shared correctness rules.
- **Query only persistence** — rejected because checkpoints can lag the current live log.
- **Cache persisted metadata and listen for writes/removals** — rejected because exact reads can ask the authoritative sources directly, while cache invalidation adds lifecycle and concurrency state before scale requires it.
- **Define a provider-neutral search protocol now** — rejected because no provider consumes it. The first SQLite FTS package should own one reconciliation/transaction state machine; a smaller shared seam can be extracted later only when a second implementation proves the boundary.
- **Include lineage, provenance, and generic filters in phase one** — rejected because no current consumer requires them and canonical logs remain sufficient to add them with evidence later.
## Consequences
Phase one has one source-resolution state variable: the currently mounted persistence service. There are no provider queues, fingerprints, extractor registries, observation generations, or derived index updates. Exact reads remain usable in live-only deployments and deterministic when persistence is present.
Cross-corpus listing and persisted exact reads perform backend I/O on each call. That is deliberate: correctness comes from current authoritative state, and scale-oriented search belongs to the phase-two database. Full-text search is unavailable until that package defines and implements its complete contract.

View File

@@ -0,0 +1,51 @@
# RFC: SQLite FTS5 session search
Status: proposed
## Problem
The exact-read `ctx.sessionQuery` service deliberately has no derived index. Large persisted histories need full-text search without scanning every event on every query, while current live sessions need an overlay newer than the last durability checkpoint. Search also needs concrete ranking, snippets, filters, pagination, cancellation, and rebuild behavior.
Splitting those concerns across a speculative provider coordinator and a database implementation would create two coupled reconciliation state machines. The first real implementation should own the source observation, extraction, SQLite transaction, generation, and query as one lifecycle.
## Proposal
Add `@deepseek-ai/dsh-session-query-sqlite` beside the exact-read package. The package will expose a search service or extend the family with the smallest API required by its actual consumers; phase one does not pre-commit a provider-registration protocol. It will depend on `ctx.sessions` and optional `ctx.sessionPersistence`, own a separate derived SQLite database, and reuse the canonical `foldSurface()` classification.
The implementation owns one serialized reconciliation/DB transaction state machine. A transaction observes authoritative persisted metadata and live snapshots, extracts semantic documents, updates derived tables, advances relevant cursor generations, and executes or enables the corresponding query. No second service maintains parallel fingerprints, dirty flags, live-id sets, or invalidation generations.
Persisted documents survive restarts. Live overrides are connection-local and shadow the persisted rows for the same session, then disappear when the live owner or database closes. The derived database remains separate from canonical persistence so index reset, corruption, tokenizer changes, and schema churn cannot endanger durable conversation logs.
## Search semantics to decide with implementation
The implementation must define both cross-session and within-session scopes from executable use cases. Each searchable event is one document with session metadata, event metadata, surface classification, normalized semantic text, and a bounded plain-text snippet. Session results group by their strongest matching event; numeric backend scores remain private.
Filters compile to parameterized SQL before ranking. Query syntax is treated as data. Ordering includes stable tie fields. Opaque cursors bind to normalized request shape and the smallest relevant generation; unrelated session changes should not invalidate a within-session cursor. Cancellation must stop caller waiting and interrupt SQLite work where the runtime permits.
Tokenizer choice remains an implementation experiment. FTS5 trigram supports substring recall but rejects useful terms shorter than three characters and increases index size; the proposal must benchmark that tradeoff against the default Unicode tokenizer before making it contract.
## Extraction and reconciliation
The package starts with first-party semantic extraction for messages, reasoning, tool calls/results, blocked prompts, context, steering, todos, and error/status detail. Structural events and stream chunks contribute no document. Unknown declaration-merged event/content types remain non-searchable unless a real extension consumer demonstrates the need for a public extractor registry.
Reconciliation may use stable fingerprints to avoid rewriting unchanged persisted sessions, but the database package owns their calculation and storage. It must never report a row current when source observation or extraction failed. Provider-schema mismatch may reset only the derived database; ordinary source changes use transactional upsert/delete. Mounted but unreadable persistence fails affected searches without affecting canonical writes or known live exact reads.
## Alternatives considered
- **Add FTS tables to the canonical persistence database** — rejected because a rebuildable index must not share the authoritative log's schema/reset/failure boundary.
- **Reintroduce phase-one provider coordination** — rejected because there is one planned implementation and no evidence for a stable multi-provider seam.
- **Persist live overrides immediately** — rejected because live events are not canonical until the existing checkpoint commits.
- **Return BM25 scores** — rejected because provider-specific numeric scales are unstable across corpus changes.
## Acceptance criteria
- Restart tests cover unchanged, new, changed, and deleted persisted sessions without rebuilding the whole index.
- Reopening preserves persisted rows and removes live rows; live rows shadow and then reveal their persisted base.
- Tests cover both search scopes, metadata filters, surface defaults, snippets, escaping, deterministic ties, pagination, scoped stale cursors, cancellation, dynamic persistence mount/unmount, and recovery after a failed transaction.
- A schema mismatch resets only the derived database.
- A keyless end-to-end test combines a real persistence backend with the real SQLite search package.
- The RFC is amended to the measured tokenizer and public API actually implemented before moving to `implemented/`.
## Risks
A single owner is simpler but initially less reusable than a provider-neutral seam. That is intentional: a second real backend can reveal what to extract. SQLite runtime differences can affect FTS ranking and snippets, so tests must pin only contract-controlled ordering and presentation. The separate database adds configuration and lifecycle work, but preserves the canonical store's safety boundary.