Merge remote-tracking branch 'origin/master' into codex/rfc-simplify-candidates
This commit is contained in:
Tianyi Cui
@@ -8,6 +8,8 @@ The harness core is deliberately tiny: a handful of abstract services plus one c
|
||||
|
||||
Requirement context: [Coding Harness MVP 需求分析][mvp-doc].
|
||||
|
||||
For a catalog of the **data structures** this architecture moves around — the core vocabulary types, their literal shapes, and the seam types grouped by capability — see [core-data-structures/](core-data-structures/core.md). This document covers behavior; that one covers the types.
|
||||
|
||||
**Contents:** [Layering](#layering) · [Service map](#service-map) · [Capability seams](#capability-seams-interface--implementation--consumer) · [The vocabulary (dsh-llm)](#the-vocabulary-dsh-llm) · [Event-sourced sessions](#event-sourced-sessions-dsh-session) · [Prompt assembly](#prompt-assembly-dsh-system-prompt) · [Tool pipeline](#tool-pipeline-dsh-tools) · [Agents and the loop](#agents-dsh-agent-and-the-loop-dsh-agent-loop) ([lifecycle](#loop-lifecycle-session--turn--step), [event taxonomy](#event-taxonomy), [waterfall semantics](#cordis-waterfall-semantics-important)) · [Plugin sanity checklist](#plugin-sanity-checklist) · [Extension cookbook](#extension-cookbook) · [Deferred work](#deferred-work-todo)
|
||||
|
||||
[microkernel-doc]: https://trtgsjkv6r.feishu.cn/wiki/VS9Lw1kQki6mDJk2UHocyuphnsc
|
||||
@@ -54,6 +56,8 @@ Dependency rule: plugins depend on interface packages, never on `dsh-agent-loop`
|
||||
|
||||
All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically.
|
||||
|
||||
For each service's full public interface (every method signature, generated from source), plus the inherited cordis-core/loader/hmr/timer surface a plugin also sees, see the `## Services` section of [cordis-catalog/events-and-services.md](cordis-catalog/events-and-services.md). This table is the at-a-glance role summary; that catalog is the exhaustive reference.
|
||||
|
||||
## Capability seams: interface / implementation / consumer
|
||||
|
||||
Swappable capabilities are split into **three packages** so each part evolves independently. The bash capability is the template:
|
||||
@@ -165,26 +169,7 @@ A failure that happens once the turn is already closed has no in-turn position f
|
||||
|
||||
### Event taxonomy
|
||||
|
||||
The `agent/*` events are declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package); each other service declares its own events (`tools/*`, `llm/*`, `system-prompt/*`, `session/*`). The table below is CI-verified against the `interface Events` declarations in source (`scripts/verify-event-taxonomy.ts`).
|
||||
|
||||
| Event | Mode | Purpose |
|
||||
|---|---|---|
|
||||
| `agent/created` / `agent/disposed` / `agent/status` / `agent/queued` | emit | lifecycle + inbox notifications |
|
||||
| `agent/turn-start` / `agent/turn-end` / `agent/step-start` / `agent/step-end` | emit | boundaries |
|
||||
| `agent/request` | **waterfall** | mutate the final `GenerateOptions` before the model call |
|
||||
| `agent/stream-chunk` | emit | token-level UI/log feed |
|
||||
| `agent/step-result` | **waterfall** | post-process the assistant message before tool dispatch |
|
||||
| `agent/steering` | emit | steering content injected |
|
||||
| `agent/turn-continuation` | **waterfall** | override the continue/stop decision |
|
||||
| `agent/error` | emit | step/turn errors |
|
||||
| `tools/execute` (dsh-tools) | **waterfall** | wrap/veto/sandbox tool execution |
|
||||
| `tools/change` (dsh-tools) | emit | a tool was registered/unregistered |
|
||||
| `llm/stream` / `llm/generate` (dsh-llm) | **waterfall** | model-call interception |
|
||||
| `llm/adapter-change` (dsh-llm) | emit | an adapter was registered/unregistered |
|
||||
| `system-prompt/assemble` (dsh-system-prompt) | **waterfall** | mutate the assembly |
|
||||
| `system-prompt/change` (dsh-system-prompt) | emit | a section/tool-provider changed |
|
||||
| `session/created` / `session/event` (dsh-session) | emit | session lifecycle + log feed |
|
||||
| `session/flush` (dsh-session) | parallel (awaited) | durability checkpoint |
|
||||
The `agent/*` events are declared in `@deepseek-ai/dsh-agent` (so nothing depends on the loop package); each other service declares its own events (`tools/*`, `llm/*`, `system-prompt/*`, `session/*`). The full catalog — every event's exact signature, dispatch mode, and prose — is **generated from source** and lives in [cordis-catalog/events-and-services.md](cordis-catalog/events-and-services.md) (the `## Events` section), alongside the `ctx.<key>` service interfaces. That file is regenerated by `scripts/gen-cordis-catalog.ts` and frozen by the `verify-cordis-catalog` freshness gate (part of `doc-sync`), so it cannot drift from the `interface Events` declarations.
|
||||
|
||||
### Cordis waterfall semantics (important)
|
||||
|
||||
|
||||
481
docs/cordis-catalog/events-and-services.md
Normal file
481
docs/cordis-catalog/events-and-services.md
Normal file
@@ -0,0 +1,481 @@
|
||||
<!-- Generated by scripts/gen-cordis-catalog.ts — do not edit by hand.
|
||||
Run `pnpm run gen-cordis-catalog` to regenerate. -->
|
||||
|
||||
# Cordis Events & Services Catalog
|
||||
|
||||
An index reference to the **wiring** a plugin author works against: every cordis event you can listen to (exact signature + dispatch mode) and every `ctx.<key>` service you can call (exact public interface). It complements [core-data-structures/](../core-data-structures/core.md), which catalogs the *data structures* these signatures move around — this page is the verbs, that page is the nouns.
|
||||
|
||||
This file is GENERATED from source (`scripts/gen-cordis-catalog.ts`) and verified fresh by `pnpm run verify-cordis-catalog` (part of `doc-sync`) — do not edit it by hand. Signature blocks use a `ts cordis-catalog` fence (skipped by doc-typecheck, since a bare signature is not standalone-compilable). Type names in a signature link to the page that documents them.
|
||||
|
||||
The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary this repo owns. The **inherited tier** at the end is the cordis-core + loader/hmr/timer surface a plugin also sees — pinned vendor source, summarized tersely.
|
||||
|
||||
## Events
|
||||
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 24 events across 5 scopes.
|
||||
|
||||
### `agent/*`
|
||||
|
||||
#### `agent/created` — emit
|
||||
|
||||
An agent was registered in the AgentRegistry and is ready to receive messages.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/created'(agent: Agent): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:140`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/disposed` — emit
|
||||
|
||||
An agent was disposed and removed from the registry; its fiber and any in-flight turn have been torn down.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/disposed'(agent: Agent): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:146`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/error` — emit
|
||||
|
||||
A step or turn errored. The loop reports a failure here (plus the logger) even when the error has no in-turn position for a session `error` event.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/error'(agent: Agent, turn: number, step: number, error: Error): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:223`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/queued` — emit
|
||||
|
||||
A message entered the agent's inbox (queued or steering). `source` is the resolved source (defaults applied), not the caller's raw options.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/queued'(agent: Agent, content: ContentBlock[], info: { source: MessageSource; steering: boolean }): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:159`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/request` — waterfall
|
||||
|
||||
Waterfall: mutate the fully-assembled GenerateOptions before the model call (hooks, compaction, model switching, tool filtering, …). Call `next()` to delegate, or return without it to short-circuit.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/request'(agent: Agent, turn: number, step: number, options: GenerateOptions, next: () => Promise<GenerateOptions>): Promise<GenerateOptions>
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:192`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/status` — emit
|
||||
|
||||
Agent status changed (`idle` ⇄ `running`, or → `disposed`). Drive lifecycle off this transition, never off a status you just requested — `send()` does not flip status to `running` before it returns.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/status'(agent: Agent, status: AgentStatus): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:153`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/steering` — emit
|
||||
|
||||
Steering content was injected into a running turn.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/steering'(agent: Agent, turn: number, content: ContentBlock[], source: MessageSource): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:217`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/step-end` — emit
|
||||
|
||||
A step ended.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/step-end'(agent: Agent, turn: number, step: number): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:183`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/step-result` — waterfall
|
||||
|
||||
Waterfall: post-process the assembled assistant Message before tool dispatch (validation, content rewriting, …).
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/step-result'(agent: Agent, turn: number, step: number, message: Message, next: () => Promise<Message>): Promise<Message>
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:198`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/step-start` — emit
|
||||
|
||||
A step (one model call plus its tool dispatch) began. `step` is 1-based within the turn; a turn runs one or more steps.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/step-start'(agent: Agent, turn: number, step: number): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:178`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/stream-chunk` — emit
|
||||
|
||||
A raw StreamChunk arrived from the model (token-level UI/log feed).
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/stream-chunk'(agent: Agent, turn: number, step: number, chunk: StreamChunk): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:212`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/turn-continuation` — waterfall
|
||||
|
||||
Waterfall: override the turn-continuation decision. The default (computed by the loop) is `hadToolCalls || steeringInjected`. Listeners can force-continue (/goal, /loop) or force-stop (budget guards).
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/turn-continuation'(agent: Agent, turn: number, defaultDecision: boolean, next: () => Promise<boolean>): Promise<boolean>
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:205`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/turn-end` — emit
|
||||
|
||||
A turn ended. `reason` distinguishes a clean stop from a truncated or aborted one (`completed` | `aborted` | `error` | `disposed` | `max-tokens`).
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/turn-end'(agent: Agent, turn: number, reason: TurnEndReason): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [TurnEndReason](../core-data-structures/session.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:172`](../../packages/agent/src/types.ts)
|
||||
|
||||
#### `agent/turn-start` — emit
|
||||
|
||||
A turn began. `turn` is the 1-based turn number within the session.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/turn-start'(agent: Agent, turn: number): void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/types.ts:166`](../../packages/agent/src/types.ts)
|
||||
|
||||
### `llm/*`
|
||||
|
||||
#### `llm/adapter-change` — emit
|
||||
|
||||
An adapter was registered or unregistered (the model→adapter map changed).
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/adapter-change'(): void
|
||||
```
|
||||
|
||||
Source: [`packages/llm/src/index.ts:43`](../../packages/llm/src/index.ts)
|
||||
|
||||
#### `llm/generate` — waterfall
|
||||
|
||||
Waterfall around every non-streaming model call. Bound to the LlmService; call `next()` to delegate to the adapter.
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/generate'(this: LlmService, options: GenerateOptions, next: () => Promise<GenerateResult>): Promise<GenerateResult>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [GenerateResult](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/llm/src/index.ts:38`](../../packages/llm/src/index.ts)
|
||||
|
||||
#### `llm/stream` — waterfall
|
||||
|
||||
Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
|
||||
```
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/src/index.ts:32`](../../packages/llm/src/index.ts)
|
||||
|
||||
### `session/*`
|
||||
|
||||
#### `session/created` — emit
|
||||
|
||||
A session was created in the store.
|
||||
|
||||
```ts cordis-catalog
|
||||
'session/created'(session: Session): void
|
||||
```
|
||||
|
||||
Source: [`packages/session/src/index.ts:30`](../../packages/session/src/index.ts)
|
||||
|
||||
#### `session/event` — emit
|
||||
|
||||
An event was appended to a session log (sync, fire-and-forget). This is the per-append feed a UI or invariant plugin tails.
|
||||
|
||||
```ts cordis-catalog
|
||||
'session/event'(session: Session, event: SessionEvent): void
|
||||
```
|
||||
|
||||
Types: [SessionEvent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/session/src/index.ts:36`](../../packages/session/src/index.ts)
|
||||
|
||||
#### `session/flush` — parallel
|
||||
|
||||
Awaited durability checkpoint. The agent loop awaits `ctx.parallel('session/flush', session)` at every turn end; persistence plugins (JSONL, SQLite) drain their write-behind buffers here and on fiber dispose. Awaited (parallel), not a waterfall: every listener runs and the loop waits for all of them, but none can veto.
|
||||
|
||||
```ts cordis-catalog
|
||||
'session/flush'(session: Session): Promise<void> | void
|
||||
```
|
||||
|
||||
Source: [`packages/session/src/index.ts:45`](../../packages/session/src/index.ts)
|
||||
|
||||
### `system-prompt/*`
|
||||
|
||||
#### `system-prompt/assemble` — waterfall
|
||||
|
||||
Waterfall around prompt assembly — mutate or extend the PromptAssembly (sections + tool schemas) before it is rendered. Bound to the SystemPrompt service; call `next()` to delegate.
|
||||
|
||||
```ts cordis-catalog
|
||||
'system-prompt/assemble'(this: SystemPrompt, assembly: PromptAssembly, next: () => Promise<PromptAssembly>): Promise<PromptAssembly>
|
||||
```
|
||||
|
||||
Source: [`packages/system-prompt/src/index.ts:24`](../../packages/system-prompt/src/index.ts)
|
||||
|
||||
#### `system-prompt/change` — emit
|
||||
|
||||
A section or tool provider was registered or unregistered (the assembly inputs changed).
|
||||
|
||||
```ts cordis-catalog
|
||||
'system-prompt/change'(): void
|
||||
```
|
||||
|
||||
Source: [`packages/system-prompt/src/index.ts:30`](../../packages/system-prompt/src/index.ts)
|
||||
|
||||
### `tools/*`
|
||||
|
||||
#### `tools/change` — emit
|
||||
|
||||
A tool was registered or unregistered (the available tool set changed).
|
||||
|
||||
```ts cordis-catalog
|
||||
'tools/change'(): void
|
||||
```
|
||||
|
||||
Source: [`packages/tools/src/index.ts:48`](../../packages/tools/src/index.ts)
|
||||
|
||||
#### `tools/execute` — waterfall
|
||||
|
||||
Waterfall around every tool execution — the single seam where sandbox, permission, hook, and plan-mode plugins wrap or veto a call. Listeners receive `(exec, next)`: call `next()` to proceed (possibly around your own logic), or return a ToolExecutionResult without calling `next()` to short-circuit (veto).
|
||||
|
||||
```ts cordis-catalog
|
||||
'tools/execute'(this: ToolRegistry, exec: ToolExecution, next: () => Promise<ToolExecutionResult>): Promise<ToolExecutionResult>
|
||||
```
|
||||
|
||||
Types: [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md)
|
||||
|
||||
Source: [`packages/tools/src/index.ts:43`](../../packages/tools/src/index.ts)
|
||||
|
||||
## Services
|
||||
|
||||
The 8 `ctx.<key>` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against.
|
||||
|
||||
### `ctx.agentLoop` — `AgentLoop`
|
||||
|
||||
The agent-loop plugin (`ctx.agentLoop`): creates ReactLoopAgents, runs their loops, and registers them in `ctx.agents`. Also implements the AgentFactory seam, so plugins create/resume agents through `ctx.agents` (the interface) without depending on this concrete package.
|
||||
|
||||
The loop itself is deliberately thin — every behavior beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy declared in @deepseek-ai/dsh-agent.
|
||||
|
||||
```ts cordis-catalog
|
||||
create(id: string, options: AgentOptions = {}): ReactLoopAgent
|
||||
createAgent(options: CreateAgentOptions): AgentHandle
|
||||
async resume(options: ResumeAgentOptions): Promise<AgentHandle>
|
||||
```
|
||||
|
||||
Source: [`packages/agent-loop/src/index.ts:60`](../../packages/agent-loop/src/index.ts)
|
||||
|
||||
### `ctx.agents` — `AgentRegistry`
|
||||
|
||||
Agent registry (`ctx.agents`): tracks live agents so UI, hook, and orchestrator plugins can find them without depending on the concrete loop package. Agent *creation* is provided by whichever plugin implements the AgentFactory (phase 1: `@deepseek-ai/dsh-agent-loop`), registered via setFactory.
|
||||
|
||||
```ts cordis-catalog
|
||||
setFactory(factory: AgentFactory): () => void
|
||||
create(options: CreateAgentOptions): AgentHandle
|
||||
async resume(options: ResumeAgentOptions): Promise<AgentHandle>
|
||||
register(agent: Agent): () => void
|
||||
get(id: string): Agent | undefined
|
||||
list(): Agent[]
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/agent/src/index.ts:105`](../../packages/agent/src/index.ts)
|
||||
|
||||
### `ctx.bash` — `BashExecutor` (abstract seam)
|
||||
|
||||
Abstract bash execution service. Subclass, implement the abstract methods, and load the subclass as a plugin — it registers as `ctx.bash` (one implementation per context; loading a second throws, which is cordis' standard duplicate-service behavior).
|
||||
|
||||
Semantics every implementation must honor:
|
||||
|
||||
- run REJECTS only for infrastructure failures (unusable workdir, missing shell, pre-aborted signal). Nonzero exits, timeout kills, and abort kills RESOLVE with a descriptive BashRunResult — reporting a failed command is the tool layer's job, not an exception.
|
||||
- start returns immediately; no timeout applies to background tasks (callers stop them via kill or the spec's AbortSignal). Completion must fire the onTaskDone listeners exactly once per task, and must NOT fire after the service is disposed.
|
||||
- readOutput is incremental: consecutive reads never re-deliver output. Implementations bound their buffers; reads that lost data flag `lossy` and point at full-stream spill files when available.
|
||||
- Disposal kills every running task and awaits their exit (no orphan processes survive `fiber.dispose()`).
|
||||
|
||||
```ts cordis-catalog
|
||||
abstract resolve(request: BashExecRequest): BashExecSpec
|
||||
abstract run(spec: BashExecSpec): Promise<BashRunResult>
|
||||
abstract start(spec: BashExecSpec): BashTask
|
||||
abstract get(id: string): BashTask | undefined
|
||||
abstract ownerOf(id: string): string | undefined
|
||||
abstract list(): BashTask[]
|
||||
abstract readOutput(id: string): BashTaskRead
|
||||
abstract kill(id: string): boolean
|
||||
onTaskDone(listener: BashTaskListener): () => void
|
||||
```
|
||||
|
||||
Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../core-data-structures/bash.md) · [BashRunResult](../core-data-structures/bash.md) · [BashTask](../core-data-structures/bash.md) · [BashTaskRead](../core-data-structures/bash.md)
|
||||
|
||||
Source: [`packages/bash/src/index.ts:58`](../../packages/bash/src/index.ts)
|
||||
|
||||
### `ctx.llm` — `LlmService`
|
||||
|
||||
The abstract `llm` service: an adapter registry plus streaming / non-streaming call surfaces, both interceptable via waterfall events.
|
||||
|
||||
```ts cordis-catalog
|
||||
registerAdapter(models: string[], adapter: LlmAdapter): () => void
|
||||
models(): string[]
|
||||
stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
async * streamBlocks(options: GenerateOptions): AsyncIterable<ContentBlock>
|
||||
generate(options: GenerateOptions): Promise<GenerateResult>
|
||||
```
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md) · [GenerateResult](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/src/index.ts:81`](../../packages/llm/src/index.ts)
|
||||
|
||||
### `ctx.sessionPersistence` — `SessionPersistence` (abstract seam)
|
||||
|
||||
Abstract durable session-persistence service. Subclass, implement the abstract methods, and load the subclass as a plugin — it registers as `ctx.sessionPersistence` (one implementation per context; loading a second throws, cordis' standard duplicate-service behavior).
|
||||
|
||||
Contracts every implementation MUST honor (a DB backend asserts them inside a transaction; a file backend appends at EOF):
|
||||
|
||||
- **Append-only; a crashed turn is closed, not truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. A crash can leave an unclosed final turn whose events are real (and possibly large); load preserves them and closes the orphaned turn with synthetic boundary events (see load). Only a never-fully-written torn tail fragment is discarded.
|
||||
- **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`. load rejects a parse error or a `seq` gap in the COMMITTED region (unloadable); append's first event `seq` MUST equal the backend's stored next-seq (after `load` has balanced any interrupted turn).
|
||||
- **JSON-serializable data.** `SessionEventMap` is merge-extensible and `event.data` is typed only as `SessionEventMap[K]`, so append REJECTS non-JSON-serializable data with an error naming the offending event type. A backend snapshots (serializes/clones) each event when it buffers, since `session.events` hands out the live mutable object.
|
||||
- **Durability.** append returns only once the batch is durable (the file backend fsyncs; a DB commits). create MAY defer the physical write until the first append (lazy materialization).
|
||||
|
||||
```ts cordis-catalog
|
||||
abstract create(meta: SessionHeader): Promise<void>
|
||||
abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
|
||||
abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>
|
||||
abstract list(): Promise<SessionHeader[]>
|
||||
abstract has(id: SessionId): Promise<boolean>
|
||||
abstract delete(id: SessionId): Promise<void>
|
||||
```
|
||||
|
||||
Types: [SessionEvent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/session-persistence/src/index.ts:98`](../../packages/session-persistence/src/index.ts)
|
||||
|
||||
### `ctx.sessions` — `SessionStore`
|
||||
|
||||
In-memory session store (`ctx.sessions`).
|
||||
|
||||
Persistence is intentionally not implemented here — persistence plugins subscribe to `session/event` and flush on `session/flush` / dispose.
|
||||
|
||||
```ts cordis-catalog
|
||||
create(id?: string, options?: CreateSessionOptions): Session
|
||||
prepare(id?: string, options?: CreateSessionOptions): Session
|
||||
enter(session: Session): () => void
|
||||
announce(session: Session): void
|
||||
get(id: string): Session | undefined
|
||||
list(): Session[]
|
||||
```
|
||||
|
||||
Source: [`packages/session/src/index.ts:222`](../../packages/session/src/index.ts)
|
||||
|
||||
### `ctx.systemPrompt` — `SystemPrompt`
|
||||
|
||||
Registry service (`ctx.systemPrompt`): plugins contribute ordered text sections and tool-schema providers; the agent loop calls `assemble()` once per step.
|
||||
|
||||
```ts cordis-catalog
|
||||
section(section: PromptSection): () => void
|
||||
tools(provider: () => ToolSchema[]): () => void
|
||||
assemble(): Promise<PromptAssembly>
|
||||
```
|
||||
|
||||
Source: [`packages/system-prompt/src/index.ts:71`](../../packages/system-prompt/src/index.ts)
|
||||
|
||||
### `ctx.tools` — `ToolRegistry`
|
||||
|
||||
Tool registry (`ctx.tools`): tool plugins register definitions; the agent loop executes calls through the `tools/execute` waterfall. The registry contributes its schemas into the system-prompt assembly.
|
||||
|
||||
```ts cordis-catalog
|
||||
register(definition: ToolDefinition): () => void
|
||||
get(name: string): ToolDefinition | undefined
|
||||
schemas(): ToolSchema[]
|
||||
async execute(exec: ToolExecution): Promise<ToolExecutionResult>
|
||||
```
|
||||
|
||||
Types: [ToolDefinition](../core-data-structures/tools.md) · [ToolExecution](../core-data-structures/tools.md) · [ToolExecutionResult](../core-data-structures/tools.md)
|
||||
|
||||
Source: [`packages/tools/src/index.ts:277`](../../packages/tools/src/index.ts)
|
||||
|
||||
## Inherited tier (cordis core + loader/hmr/timer)
|
||||
|
||||
The framework surface every plugin inherits, beyond the harness vocabulary above. This is pinned vendor source ([vendoring policy](../../vendor/README.md)); it is summarized here so the catalog is a complete picture of what `ctx` and the event bus offer, without elevating framework internals to the harness tier's prominence.
|
||||
|
||||
### Inherited events
|
||||
|
||||
- `internal/plugin` — A plugin fiber was created. ([`vendor/cordis/src/events.ts:197`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/status` — A fiber changed lifecycle state. ([`vendor/cordis/src/events.ts:198`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/service` — Interception hook for a service binding (no core producer). ([`vendor/cordis/src/events.ts:199`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/update` — Waterfall: a fiber config update is being applied. ([`vendor/cordis/src/events.ts:200`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/get` — Waterfall: a service is being read from the store. ([`vendor/cordis/src/events.ts:201`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/set` — Waterfall: a service is being written to the store. ([`vendor/cordis/src/events.ts:202`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/listener` — A listener was registered. ([`vendor/cordis/src/events.ts:203`](../../vendor/cordis/src/events.ts))
|
||||
- `internal/dispatch` — An event is being dispatched to listeners. ([`vendor/cordis/src/events.ts:204`](../../vendor/cordis/src/events.ts))
|
||||
- `hmr/change` — A watched source file changed on disk. ([`vendor/hmr/src/index.ts:20`](../../vendor/hmr/src/index.ts))
|
||||
- `hmr/reload` — Plugins are being reloaded after a change. ([`vendor/hmr/src/index.ts:21`](../../vendor/hmr/src/index.ts))
|
||||
- `exit` — The process is exiting on a signal. ([`vendor/loader/src/index.ts:23`](../../vendor/loader/src/index.ts))
|
||||
- `loader/config-update` — The loader config tree changed. ([`vendor/loader/src/index.ts:24`](../../vendor/loader/src/index.ts))
|
||||
- `loader/entry-init` — A config entry is being initialized. ([`vendor/loader/src/index.ts:25`](../../vendor/loader/src/index.ts))
|
||||
- `loader/partial-dispose` — An entry is being partially disposed on reload. ([`vendor/loader/src/index.ts:26`](../../vendor/loader/src/index.ts))
|
||||
- `loader/patch-context` — A context is being patched during a reload. ([`vendor/loader/src/index.ts:27`](../../vendor/loader/src/index.ts))
|
||||
|
||||
### Inherited `ctx` members
|
||||
|
||||
- `ctx.on / ctx.once` — Register an event listener (disposable). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts))
|
||||
- `ctx.emit / ctx.parallel / ctx.serial / ctx.bail / ctx.waterfall` — Dispatch an event (sync / awaited / first-non-nullish / veto-chain). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts))
|
||||
- `ctx.plugin / ctx.inject` — Load a plugin / declare required services. ([`vendor/cordis/src/registry.ts:144`](../../vendor/cordis/src/registry.ts))
|
||||
- `ctx.effect` — Register a disposable side effect tied to the fiber. ([`vendor/cordis/src/fiber.ts:9`](../../vendor/cordis/src/fiber.ts))
|
||||
- `ctx.get / ctx.set / ctx.provide / ctx.accessor / ctx.mixin` — Low-level service-store access and binding. ([`vendor/cordis/src/reflect.ts:7`](../../vendor/cordis/src/reflect.ts))
|
||||
- `ctx.extend / ctx.isolate / ctx.intercept` — Derive a child context (scoped services / isolation / interception). ([`vendor/cordis/src/context.ts:35`](../../vendor/cordis/src/context.ts))
|
||||
- `ctx.root / ctx.scope / ctx.fiber / ctx.registry / ctx.reflect / ctx.events / ctx.logger` — Ambient handles onto the running context graph. ([`vendor/cordis/src/context.ts:16`](../../vendor/cordis/src/context.ts))
|
||||
- `ctx.timer (+ interval / timeout / throttle / debounce / setTimeout / setInterval)` — Disposable timer helpers. The `timer` key is provided at runtime; the six helpers are mixed onto ctx directly (declared via Pick). ([`vendor/timer/src/index.ts:4`](../../vendor/timer/src/index.ts))
|
||||
- `ctx.loader` — The config Loader that booted the app (present under the loader). ([`vendor/loader/src/index.ts:30`](../../vendor/loader/src/index.ts))
|
||||
- `ctx.hmr` — The hot-module-reload watcher (present under the hmr plugin). ([`vendor/hmr/src/index.ts:15`](../../vendor/hmr/src/index.ts))
|
||||
123
docs/core-data-structures/bash.md
Normal file
123
docs/core-data-structures/bash.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Bash Executor
|
||||
|
||||
The bash execution seam — the canonical [capability seam](../rfc/implemented/2026-06-13-capability-seams.md) example, split across three packages: interface ([dsh-bash](../../packages/bash), `ctx.bash`), implementation ([dsh-bash-local](../../packages/bash-local), local subprocesses), and consumer ([dsh-tool-bash](../../packages/tool-bash), the `bash`/`bash_output`/`bash_kill` tool schemas). Bash is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A sandboxed, containerized, or remote backend is a sibling package implementing the same interface.
|
||||
|
||||
Source: [`packages/bash/src/types.ts`](../../packages/bash/src/types.ts)
|
||||
|
||||
## Request vs. spec: the `resolve()` split
|
||||
|
||||
The seam separates the **model-/plugin-facing request** (optional `workdir`/`timeoutMs`, filled from config) from the **fully-resolved spec** the executor acts on (those fields required). The tool layer calls `ctx.bash.resolve(request)` between them — this is the repo's "explicit > implicit at package seams" rule made concrete: the reader of a `BashExecSpec` never wonders where the working directory came from.
|
||||
|
||||
```ts type-equiv
|
||||
interface BashExecRequest {
|
||||
command: string
|
||||
/** Working directory override (default: implementation-configured). */
|
||||
workdir?: string | undefined
|
||||
/** Timeout override in milliseconds (implementations cap it). */
|
||||
timeoutMs?: number | undefined
|
||||
/** Abort signal — implementations kill the command when it fires. */
|
||||
signal?: AbortSignal | undefined
|
||||
/**
|
||||
* Opaque OWNER token for a background task — the consumer's isolation key
|
||||
* (the tool layer passes the owning agent's `session.header.id`). The
|
||||
* executor stores it on the task and exposes it via {@link BashExecutor.ownerOf};
|
||||
* the executor itself NEVER interprets it (no access policy lives in the
|
||||
* seam — that is the consumer's job). Absent for foreground runs and for an
|
||||
* ownerless background start (a non-agent caller).
|
||||
*/
|
||||
owner?: string | undefined
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
interface BashExecSpec {
|
||||
command: string
|
||||
workdir: string
|
||||
timeoutMs: number
|
||||
/** Abort signal — implementations kill the command when it fires. */
|
||||
signal?: AbortSignal | undefined
|
||||
/**
|
||||
* Opaque owner token, REQUIRED-but-nullable (mirrors `workdir`/`timeoutMs`
|
||||
* being required on the resolved spec): {@link BashExecutor.resolve} carries
|
||||
* the request's `owner` through, defaulting a missing one to `undefined`. A
|
||||
* required field makes a forgotten owner a VISIBLE `undefined` rather than a
|
||||
* silently-absent property that yields an unowned (cross-session-readable)
|
||||
* task. `start()` stores it; `run()` (foreground) ignores it.
|
||||
*/
|
||||
owner: string | undefined
|
||||
}
|
||||
```
|
||||
|
||||
The `owner` token is the isolation key: the executor stores it but never interprets it (access policy is the consumer's job), so a background task started by one agent isn't readable cross-session. A required-but-nullable field makes a forgotten owner a visible `undefined` rather than a silently-unowned task.
|
||||
|
||||
## Foreground runs: `BashRunResult`
|
||||
|
||||
The outcome of one completed (or killed) foreground run. Orthogonal outcomes are reported **independently** — a process can both time out AND exit 0 because it trapped the signal — so `timedOut`, `aborted`, `signal`, and `exitCode` are each their own field; a caller never reads a cut-short run as a clean success.
|
||||
|
||||
```ts type-equiv
|
||||
interface BashRunResult {
|
||||
/** Exit code; null when the process died from a signal. */
|
||||
exitCode: number | null
|
||||
/** Terminating signal (e.g. 'SIGTERM'); null on normal exit. */
|
||||
signal: NodeJS.Signals | null
|
||||
/** True when the executor's own timeout killed the command. */
|
||||
timedOut: boolean
|
||||
/** True when the caller's AbortSignal killed the command. */
|
||||
aborted: boolean
|
||||
/** The effective timeout applied to this run (after defaulting/capping). */
|
||||
timeoutMs: number
|
||||
stdout: CollectedOutput
|
||||
stderr: CollectedOutput
|
||||
}
|
||||
```
|
||||
|
||||
Each stream is a `CollectedOutput` — the (possibly truncated) text plus recovery info. When truncated, `text` is the **tail** and the complete stream spills to a private file:
|
||||
|
||||
```ts type-equiv
|
||||
interface CollectedOutput {
|
||||
/** Collected text — the TAIL of the stream when truncated. */
|
||||
text: string
|
||||
/** True when bytes were dropped from `text`. */
|
||||
truncated: boolean
|
||||
/** Path to a file holding the COMPLETE stream, when truncated and available. */
|
||||
spillPath?: string
|
||||
}
|
||||
```
|
||||
|
||||
## Background tasks: `BashTask`
|
||||
|
||||
A long-running command started with `start()` is tracked as a `BashTask`. `BashTaskStatus` is `'running' | 'completed' | 'killed'`; `done` resolves when the underlying process closes and never rejects.
|
||||
|
||||
```ts type-equiv
|
||||
interface BashTask {
|
||||
readonly id: string
|
||||
readonly command: string
|
||||
status: BashTaskStatus
|
||||
/** Exit code once finished (null = killed by signal / still running). */
|
||||
exitCode: number | null
|
||||
/** Terminating signal name, when signal-killed. */
|
||||
signal: NodeJS.Signals | null
|
||||
/** Resolves when the underlying process closes (never rejects). */
|
||||
readonly done: Promise<void>
|
||||
}
|
||||
```
|
||||
|
||||
`readOutput()` returns an incremental `BashTaskRead` — the output produced since the previous read, with a `lossy` flag when truncation dropped unread bytes:
|
||||
|
||||
```ts type-equiv
|
||||
interface BashTaskRead {
|
||||
task: BashTask
|
||||
/** Output produced since the previous read (stderr in a marked section). */
|
||||
delta: string
|
||||
/** True when truncation dropped unread bytes the delta cannot include. */
|
||||
lossy: boolean
|
||||
/** Full stdout spill file, when stdout truncation occurred and a safe path is available. */
|
||||
stdoutSpillPath?: string
|
||||
/** Full stderr spill file, when stderr truncation occurred and a safe path is available. */
|
||||
stderrSpillPath?: string
|
||||
}
|
||||
```
|
||||
|
||||
## The service
|
||||
|
||||
`BashExecutor` (`ctx.bash`, abstract — defined in [`packages/bash/src/index.ts`](../../packages/bash/src/index.ts)) mirrors the `LlmService`/`LlmAdapter` split: `resolve` (request → spec), `run` (foreground), `start` (background), `get`/`ownerOf`/`list`/`readOutput`/`kill`, and `onTaskDone` (a `BashTaskListener` completion callback). Spawned commands get a **scrubbed env** (dropping `*KEY*`/`*SECRET*`/`*TOKEN*`) and spill files use a private 0700 dir with random names and owner-only opens — model output never gets the ambient environment or a predictable path. The implementation that provides all this is `dsh-bash-local`; the model-facing `bash`/`bash_output`/`bash_kill` schemas that call it are in `dsh-tool-bash` (and present as terminals via the [tool-presentation vocabulary](tools.md#tool-presentation-ui-vocabulary)).
|
||||
300
docs/core-data-structures/core.md
Normal file
300
docs/core-data-structures/core.md
Normal file
@@ -0,0 +1,300 @@
|
||||
# Core Data Structures
|
||||
|
||||
This folder catalogs the **data structures** of the DeepSeek Harness — what each core type represents, its literal shape, and where the full detail lives. It complements [architecture.md](../architecture.md), which describes *behavior* (the service map, the session/turn/step lifecycle, the event taxonomy); this page describes the *vocabulary* that behavior moves around.
|
||||
|
||||
## What counts as "core"
|
||||
|
||||
The harness is a microkernel: a tiny core plus many plugins. Most types belong to one plugin or one capability. A handful, though, are the **spine** — the language the agent loop and its events traffic in on *every* turn, no matter which optional plugins are loaded. Those are "core".
|
||||
|
||||
Precisely, a data structure is **core** if either:
|
||||
|
||||
1. it flows through the agent-loop spine — the loop holds it, derives it, streams it, or logs it on every turn (a `Message`, a `StreamChunk`, a `SessionEvent`, the `Agent` handle itself), independent of which plugins are present; **or**
|
||||
2. it is the single headline type a plugin author writes against a pipeline — `ToolDefinition` (what every tool *is*).
|
||||
|
||||
Everything else is documented on a **sub-page**, not here. The rule that draws the line: *the type you write, hold, or receive is core; the machinery that types it, renders it, or persists it is a sub-page detail.* So `ToolDefinition` is core, but the `SchemaSpec`/`InferArgs` DSL that types it, the `ToolCallPresentation` vocabulary that renders it, and the `SessionPersistence` seam that stores the event log are not — they live on the sub-pages below.
|
||||
|
||||
| Sub-page | Owns |
|
||||
|---|---|
|
||||
| [llm-streaming.md](llm-streaming.md) | the `StreamChunk` wire protocol + adapter contract, `BlockAssembler`, the `LlmAdapter` seam |
|
||||
| [session.md](session.md) | the full `SessionEventMap` variant catalog, `TurnTrigger`/`TurnEndReason`, `deriveMessages()`, the turn-enclosure invariant |
|
||||
| [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` |
|
||||
| [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, the `tools/execute` waterfall |
|
||||
| [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashTask`s |
|
||||
|
||||
> Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts.
|
||||
|
||||
## The `…Map → derived-union` pattern
|
||||
|
||||
Almost every extensible sum type in the harness follows one shape: an interface keyed by a discriminant tag (the `…Map`), from which the union is derived with `keyof`. Plugins add variants by **declaration merging** — no edit to the owning package.
|
||||
|
||||
```ts ignore-check
|
||||
// The pattern, schematically:
|
||||
interface ThingMap {
|
||||
'a': { kind: 'a'; /* … */ }
|
||||
'b': { kind: 'b'; /* … */ }
|
||||
}
|
||||
type ThingKind = keyof ThingMap // 'a' | 'b'
|
||||
type Thing = ThingMap[keyof ThingMap] // the discriminated union
|
||||
|
||||
// A plugin extends it without touching the source package:
|
||||
declare module '@deepseek-ai/dsh-llm' {
|
||||
interface ThingMap {
|
||||
'c': { kind: 'c'; /* … */ }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Six canonical maps use this pattern; a plugin author extends these:
|
||||
|
||||
| Map | Package | Derives | Catalog |
|
||||
|---|---|---|---|
|
||||
| `ContentBlockMap` | dsh-llm | `ContentBlock` | [below](#content-blocks-and-messages) |
|
||||
| `MessageSourceMap` | dsh-llm | `MessageSource` | [below](#content-blocks-and-messages) |
|
||||
| `FinishReasonMap` | dsh-llm | `FinishReason` | [below](#the-model-request-and-result) |
|
||||
| `TurnTriggerMap` | dsh-session | `TurnTrigger` | [session.md](session.md) |
|
||||
| `TurnEndReasonMap` | dsh-session | `TurnEndReason` | [session.md](session.md) |
|
||||
| `SessionEventMap` | dsh-session | `SessionEvent` | [session.md](session.md) |
|
||||
|
||||
Two large discriminated unions are the ones consumers `switch` over most: **`StreamChunk`** (the streaming protocol) and **`SessionEvent`** (the log entry). Per the repo convention, `switch` on the tag — don't chain `if`s — so each arm narrows and a typo'd tag fails to compile.
|
||||
|
||||
## Branded IDs
|
||||
|
||||
IDs that cross package boundaries are **branded** — structurally strings, but non-interchangeable at the type level (an `AgentId` can't be passed where a `CallId` is expected). Construction goes through a per-type factory; comparison, logging, and JSON behave as ordinary strings.
|
||||
|
||||
Source: [`packages/llm/src/brand.ts`](../../packages/llm/src/brand.ts)
|
||||
|
||||
```ts type-equiv
|
||||
type Branded<B extends string> = string & { readonly [BRAND]: B }
|
||||
```
|
||||
|
||||
The three core IDs: `CallId` (correlates a tool call with its result; dsh-llm), `SessionId` (dsh-session), `AgentId` (dsh-agent). Each is `Branded<'CallId'>` etc. plus a same-named factory function.
|
||||
|
||||
## Content blocks and messages
|
||||
|
||||
A conversation is `Message`s; a message is an array of typed **content blocks**. The block union derives from `ContentBlockMap`.
|
||||
|
||||
Source: [`packages/llm/src/types.ts`](../../packages/llm/src/types.ts)
|
||||
|
||||
```ts type-equiv
|
||||
interface ContentBlockMap {
|
||||
'text': TextBlock
|
||||
'reasoning': ReasoningBlock
|
||||
'tool-call': ToolCallBlock
|
||||
'tool-result': ToolResultBlock
|
||||
'image': ImageBlock
|
||||
}
|
||||
```
|
||||
|
||||
The block interfaces (full fields in source): `TextBlock` (`text`), `ReasoningBlock` (thinking, distinct from visible text), `ToolCallBlock` (`id: CallId`, `name`, raw-JSON `arguments`), `ToolResultBlock` (`toolCallId`, nested `content: ContentBlock[]`, `isError?`), `ImageBlock` (`url`, `mimeType?`). `ContentBlock = ContentBlockMap[ContentBlockType]`.
|
||||
|
||||
A `Message` is a role plus blocks:
|
||||
|
||||
```ts type-equiv
|
||||
interface Message {
|
||||
role: 'system' | 'user' | 'assistant'
|
||||
content: ContentBlock[]
|
||||
}
|
||||
```
|
||||
|
||||
Where a message came from is itself a merge-extensible sum type:
|
||||
|
||||
```ts type-equiv
|
||||
interface MessageSourceMap {
|
||||
user: { kind: 'user' }
|
||||
plugin: { kind: 'plugin'; plugin: string }
|
||||
agent: { kind: 'agent'; agentId: string }
|
||||
}
|
||||
```
|
||||
|
||||
## Streaming
|
||||
|
||||
Adapters emit a raw **chunk** protocol; the loop logs the chunks (replay fidelity) while feeding the same chunks through a `BlockAssembler` to rebuild blocks and messages. `StreamChunk` is a closed discriminated union over `type` — `block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`.
|
||||
|
||||
The full union, the adapter contract (usage-before-finish, raw-JSON tool arguments, the two sanctioned error paths), and `BlockAssembler` live on **[llm-streaming.md](llm-streaming.md)**.
|
||||
|
||||
## The model request and result
|
||||
|
||||
One model call is a fully-assembled `GenerateOptions`; the non-streaming result is `GenerateResult`.
|
||||
|
||||
Source: [`packages/llm/src/types.ts`](../../packages/llm/src/types.ts)
|
||||
|
||||
```ts type-equiv
|
||||
interface GenerateOptions {
|
||||
model: string
|
||||
messages: Message[]
|
||||
/** System prompt text (adapters map to the provider's system slot). */
|
||||
system?: string
|
||||
/** Tool schemas (adapters map to the provider's `tools` field). */
|
||||
tools?: ToolSchema[]
|
||||
/** Assistant prefix continuation (prefill). */
|
||||
prefill?: ContentBlock[]
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
/**
|
||||
* Stop sequences: generation halts as soon as the model produces any one of
|
||||
* these strings (adapters map to the provider's stop field, e.g. OpenAI
|
||||
* `stop`). The stop string itself is not included in the output.
|
||||
*/
|
||||
stop?: string[]
|
||||
signal?: AbortSignal
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
interface GenerateResult {
|
||||
message: Message
|
||||
usage?: TokenUsage
|
||||
finish: FinishReason
|
||||
}
|
||||
```
|
||||
|
||||
Why a model response stopped is a merge-extensible reason:
|
||||
|
||||
```ts type-equiv
|
||||
interface FinishReasonMap {
|
||||
'stop': { kind: 'stop' }
|
||||
'tool-calls': { kind: 'tool-calls' }
|
||||
'max-tokens': { kind: 'max-tokens' }
|
||||
'aborted': { kind: 'aborted' }
|
||||
'error': { kind: 'error'; message: string; code?: string }
|
||||
}
|
||||
```
|
||||
|
||||
`FinishReason = FinishReasonMap[keyof FinishReasonMap]`. `TokenUsage` (per-call accounting with disjoint cache fields) is detailed on [llm-streaming.md](llm-streaming.md).
|
||||
|
||||
`GenerateOptions.tools` carries `ToolSchema` — the JSON-schema description of a tool, as sent to the model. It is declared in dsh-llm (not dsh-tools) precisely because it is part of the request the loop assembles every step:
|
||||
|
||||
```ts type-equiv
|
||||
interface ToolSchema {
|
||||
name: string
|
||||
description: string
|
||||
/** JSON Schema object for the arguments. */
|
||||
parameters: Record<string, unknown>
|
||||
strict?: boolean
|
||||
}
|
||||
```
|
||||
|
||||
The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` that produces it (schema + `execute`) is on [tools.md](tools.md).
|
||||
|
||||
## Sessions
|
||||
|
||||
A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`), not stored separately. The event vocabulary derives from `SessionEventMap`:
|
||||
|
||||
Source: [`packages/session/src/types.ts`](../../packages/session/src/types.ts)
|
||||
|
||||
```ts type-equiv
|
||||
type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
[K in SessionEventType]: {
|
||||
type: K
|
||||
/** Monotonic sequence number within the session. */
|
||||
seq: number
|
||||
/** Unix epoch milliseconds. */
|
||||
time: number
|
||||
data: SessionEventMap[K]
|
||||
}
|
||||
}[T]
|
||||
```
|
||||
|
||||
The thirteen event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`, `usage`, `error`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
|
||||
## The agent handle
|
||||
|
||||
`Agent` is the surface every plugin (UI, hooks, orchestrators) programs against. The concrete implementation is `ReactLoopAgent` in dsh-agent-loop; nothing outside the loop depends on the implementation.
|
||||
|
||||
Source: [`packages/agent/src/types.ts`](../../packages/agent/src/types.ts)
|
||||
|
||||
```ts type-equiv
|
||||
interface Agent {
|
||||
readonly id: AgentId
|
||||
readonly options: AgentOptions
|
||||
readonly session: Session
|
||||
readonly status: AgentStatus
|
||||
|
||||
/** Queue a user message. Starts a turn when idle; otherwise waits for the next turn. */
|
||||
send(content: ContentBlock[], options?: SendOptions): void
|
||||
|
||||
/**
|
||||
* Steer a running turn: content is injected between steps of the current
|
||||
* turn. When idle, behaves like {@link send}.
|
||||
*/
|
||||
steer(content: ContentBlock[], options?: SendOptions): void
|
||||
|
||||
/**
|
||||
* Inject in-session context (file-change notices, skill content, cron
|
||||
* notifications, …): appends a `context/message` session event the next model
|
||||
* request sees at its chronological position, rendered as tagged synthetic
|
||||
* context rather than a user prompt. Does not run the model.
|
||||
*
|
||||
* Turn-enclosure (the turn-enclosure RFC): an inject while a turn is open joins that turn;
|
||||
* an inject while idle wraps its `context/message` in a one-shot `injection`
|
||||
* turn (`turn/start` → `context/message` → `turn/end`) and checkpoints it for
|
||||
* durability, so every event stays inside a turn and a persistence backend
|
||||
* never loses a between-turn notice. The idle checkpoint is fire-and-forget
|
||||
* (inject is synchronous): a failing flush is reported via `agent/error`
|
||||
* (step `0`) and the logger, never thrown into the caller.
|
||||
*
|
||||
* Live-adapter review has validated the tagged-envelope rendering against
|
||||
* current DeepSeek behavior; provider-specific mismatches belong in that
|
||||
* adapter, not in the canonical session vocabulary.
|
||||
*/
|
||||
inject(content: ContentBlock[], options?: SendOptions): void
|
||||
|
||||
/** Abort the in-flight step (if any); the turn ends with reason 'aborted'. */
|
||||
abort(reason?: string): void
|
||||
|
||||
/**
|
||||
* Cancel ALL pending work for the agent — the narrower {@link abort} kills
|
||||
* only the in-flight step. `cancel()`:
|
||||
*
|
||||
* - clears the queued FIFO (un-started prompts never run) and the steering
|
||||
* FIFO (steering for the cancelled turn is dropped, not re-enqueued);
|
||||
* - aborts the in-flight step if one is running (the turn ends `aborted`);
|
||||
* - drops a turn that is about to start (a `cancel()` landing in the
|
||||
* pre-step window — after a `send()` queued but before the loop flips to
|
||||
* `running`, or after `running` is emitted but before the first step) so
|
||||
* that queued prompt does not run and cannot be batched into the cancelled
|
||||
* turn.
|
||||
*
|
||||
* After `cancel()`, `whenIdle()` resolves on the post-cancel quiescent state.
|
||||
* `cancel()` on an idle agent with nothing queued or running is a safe no-op
|
||||
* — it does NOT arm anything that would drop a later legitimate prompt.
|
||||
*/
|
||||
cancel(reason?: string): void
|
||||
|
||||
/**
|
||||
* Resolve once the agent has reached quiescence after settling out of
|
||||
* `running`, or immediately if it is already idle with no queued work. The
|
||||
* quiescence signal a teardown awaits: `agent.abort()` then
|
||||
* `await agent.whenIdle()` guarantees queued/running work has fully stopped
|
||||
* before the caller proceeds (a closing ACP connection, a disposing UI
|
||||
* plugin), rather than returning while the driver is still streaming or about
|
||||
* to start a queued turn.
|
||||
*
|
||||
* "Quiescence", not merely "status changed": a disposed agent emits
|
||||
* `agent/status('disposed')` from inside its disposer, BEFORE the driver loop
|
||||
* has unwound — so `whenIdle()` resolving on `disposed` must wait for the loop
|
||||
* to actually exit (the implementation chains the loop-exit promise), not just
|
||||
* observe the status flip. A mid-step disposal that never reaches `idle` still
|
||||
* unblocks the await this way.
|
||||
*
|
||||
* Distinct from disposal: `whenIdle()` observes the transition WITHOUT tearing
|
||||
* the agent down. A consumer that owns the agent's lifecycle disposes it
|
||||
* separately.
|
||||
*/
|
||||
whenIdle(): Promise<void>
|
||||
|
||||
// TODO(sub-agents): spawn/fork seams — semantics deliberately deferred.
|
||||
// The intended shape: a creation option referencing a parent agent
|
||||
// (fork = seed the child Session with the parent's event log; spawn =
|
||||
// fresh Session), with the child returned as an Agent handle so steer()
|
||||
// and event subscription work uniformly. See docs/architecture.md.
|
||||
}
|
||||
```
|
||||
|
||||
`AgentStatus` is `'idle' | 'running' | 'disposed'`. `AgentId` is a branded string. `AgentOptions` (`model?`, `systemPrompt?`) is merge-extensible — plugins add creation options by declaration merging. The `agent/*` event taxonomy (lifecycle, turn/step boundaries, the `agent/request`/`agent/step-result`/`agent/turn-continuation` waterfalls) is in [architecture.md § Event taxonomy](../architecture.md#event-taxonomy).
|
||||
|
||||
## `ToolDefinition`
|
||||
|
||||
The one pipeline-authoring type that is core: what every registered tool *is* — a model-facing `ToolSchema` plus an `execute` function and optional UI presenters. A tool author rarely constructs it by hand (the `defineTool` DSL builds it with typed args), but it is the contract the registry holds and the loop dispatches through.
|
||||
|
||||
Its full fields, the `defineTool`/`SchemaSpec`/`InferArgs` typed schema DSL, the `ToolExecution`/`ToolExecutionResult` waterfall shapes, and the tool-presentation UI vocabulary are on **[tools.md](tools.md)**.
|
||||
66
docs/core-data-structures/llm-streaming.md
Normal file
66
docs/core-data-structures/llm-streaming.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# LLM Streaming
|
||||
|
||||
The wire-level streaming vocabulary of [dsh-llm](../../packages/llm). [core.md](core.md) introduces `StreamChunk`, `Message`, and `ContentBlock`; this page owns the full chunk protocol, the adapter contract every adapter must obey, and the shared assembler.
|
||||
|
||||
Source: [`packages/llm/src/types.ts`](../../packages/llm/src/types.ts)
|
||||
|
||||
## `StreamChunk` — the raw protocol
|
||||
|
||||
A streaming response interleaves several typed blocks (text, reasoning, multiple tool calls). `index` ties each delta to its block; `block-end` carries the fully-assembled `ContentBlock` so consumers don't have to re-assemble deltas themselves. It is a **closed** discriminated union — a `switch` over `type` ends with `assertNever`, so adding a variant breaks compilation at every consumer that must handle it.
|
||||
|
||||
```ts type-equiv
|
||||
type StreamChunk =
|
||||
| { type: 'block-start'; index: number; blockType: ContentBlockType }
|
||||
| { type: 'text-delta'; index: number; text: string }
|
||||
| { type: 'reasoning-delta'; index: number; text: string }
|
||||
| { type: 'tool-call-delta'; index: number; id: CallId; name?: string; argumentsDelta: string }
|
||||
| { type: 'block-end'; index: number; block: ContentBlock }
|
||||
| { type: 'usage'; usage: TokenUsage }
|
||||
| { type: 'finish'; reason: FinishReason }
|
||||
```
|
||||
|
||||
## The adapter contract
|
||||
|
||||
Every adapter MUST obey these, and every consumer may rely on them:
|
||||
|
||||
- **`usage` before `finish`, nothing after `finish`.** Defer both to the provider's end-of-stream marker so a trailing usage-only chunk can't violate the ordering.
|
||||
- **Tool-call `arguments` stay raw JSON strings end-to-end.** Partial fragments stream via `argumentsDelta`; a provider that hands back parsed objects re-stringifies at `block-end`.
|
||||
- **Two sanctioned error paths.** A failure may either THROW from `stream()` (transport/protocol errors) **or** end the stream with `finish {kind:'error'|'aborted'}` (provider in-band errors, for adapters that can't throw mid-stream). Consumers must handle *both*. The agent loop translates a finish-error/aborted into a turn error — it never logs a normal completed assistant message for a failed step.
|
||||
|
||||
This contract is why two adapters exist as a deliberate pair: `dsh-llm-deepseek` (hand-rolled fetch/SSE) and `dsh-llm-pi-ai` (the same endpoint through `@earendil-works/pi-ai`). Two independent internals over one contract is what pinned the protocol down — the library-backed adapter can't throw mid-stream, so it exercises the finish-chunk error path the hand-rolled one might not.
|
||||
|
||||
## `TokenUsage`
|
||||
|
||||
Per-call token accounting. Counts are **disjoint**: `inputTokens` is uncached input only; cached input is reported separately, and billed input is the sum of the three. Adapters whose providers fold cache hits into a single prompt total (DeepSeek's `prompt_tokens`) subtract them back out.
|
||||
|
||||
```ts type-equiv
|
||||
interface TokenUsage {
|
||||
inputTokens: number
|
||||
outputTokens: number
|
||||
cacheReadTokens?: number
|
||||
cacheWriteTokens?: number
|
||||
reasoningTokens?: number
|
||||
}
|
||||
```
|
||||
|
||||
## `BlockAssembler`
|
||||
|
||||
`BlockAssembler` ([`packages/llm/src/assembler.ts`](../../packages/llm/src/assembler.ts)) is the single shared implementation that folds a `StreamChunk` stream back into `ContentBlock`s and a final `Message`. The loop logs the raw chunks (for replay fidelity) while feeding the same chunks through an assembler — so the canonical log keeps token-level detail and the derived message is rebuilt deterministically. A consumer that needs the assembled result without re-implementing the fold uses this.
|
||||
|
||||
## The seam
|
||||
|
||||
`LlmAdapter` is the provider seam: subclass, implement `stream()`, register with `ctx.llm.registerAdapter(models, adapter)`. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. The consumer surface (`ctx.llm.stream()` / `streamBlocks()` / `generate()`) and the `llm/stream` waterfall are described in [architecture.md § The vocabulary](../architecture.md#the-vocabulary-dsh-llm).
|
||||
|
||||
`ContentBlockType` (the key set the `index`-correlated blocks carry) derives from `ContentBlockMap`:
|
||||
|
||||
```ts type-equiv
|
||||
interface ContentBlockMap {
|
||||
'text': TextBlock
|
||||
'reasoning': ReasoningBlock
|
||||
'tool-call': ToolCallBlock
|
||||
'tool-result': ToolResultBlock
|
||||
'image': ImageBlock
|
||||
}
|
||||
```
|
||||
|
||||
See [core.md § Content blocks and messages](core.md#content-blocks-and-messages) for the block interfaces.
|
||||
63
docs/core-data-structures/persistence.md
Normal file
63
docs/core-data-structures/persistence.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Session Persistence
|
||||
|
||||
The **durability seam** for the event log. [session.md](session.md) describes the in-memory `Session` — the append-only `SessionEvent` log that is the source of truth. This page describes how that log is made durable: the abstract `SessionPersistence` service, its backends, the flush checkpoint, crash recovery, and the metadata header that travels alongside the log.
|
||||
|
||||
The seam is a textbook [capability seam](../rfc/implemented/2026-06-13-capability-seams.md): one abstract service ([dsh-session-persistence](../../packages/session-persistence), `ctx.sessionPersistence`) defining create/append/load/list/has/delete over the existing `SessionEvent` — **no parallel persisted type** — and two interchangeable backends that pass the same `runPersistenceContract` suite. See the [session-persistence RFC](../rfc/implemented/2026-06-14-session-persistence.md).
|
||||
|
||||
## The flush checkpoint
|
||||
|
||||
`session/event` is a *synchronous* notification; persistence plugins buffer it (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. Flush is `ctx.parallel` (awaited): a turn's events are durably committed before the next turn starts, and the turn boundary is the commit boundary. A rejecting flush is reported via `agent/error` and the logger — never as a session event (it would land past the commit boundary), so the backend keeps its buffered events for the next flush.
|
||||
|
||||
## Crash recovery preserves an interrupted turn
|
||||
|
||||
A backend that reloads a log crashed mid-turn finds an open `turn/start` with no `turn/end`. It does **not** truncate — a single turn can be huge in a long-horizon task (many steps, large tool output), and those events were durably appended before the crash. Instead it closes the orphaned turn with a synthetic `turn/end { reason: { kind: 'interrupted' } }`, keeping the log balanced and the turn-enclosure invariant intact. `interrupted` is the one `TurnEndReason` no loop emits (see [session.md](session.md#why-a-turn-ended-turnendreasonmap)).
|
||||
|
||||
## `SessionHeader` — metadata beside the log
|
||||
|
||||
Per-session metadata travels **separately** from the event log: format version, cwd, and lineage are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`.
|
||||
|
||||
Source: [`packages/session/src/types.ts`](../../packages/session/src/types.ts)
|
||||
|
||||
```ts type-equiv
|
||||
interface SessionHeader {
|
||||
/** On-disk format version; a persistence backend rejects unknown versions. */
|
||||
version: number
|
||||
/** The session's id (mirrors the {@link Session}'s id). */
|
||||
id: SessionId
|
||||
/** Unix epoch milliseconds when the session was created. */
|
||||
createdAt: number
|
||||
/** Absolute working directory the session was created in (if any). */
|
||||
cwd?: string
|
||||
/** The session this one was forked from (seed lineage), if any. */
|
||||
parentSession?: SessionId
|
||||
}
|
||||
```
|
||||
|
||||
## `CreateSessionOptions` — seeding and metadata
|
||||
|
||||
Creating a `Session` through the store takes a `seed` (replay/fork an existing event log) and `meta` (the storage-level fields the store folds into a `SessionHeader`). The store fills in `version`/`id` and defaults `createdAt`; the caller supplies the validated absolute `cwd`, the `parentSession` lineage, and — only when reconstructing a persisted session — the original `createdAt` to preserve it.
|
||||
|
||||
```ts type-equiv
|
||||
interface CreateSessionOptions {
|
||||
/** Events to seed the new session with (replay/fork). */
|
||||
seed?: SessionEvent[]
|
||||
/**
|
||||
* Creation metadata. The store fills in `version`/`id` and defaults
|
||||
* `createdAt` to now; the caller supplies the storage-level fields (validated
|
||||
* absolute `cwd`, `parentSession` lineage, and — when reconstructing a
|
||||
* persisted session — the original `createdAt` to preserve it).
|
||||
*/
|
||||
meta?: { cwd?: string; parentSession?: SessionId; createdAt?: number }
|
||||
}
|
||||
```
|
||||
|
||||
Replay/fork is therefore `ctx.sessions.create(id, { seed: seedEvents })`; resuming a *persisted* session into a live agent is `ctx.agents.resume({ resumeSessionId })`.
|
||||
|
||||
## The backends
|
||||
|
||||
Both implement the same abstract `SessionPersistence` (create/append/load/list/has/delete over `SessionEvent`) and pass `runPersistenceContract`, proving the seam is genuinely backend-agnostic:
|
||||
|
||||
- **[dsh-session-persistence-jsonl](../../packages/session-persistence-jsonl)** — an append-only JSONL log per session with crash-safe atomic writes, the interrupted-turn crash recovery above, and a read/replay path.
|
||||
- **[dsh-session-persistence-sqlite](../../packages/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data)` maps 1:1 onto the event, so there is no parallel persisted schema to keep in sync.
|
||||
|
||||
Multiple backends sharing one on-disk session coordinate writes through the [shared persistence write-coordinator](../rfc/implemented/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
119
docs/core-data-structures/session.md
Normal file
119
docs/core-data-structures/session.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Sessions
|
||||
|
||||
The in-memory, event-sourced model of [dsh-session](../../packages/session). A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth for an agent's whole interaction history. The LLM message history is *derived* from the log, never stored separately; replay is re-derivation from the same events. How the log is made **durable** (the persistence seam, backends, crash recovery) is the sibling concern on [persistence.md](persistence.md).
|
||||
|
||||
Source: [`packages/session/src/types.ts`](../../packages/session/src/types.ts)
|
||||
|
||||
## `SessionEventMap` — the event vocabulary
|
||||
|
||||
The append-only event types. Merge-extensible: a plugin (e.g. compaction) declares extra event types via declaration merging.
|
||||
|
||||
```ts type-equiv
|
||||
interface SessionEventMap {
|
||||
'turn/start': { turn: number; trigger: TurnTrigger }
|
||||
'turn/end': { turn: number; reason: TurnEndReason }
|
||||
'step/start': { turn: number; step: number }
|
||||
'step/end': { turn: number; step: number }
|
||||
/** A user-visible prompt (queued message drained at turn start). */
|
||||
'user/message': { content: ContentBlock[]; source: MessageSource }
|
||||
/**
|
||||
* In-session context injection (file-change notices, subdir AGENTS.md,
|
||||
* skill content, cron notifications, …). Rendered into the derived history
|
||||
* as tagged synthetic context — NOT a user prompt.
|
||||
*/
|
||||
'context/message': { content: ContentBlock[]; source: MessageSource }
|
||||
/** Raw stream chunk — token-level replay fidelity. */
|
||||
'assistant/chunk': { turn: number; step: number; chunk: StreamChunk }
|
||||
/** Assembled assistant message for one step (derived history uses this). */
|
||||
'assistant/message': { turn: number; step: number; content: ContentBlock[] }
|
||||
'tool/call': { turn: number; step: number; callId: CallId; name: string; arguments: string }
|
||||
'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string } }
|
||||
/** Steering content injected between steps of a running turn. */
|
||||
'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource }
|
||||
'usage': { turn: number; step: number; usage: TokenUsage }
|
||||
'error': { turn: number; step: number; message: string; code?: string }
|
||||
}
|
||||
```
|
||||
|
||||
## `SessionEvent<T>` — one log entry
|
||||
|
||||
A proper discriminated union over `type` (not independent `type`/`data` unions), so `switch (event.type)` narrows `event.data` without casts. `seq` is the monotonic position in the log (`seq = log.length`); `time` is epoch ms.
|
||||
|
||||
```ts type-equiv
|
||||
type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
[K in SessionEventType]: {
|
||||
type: K
|
||||
/** Monotonic sequence number within the session. */
|
||||
seq: number
|
||||
/** Unix epoch milliseconds. */
|
||||
time: number
|
||||
data: SessionEventMap[K]
|
||||
}
|
||||
}[T]
|
||||
```
|
||||
|
||||
`SessionEventType = keyof SessionEventMap`. Because `SessionEventMap` is merge-extensible, switches over `SessionEvent` must NOT use `assertNever` — a plugin-added variant is a valid unknown value; handle the known cases and fall through `default`.
|
||||
|
||||
## Derived history: `deriveMessages()`
|
||||
|
||||
`Session.deriveMessages()` projects the event log into the `Message[]` the model sees. The projection rules:
|
||||
|
||||
- `user/message` → a user message.
|
||||
- `assistant/message` → an assistant message. Raw `assistant/chunk` events are replay/UI data and are **skipped** in derivation (the assembled message is authoritative).
|
||||
- `tool/result` → a user message carrying a `tool-result` block.
|
||||
- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`<context source="…">…</context>`) at their chronological position — the "system-reminder" pattern; the model distinguishes them from real prompts by the envelope.
|
||||
|
||||
Everything else (`turn/*`, `step/*`, `usage`, `error`) is structural/telemetry and does not project into a message.
|
||||
|
||||
## What started a turn: `TurnTriggerMap`
|
||||
|
||||
```ts type-equiv
|
||||
interface TurnTriggerMap {
|
||||
message: { kind: 'message'; source: MessageSource }
|
||||
continuation: { kind: 'continuation' }
|
||||
/**
|
||||
* An out-of-band context injection (`agent.inject()`) made while the agent
|
||||
* was idle. The loop wraps the injected `context/message` in a one-shot turn
|
||||
* (`turn/start` → `context/message` → `turn/end`) so every event in the log
|
||||
* stays turn-enclosed — the durability/replay boundary is the turn, and a
|
||||
* bare event between turns would otherwise be indistinguishable from a crash
|
||||
* tail on reload.
|
||||
*/
|
||||
injection: { kind: 'injection'; source: MessageSource }
|
||||
}
|
||||
```
|
||||
|
||||
## Why a turn ended: `TurnEndReasonMap`
|
||||
|
||||
```ts type-equiv
|
||||
interface TurnEndReasonMap {
|
||||
completed: { kind: 'completed' }
|
||||
aborted: { kind: 'aborted'; reason?: string }
|
||||
error: { kind: 'error'; message: string; code?: string }
|
||||
disposed: { kind: 'disposed' }
|
||||
'max-tokens': { kind: 'max-tokens' }
|
||||
/**
|
||||
* The turn never ended on its own: the process crashed mid-turn and a
|
||||
* persistence backend later closed the orphaned (open) turn on reload so the
|
||||
* log stays balanced. SYNTHESIZED by the backend's crash-recovery repair — no
|
||||
* loop ever emits this. Its events are real (they were durably appended before
|
||||
* the crash) and are PRESERVED, not discarded: a single turn can be huge in a
|
||||
* long-horizon task (many steps, large tool output), so truncating it would
|
||||
* lose real work. The marker records that the turn was cut short, not that the
|
||||
* model completed it. See the session-persistence RFC.
|
||||
*/
|
||||
interrupted: { kind: 'interrupted' }
|
||||
}
|
||||
```
|
||||
|
||||
`max-tokens` mirrors the model-call `FinishReason` of the same name: any `max-tokens` step in a turn makes the whole turn end `max-tokens` (the cut-short fact wins over a later continuation), so a consumer can tell a clean stop from a truncated one. `interrupted` is the one reason no loop emits — it is synthesized by crash recovery (see [persistence.md](persistence.md)). Both maps are merge-extensible.
|
||||
|
||||
## The turn-enclosure invariant
|
||||
|
||||
Every session event lives **inside** a turn (between a `turn/start` and its `turn/end`). The loop appends queued `user/message` events *after* `turn/start`, and an idle `agent.inject()` wraps its `context/message` in a one-shot `injection` turn. This makes the turn the single durability/replay boundary: a backend can treat anything after the last `turn/end` as an interrupted-crash tail without risking the loss of legitimately-recorded between-turn context. The `dsh-invariants` plugin enforces it in dev (a message event outside an open turn throws). See [the turn-enclosure invariant RFC](../rfc/implemented/2026-06-15-turn-enclosure-invariant.md).
|
||||
|
||||
## Durability contract
|
||||
|
||||
What a persistence backend relies on: the durable log persists every event verbatim, **including** `assistant/chunk` — `seq` must stay contiguous, so chunks cannot be filtered out of the canonical log. All `event.data` must be JSON-serializable; `Session.append` enforces this at the source (throwing on non-serializable data), so a bad event never enters the log and `session.events` always equals what a backend can persist. Adding an event type that carries non-serializable data, or that breaks the turn/step nesting the invariants plugin checks, is a breaking change to the on-disk format.
|
||||
|
||||
The backends that consume this contract are on [persistence.md](persistence.md).
|
||||
112
docs/core-data-structures/tools.md
Normal file
112
docs/core-data-structures/tools.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# Tools
|
||||
|
||||
The tool pipeline of [dsh-tools](../../packages/tools). [core.md](core.md) introduces `ToolDefinition` as the one pipeline-authoring type promoted to the spine and `ToolSchema` as the model-facing wire shape. This page owns the full `ToolDefinition`, the typed schema DSL that builds it, the waterfall execution shapes, and the UI-presentation vocabulary.
|
||||
|
||||
Source: [`packages/tools/src/index.ts`](../../packages/tools/src/index.ts) · [`packages/tools/src/schema.ts`](../../packages/tools/src/schema.ts)
|
||||
|
||||
## `ToolDefinition` — a registered tool
|
||||
|
||||
A `ToolSchema` (the model-facing fields) plus the `execute` function and optional UI presenters. The registry holds these; the loop dispatches calls through them. The registry's `schemas()` builds the model-facing `ToolSchema[]` by an explicit allowlist — `execute`/`presentCall`/`presentResult` must never leak into a model request.
|
||||
|
||||
```ts type-equiv
|
||||
interface ToolDefinition extends ToolSchema {
|
||||
execute(args: unknown, exec: ToolExecution): Promise<ContentBlock[]>
|
||||
/**
|
||||
* Optional: how to present the PENDING state of one call in a UI, derived
|
||||
* from the call's `args` (parsed arguments, `unknown` — the tool validates/
|
||||
* narrows its own input). Returning `undefined` (or omitting the method) tells
|
||||
* a UI to fall back to a generic presentation (title = tool name, raw args as
|
||||
* input). Pure and side-effect-free: a UI may call it during live streaming
|
||||
* AND a session-log replay, so it must depend only on `args`.
|
||||
*/
|
||||
presentCall?(args: unknown): ToolCallPresentation | undefined
|
||||
/**
|
||||
* Optional: how to present the COMPLETED state, given the same `args` and the
|
||||
* `result` (`execute`'s content + whether it errored). Returning `undefined`
|
||||
* (or omitting the method) tells a UI to keep the pending title and render the
|
||||
* raw result content. Pure and side-effect-free for the same replay reason.
|
||||
*/
|
||||
presentResult?(args: unknown, result: ToolResult): ToolResultPresentation | undefined
|
||||
}
|
||||
```
|
||||
|
||||
`execute` receives `args: unknown` — a raw `ToolDefinition` validates its own input. First-party tools don't write that by hand; they use `defineTool`, which validates and narrows for them.
|
||||
|
||||
## The typed schema DSL
|
||||
|
||||
Plugin authors write per-property specs with a boolean `required: true`, and a type-level helper maps the spec to the `execute` argument type — zero casts. The DSL is *machinery that types* `ToolDefinition`; it is intentionally a sub-page detail, not core.
|
||||
|
||||
Source: [`packages/tools/src/schema.ts`](../../packages/tools/src/schema.ts)
|
||||
|
||||
```ts type-equiv
|
||||
interface SchemaProp {
|
||||
type: SchemaType
|
||||
/** Per-property required flag (NOT the JSON Schema top-level required array). */
|
||||
required?: true
|
||||
/** Human-readable description, surfaced in the JSON Schema as well. */
|
||||
description?: string
|
||||
/** Enum of allowed values (strings only). */
|
||||
enum?: string[]
|
||||
/** Default value. */
|
||||
default?: unknown
|
||||
/** Nested properties for type: 'object'. */
|
||||
properties?: SchemaSpec
|
||||
/** Items schema for type: 'array'. */
|
||||
items?: SchemaProp
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
type SchemaSpec = Record<string, SchemaProp>
|
||||
```
|
||||
|
||||
`SchemaType` is the primitive union `'string' | 'number' | 'boolean' | 'object' | 'array'`. `InferArgs<S>` maps a `SchemaSpec` to the TS argument type — `required: true` props become required keys, everything else genuinely optional:
|
||||
|
||||
```ts type-equiv
|
||||
type InferArgs<S extends SchemaSpec> = Simplify<
|
||||
& { [K in RequiredKeys<S>]: InferPropValue<S[K]> }
|
||||
& { [K in Exclude<keyof S, RequiredKeys<S>>]?: InferPropValue<S[K]> }
|
||||
>
|
||||
```
|
||||
|
||||
`defineTool({ name, description, parameters, execute, … })` ties it together: `parameters` is a `SchemaSpec`, `execute(args, exec)` gets `args: InferArgs<typeof parameters>`, and the helper converts the spec to JSON Schema (`schemaSpecToJsonSchema`) for the wire and validates model-generated args (`validateArgs`) before the typed body runs. A mismatch throws `ToolArgsError` (`code: 'INVALID_ARGS'`), which the registry turns into an `isError` result so the model can self-correct. Why a custom DSL and not schemastery: tool parameters need JSON Schema (the LLM wire format), not validation/transformation — the lightweight DSL gives the best authoring DX with the smallest surface.
|
||||
|
||||
## Execution: the `tools/execute` waterfall shapes
|
||||
|
||||
`ctx.tools.execute()` runs each call through the `tools/execute` waterfall — the single seam where sandbox, permission, hook, and plan-mode plugins wrap or veto. The pending call is a `ToolExecution`; the outcome is a `ToolExecutionResult`.
|
||||
|
||||
```ts type-equiv
|
||||
interface ToolExecution {
|
||||
callId: CallId
|
||||
name: string
|
||||
/** Parsed JSON arguments (unknown — tools validate their own input). */
|
||||
arguments: unknown
|
||||
/** The agent on whose behalf the call runs (set by the agent loop). */
|
||||
agent?: Agent
|
||||
signal?: AbortSignal
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
interface ToolExecutionResult {
|
||||
callId: CallId
|
||||
content: ContentBlock[]
|
||||
isError: boolean
|
||||
/**
|
||||
* Set when the call failed with a {@link HarnessError}: machine-routable
|
||||
* `{ name, code }` for retry/sandbox plugins and replay. The model-facing
|
||||
* text in `content` is always present; this is extra structure for code.
|
||||
*/
|
||||
error?: ToolErrorInfo
|
||||
}
|
||||
```
|
||||
|
||||
A waterfall listener receives `(exec, next)`: call `next()` to proceed (possibly around your own logic), or return a `ToolExecutionResult` without calling `next()` to veto. An unregistered tool routes through the same catch as a tool-thrown error, so both failure classes get a structured `{ name, code }` (`ToolNotFoundError` → `UNKNOWN_TOOL`) — the loop records a failed tool call instead of failing the whole turn.
|
||||
|
||||
## Tool-presentation UI vocabulary
|
||||
|
||||
How a tool wants its call shown in a UI (an editor tool-call card, a CLI log line), provider-neutral so a tool describes itself without depending on any client protocol. `presentCall` returns a `ToolCallPresentation` (pending state: `title`, `kind`, `rawInput`, `content`, optional `terminal`); `presentResult` returns a `ToolResultPresentation` (completed state: replacement `title`, reformatted `content`, terminal `output`/exit). `ToolCallKind` (`'read' | 'edit' | 'delete' | 'move' | 'search' | 'execute' | 'fetch' | 'other'`) picks an icon. A `ToolTerminal` asks a capable UI to render the call as a terminal card (cwd header, output, exit-status pill).
|
||||
|
||||
> These shapes carry a `FIXME(tool-presentation)` in source: they grew incrementally and the call-vs-result terminal split is muddy. Before more tools/UIs depend on them, they will be redesigned (a tagged union over card kinds) and pinned in an RFC, migrating `dsh-tool-bash` and the ACP bridge together. Treat the field-level shapes here as provisional; the source is authoritative.
|
||||
|
||||
The full presentation field docs live in [`packages/tools/src/index.ts`](../../packages/tools/src/index.ts). The bash tool's own schemas (`bash`/`bash_output`/`bash_kill`) and the executor they drive are on [bash.md](bash.md).
|
||||
@@ -93,16 +93,18 @@ pnpm run typecheck # build declarations, then typecheck source, tests, and
|
||||
pnpm run lint # eslint .
|
||||
pnpm run lint:fix # eslint . --fix
|
||||
pnpm run doc-typecheck # compile checked TypeScript snippets in Markdown docs
|
||||
pnpm run verify-event-taxonomy # compare docs/architecture.md event names with source
|
||||
pnpm run gen-cordis-catalog # regenerate docs/cordis-catalog/events-and-services.md from source
|
||||
pnpm run verify-cordis-catalog # fail if the cordis events/services catalog is stale
|
||||
pnpm run verify-md-wrap # fail on hard-wrapped prose paragraphs in docs/README markdown
|
||||
pnpm run doc-sync # doc-typecheck, event taxonomy, and markdown wrap verification
|
||||
pnpm run verify-type-equiv # fail if a ```ts type-equiv doc block drifts from its source type
|
||||
pnpm run doc-sync # doc-typecheck, cordis-catalog freshness, markdown wrap/link, and type-equiv verification
|
||||
pnpm run gen-module-graph # regenerate docs/module-graph.md from package peerDeps
|
||||
pnpm run verify-module-graph # fail if docs/module-graph.md is stale
|
||||
pnpm run build # build declarations and JS bundles
|
||||
pnpm run hygiene # knip, publint, and workspace constraints
|
||||
```
|
||||
|
||||
When changing package public behavior, update the relevant README or JSDoc in the same change. `pnpm run doc-sync` catches checked TypeScript snippets, event-taxonomy drift, and hard-wrapped markdown prose, but broader prose/API sync still needs review.
|
||||
When changing package public behavior, update the relevant README or JSDoc in the same change. `pnpm run doc-sync` catches checked TypeScript snippets, cordis events/services catalog drift, and hard-wrapped markdown prose, but broader prose/API sync still needs review.
|
||||
|
||||
## Demos
|
||||
|
||||
@@ -128,6 +130,16 @@ Use one of three comment tags to flag known issues in the code, ordered by urgen
|
||||
|
||||
Pick the tag that matches the urgency so anyone scanning the code can tell a release blocker from a someday-maybe.
|
||||
|
||||
## Documenting types verbatim (`ts type-equiv`)
|
||||
|
||||
The [core data structures](core-data-structures/core.md) docs paste real type definitions so a reader sees the exact shape. To keep a paste from drifting when source changes, fence it as ` ```ts type-equiv ` (instead of ` ```ts `) and register it in `scripts/type-equiv.manifest.json` with the source file and symbol it mirrors:
|
||||
|
||||
```json
|
||||
{ "doc": "docs/core-data-structures/session.md", "symbol": "SessionEvent", "source": "packages/session/src/types.ts" }
|
||||
```
|
||||
|
||||
`pnpm run verify-type-equiv` (part of `doc-sync`) then extracts that symbol's declaration from source via the TypeScript parser and asserts the block matches it (whitespace- and comment-insensitive, so a doc block may show a clean definition and the prose can carry the semantics). It also enforces a 1:1 correspondence: every `ts type-equiv` block has exactly one manifest entry and vice-versa, so a block can't go silently unchecked and a stale entry can't linger. `doc-typecheck` skips `ts type-equiv` blocks (they aren't standalone-compilable) and excludes them from its opt-out ratio. When you change a documented type, the gate fails until you update the paste; when you add or remove a block, update the manifest in the same change.
|
||||
|
||||
## Architecture context
|
||||
|
||||
Read `docs/architecture.md` before changing anything under `packages/`. The codebase is built around Cordis plugins, event-sourced sessions, typed service seams, and explicit extension points.
|
||||
|
||||
@@ -73,6 +73,8 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Drop the mutable session summary](implemented/2026-06-19-drop-mutable-session-summary.md) | 2026-06-19 |
|
||||
| [Shared persistence write coordinator](implemented/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 |
|
||||
| [Agent lifecycle and ownership seams](implemented/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 |
|
||||
| [Core-data-structures catalog and the `ts type-equiv` drift gate](implemented/2026-06-20-core-data-structures-catalog.md) | 2026-06-20 |
|
||||
| [Generated cordis events + services catalog](implemented/2026-06-20-generated-cordis-catalog.md) | 2026-06-20 |
|
||||
|
||||
## Rejected
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ AGENTS.md promises that docs and code stay strictly in sync, but the promise was
|
||||
Two gates, mirroring the existing `scripts/` style (tsx ESM, one job each):
|
||||
|
||||
1. **`doc-typecheck`** extracts every fenced ` ```ts ` block from `README.md`, `docs/**`, and `packages/*/README.md`, writes them to a temp project, and compiles with `tsc --noEmit`. The temp tsconfig copies only resolution-relevant options and the workspace `paths` map from `tsconfig.typecheck.json` (vendor → built `lib`, harness → `src`) — resolving vendor to `lib` is essential, or tsc type-checks raw vendor source and floods the run. A block that is a deliberate sketch opts out with an explicit ` ```ts ignore-check ` info string; the script reports the opt-out ratio and fails if it exceeds half, so the escape hatch can't quietly become the norm.
|
||||
2. **`verify-event-taxonomy`** extracts the event names from the `interface Events` blocks across `packages/*/src` and from the taxonomy table in `docs/architecture.md`, and asserts the two sets match exactly. Verify, don't generate: the table keeps its hand-written Mode/Purpose columns; only the set of names is checked. (Landing this surfaced three events the table had been missing — `tools/change`, `llm/adapter-change`, `system-prompt/change`.)
|
||||
2. **`verify-event-taxonomy`** extracts the event names from the `interface Events` blocks across `packages/*/src` and from the taxonomy table in `docs/architecture.md`, and asserts the two sets match exactly. Verify, don't generate: the table keeps its hand-written Mode/Purpose columns; only the set of names is checked. (Landing this surfaced three events the table had been missing — `tools/change`, `llm/adapter-change`, `system-prompt/change`.) **Superseded** by [the generated cordis catalog](2026-06-20-generated-cordis-catalog.md): this gate and its `architecture.md` table are retired in favor of a fully-generated `docs/cordis-catalog/events-and-services.md` and its `verify-cordis-catalog` freshness gate. The other gates here (`doc-typecheck`, and the `verify-md-wrap` amendment below) are unaffected.
|
||||
|
||||
Both run via a shared `doc-sync` package.json script that the lefthook pre-push hook and CI both invoke ([mechanical quality gates](2026-06-11-quality-gates.md): hooks and CI call the same scripts, so the gate fires locally before a push — not only after it). They run after `pnpm run typecheck` (which emits the vendor `lib/` that doc-typecheck resolves against). API-extractor golden reports ([the deferred API-extractor-reports proposal](../proposed/2026-06-11-api-extractor-reports.md)) were deliberately **deferred** — low value for an internal monorepo where reviewers already see the source diff, and a heavy, finicky dependency.
|
||||
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# RFC: Core-data-structures catalog and the `ts type-equiv` drift gate
|
||||
|
||||
Status: implemented (accepted 2026-06-20)
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Context
|
||||
|
||||
A reader trying to understand the harness could find its *behavior* in [architecture.md](../../architecture.md) (the service map, the session/turn/step lifecycle, the event taxonomy) but had no single place describing its *vocabulary* — the data structures that behavior moves around. The type shapes lived only in source, scattered across `packages/*/src/types.ts`, so understanding "what is a `Message`, a `SessionEvent`, a `StreamChunk`" meant reading the declarations directly. A prose catalog would help, but a catalog that paraphrases or paste-copies type definitions rots the instant a field changes — and an out-of-sync type doc is worse than none, because a reader trusts it.
|
||||
|
||||
So the work had two intertwined questions: **what belongs in such a catalog** (the scoping problem — a harness has dozens of cross-package types and dumping all of them helps no one), and **how to keep pasted type definitions from drifting** (the durability problem). This RFC records both decisions. Its sibling, [the generated cordis events + services catalog](2026-06-20-generated-cordis-catalog.md), is the *wiring*-axis complement: this one catalogs the data structures, that one the events and services that move them.
|
||||
|
||||
## Decision
|
||||
|
||||
A new `docs/core-data-structures/` folder catalogs the vocabulary, with a new `verify-type-equiv` doc-sync gate that keeps every pasted type definition byte-identical to its source.
|
||||
|
||||
### What counts as "core" — the spine-vs-seam line
|
||||
|
||||
The scoping line was not picked top-down; it was discovered by testing candidate definitions against concrete borderline types until one rule survived every case. The decisive test was `BashExecRequest`/`BashExecSpec`/`BashRunResult`: bash is a capability *seam*, not part of the agent-loop spine, so if those are "core" then "core" means *all cross-package vocabulary* and the catalog is a flat dump; if they are not, "core" means *the central spine* and bash vocabulary belongs on a sub-page. The latter won, which set the whole structure: a **tiered folder**, not a flat document.
|
||||
|
||||
The rule that settled the remaining cases: ***the type you write, hold, or receive is core; the machinery that types it, renders it, or persists it is a sub-page detail.*** Worked through:
|
||||
|
||||
- A data structure is **core** if it flows through the agent-loop spine — the loop holds, derives, streams, or logs it on every turn regardless of which plugins load (`Message`, `StreamChunk`, `SessionEvent`, the `Agent` handle) — **or** it is the single headline type a plugin author writes against a pipeline (`ToolDefinition`).
|
||||
- `ToolDefinition` is core (it is what every tool author writes) **even though the loop never holds one** — authoring-importance overrides the strict flows-through-spine rule for this one headline type. But its typing machinery — the `SchemaSpec`/`InferArgs` DSL — is a sub-page detail (you write a `ToolDefinition`; the type-level machinery that types it you do not). That is the spine-vs-seam line made sharp.
|
||||
- `ToolSchema` is core (it is a field of `GenerateOptions`, the model request that flows through every step) even though it is conceptually part of the tool pipeline — *flows through the spine* wins over *conceptual home* when they conflict.
|
||||
- The tool-presentation vocabulary (`ToolCallPresentation`, …, carrying a `FIXME(tool-presentation)` redesign marker), the `SessionPersistence` durability seam, and bash vocabulary are sub-pages.
|
||||
|
||||
`core.md` is a **self-contained spine doc**: it states the exact type definition of each spine structure with minimal prose and links to sub-pages for the per-seam detail. The sub-pages are `llm-streaming.md`, `session.md`, `persistence.md` (split from session along the in-memory-model vs. durability-seam line), `tools.md`, and `bash.md`.
|
||||
|
||||
### The `ts type-equiv` mechanism — literal AND drift-proof
|
||||
|
||||
The durability requirement was specific: the doc should show the **literal** current type definition (so a reader sees the real shape, not a paraphrase) **and** be mechanically guaranteed to match source. The repo already compiles fenced ` ```ts ` blocks (`doc-typecheck`), but a real typechecked block needs import noise and proves only *assignability*, not *byte-equality* — a renamed field with the same type would pass. So:
|
||||
|
||||
- Type definitions are pasted verbatim into a dedicated ` ```ts type-equiv ` fence. `doc-typecheck` recognizes the fence and skips it (a bare definition is not standalone-compilable), and **excludes it from the opt-out ratio** — it is a separately-checked category, not an unchecked sketch.
|
||||
- A new `scripts/verify-type-equiv.ts` extracts each block via the TypeScript parser and asserts a **verbatim source match** against the declared symbol — chosen over a compiled `_Check` assertion precisely because byte-equality, not assignability, is the property we want.
|
||||
- Provenance lives in a central `scripts/type-equiv.manifest.json` (`{ doc, symbol, source }` entries), **not** in directive comments in the prose. The script enforces a **1:1 correspondence**: every type-equiv block has exactly one manifest entry and vice versa, so a block can never be silently unchecked and an entry can never rot.
|
||||
- Wired into `doc-sync`, so it runs in the same lefthook pre-push and CI paths as the other doc gates.
|
||||
|
||||
### Maintenance is the author's job, with a gate backstop
|
||||
|
||||
`verify-type-equiv` catches a *drifted paste* of an already-documented type, but it cannot tell you a brand-new core type went undocumented. So AGENTS.md and the `dsh-code-review` skill were updated to require keeping the catalog in sync when a change adds or reshapes a documented type — the gate handles drift, the human handles new surface.
|
||||
|
||||
## Process
|
||||
|
||||
The design was driven entirely by a one-question-at-a-time grilling that walked the scoping decision tree through concrete examples (`BashExecRequest`, `ToolSchema`, `ToolDefinition`, the schema DSL, the presentation types, the session/persistence split) before committing to the spine-vs-seam rule — the rule was the *output* of the examples, not an a-priori axiom. The implementation landed as four commits mirroring the structure of the work: the gate (`e97f94b`), the catalog (`7e33c7b`), the maintenance-guard updates (`53e01a0`), and a review-fix commit (`6da7a0f`).
|
||||
|
||||
That last commit is why the process is worth recording: an independent Codex review (gpt-5.5:xhigh) found a real **scan-gap bug** — `verify-type-equiv` only scanned the docs the manifest named, so a type-equiv block added to an *unmanifested* doc was silently skipped, defeating the 1:1 guarantee in one direction. The fix scans every doc in the markdown scope and reports an unmanifested block as an orphan. The same review corrected a `SessionPersistence` surface-listing prose error (`has`/`delete`) and the `doc-sync` command summary. The bug is the point: a drift gate that silently skips part of its input is worse than no gate, and only an adversarial reader caught it.
|
||||
|
||||
This decision shipped in #71 **without** an RFC at the time — the judgment was that the `ts type-equiv` convention was small enough to document in `development.md`. This RFC is the retroactive record: the spine-vs-seam scoping rule and the verbatim-match-over-assignability choice are exactly the kind of "why was it done this way?" decisions a future maintainer would otherwise re-litigate, and its sibling catalog ([generated cordis events + services](2026-06-20-generated-cordis-catalog.md)) does carry an RFC, so the pair should be documented symmetrically.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The vocabulary now has a single home that **cannot silently drift**: a field rename in source fails `verify-type-equiv` in the pre-push hook and CI until the paste is refreshed.
|
||||
- The spine-vs-seam line is a reusable scoping tool, not a one-off: the same "the thing you write/hold/receive is core; the machinery that types/renders/persists it is a detail" rule is what later scoped the events/services catalog's harness-vs-inherited tiering.
|
||||
- The `ts type-equiv` fence is a third doc-block category alongside ` ```ts ` (compiled) and ` ```ts ignore-check ` (sketch). A later sibling added a fourth, ` ```ts cordis-catalog ` (generated signature), reusing the same skip-and-exclude treatment.
|
||||
- Adding or reshaping a core type now carries a documentation obligation the author must honor (the gate cannot detect a missing *new* type), backstopped by the `dsh-code-review` checklist.
|
||||
35
docs/rfc/implemented/2026-06-20-generated-cordis-catalog.md
Normal file
35
docs/rfc/implemented/2026-06-20-generated-cordis-catalog.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# RFC: Generated cordis events + services catalog
|
||||
|
||||
Status: implemented (accepted 2026-06-20)
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Context
|
||||
|
||||
A plugin author needs two reference surfaces that no single document gave them: every cordis **event** they can listen to (with its exact signature and dispatch mode) and every `ctx.<key>` **service** they can call (with its exact interface). The pieces existed but were scattered — a hand-maintained event-taxonomy *table* in `docs/architecture.md` (names + prose Mode/Purpose, name-set-checked by `verify-event-taxonomy`), a Service-map table (8 rows of role prose), and the `interface Events` / `interface Context` declarations themselves. The taxonomy table also could not catch a brand-new *undocumented* event: a name-set verifier only checks the names that are already in the table on both sides.
|
||||
|
||||
This is the wiring-axis complement to the [core-data-structures catalog](../../core-data-structures/core.md) ([its RFC](2026-06-20-core-data-structures-catalog.md)): that one catalogs the *data structures* the loop moves around (verified hand-pastes); this one catalogs the *events and services* that move them.
|
||||
|
||||
## Decision
|
||||
|
||||
Generate the catalog from source instead of hand-maintaining a table and verifying a subset.
|
||||
|
||||
`scripts/gen-cordis-catalog.ts` walks the `interface Events` and `interface Context` declarations (plus the service classes) with the TypeScript compiler API and emits `docs/cordis-catalog/events-and-services.md` — one `## Events` section (grouped by scope, each event rendered as signature + mode badge + its source JSDoc) and one `## Services` section (each `ctx.<key>` with its public method signatures + class JSDoc). It mirrors the `gen-module-graph` pattern exactly: `--write` regenerates, `--check` fails if the committed file is stale, output is deterministic (sorted), and the file is a build artifact that is never hand-edited. `verify-cordis-catalog` (the `--check`) runs inside `doc-sync`, so the freshness gate fires in the same lefthook pre-push and CI paths as every other doc gate.
|
||||
|
||||
Pure generation is correct here because the codebase is disciplined enough that the AST is the whole truth: every event/service name is a string literal that round-trips to a static declaration — there are no dynamically-named events and no runtime-only services. So a generated doc cannot be wrong, and it closes the undocumented-event gap structurally (generation enumerates source rather than checking a hand-written subset).
|
||||
|
||||
Specific choices:
|
||||
|
||||
- **`@mode` tag, cross-checked.** Each harness event's JSDoc carries an explicit `@mode emit|waterfall|parallel` tag; the generator hard-errors on a missing tag. Where the signature shape is conclusive — a trailing `next: () => …` parameter is structurally a waterfall — it asserts the tag agrees and hard-errors on a contradiction. The emit-vs-parallel distinction is not structurally visible (`session/flush` returns `Promise<void> | void` with no `next`), so it is trusted from the tag. The authoring rule lives in [AGENTS.md](../../../AGENTS.md).
|
||||
- **Tiered scope.** The harness tier (the 8 `@deepseek-ai/dsh-*` services + their events) is rendered in full from source. The inherited tier (cordis-core `ctx.on/emit/effect/provide/…` + the `internal/*` events + loader/hmr/timer) is pinned vendor source a plugin also sees; it is rendered tersely (name + one-line + source pointer) from a curated table in the generator, NOT walked from the vendor AST — the cordis-core `Context` mixes true ctx members with non-service fields (`root`, `baseUrl`, `logger`), and the vendor surface changes only on a deliberate vendor sync.
|
||||
- **Cross-links to the data-structure catalog.** A type name in a signature (`GenerateOptions`, `StreamChunk`, `ToolDefinition`, …) links to the core-data-structures page that documents it. The map is a small hand-curated const in the generator — NOT `type-equiv.manifest.json`, which documents the `…Map` symbols while signatures reference the derived union names, and lists a few symbols on two pages.
|
||||
- **A dedicated fence.** Signature blocks use a ` ```ts cordis-catalog ` info string that `doc-typecheck` recognizes and skips (a bare signature fragment is not standalone-compilable), excluded from the opt-out ratio — the same treatment `type-equiv` blocks get.
|
||||
|
||||
This **supersedes the event-taxonomy half** of [doc-sync enforcement](2026-06-11-doc-sync-enforcement.md): `verify-event-taxonomy` and its `docs/architecture.md` table are retired (the architecture.md heading stays, its body now points at the catalog; the Service-map role table stays as curated prose). The verify-don't-generate principle that RFC chose for the taxonomy is reversed *for this surface only* — the data here is mechanically complete, so generation is strictly stronger (full signatures, cannot drift, catches undocumented events) than a name-set check of a hand-table. doc-typecheck, verify-md-wrap, verify-md-links, and verify-type-equiv are unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The catalog cannot drift: a source change that the committed file doesn't reflect fails `verify-cordis-catalog` in the pre-push hook and CI. A new event with no `@mode` tag, or a tag that contradicts its signature, fails the generator outright.
|
||||
- Event prose now has a single home — the JSDoc at the declaration. Thin JSDoc yields a thin catalog entry, which pressures authors to document at the source (the generator is a forcing function for the AGENTS.md "every export has a semantic JSDoc" rule).
|
||||
- The inherited tier is hand-summarized, so a vendor sync that adds/renames a cordis-core event or `ctx` member needs a matching edit to the curated table in `gen-cordis-catalog.ts`. This is the deliberate cost of not walking pinned vendor source; it changes rarely and is called out in the generator.
|
||||
- `verify-event-taxonomy.ts` is deleted and the `docs/architecture.md` event table is gone; anyone who linked to a specific table row now lands on the generated catalog instead.
|
||||
Reference in New Issue
Block a user