Merge branch 'master' into docs/chinese-terminology
This commit is contained in:
Ziya
@@ -24,6 +24,7 @@ For a catalog of the **data structures** this architecture moves around — the
|
||||
│ @deepseek-ai/dsh-agent-loop (the ONE concrete plugin) │
|
||||
│ @deepseek-ai/dsh-bash-local (bash impl) │
|
||||
│ @deepseek-ai/dsh-tool-bash (bash tool schemas) │
|
||||
│ @deepseek-ai/dsh-subagent-* (subagent providers) │
|
||||
│ @deepseek-ai/dsh-session-persistence-jsonl (persistence impl)│
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ @deepseek-ai/dsh-agent (vocabulary + registry) │
|
||||
@@ -33,6 +34,8 @@ For a catalog of the **data structures** this architecture moves around — the
|
||||
│ @deepseek-ai/dsh-session-persistence (persistence seam) │
|
||||
│ @deepseek-ai/dsh-llm (abstract model service) │
|
||||
│ @deepseek-ai/dsh-bash (abstract bash executor) │
|
||||
│ @deepseek-ai/dsh-compact (abstract compaction seam) │
|
||||
│ @deepseek-ai/dsh-subagent (provider registry seam) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ vendor/: cordis, loader, include, group, timer, hmr, │
|
||||
│ logger-console, cosmokit, schemastery │
|
||||
@@ -53,6 +56,8 @@ Dependency rule: **extension** plugins depend on interface packages, never on `d
|
||||
| `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles + the create/resume factory seam (returns an `AgentHandle` = `{ agent, dispose() }` for owned per-agent teardown) |
|
||||
| `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `ReactLoopAgent`s and drives their loops |
|
||||
| `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks |
|
||||
| `ctx.compact` | `CompactService` (abstract) | dsh-compact | compaction seam: decide when history is too large, summarize an older range into a single surface node |
|
||||
| `ctx.subagents` | `SubagentService` | dsh-subagent | named provider registry for delegating a task to child agents |
|
||||
|
||||
All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically.
|
||||
|
||||
@@ -85,11 +90,11 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source
|
||||
- `user/message` → user message
|
||||
- `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation; an empty-content `assistant/message`, which exists only to host a max-tokens step's `usage`, is skipped too)
|
||||
- `tool/result` → user message carrying a `tool-result` block
|
||||
- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`<context source="…">…</context>`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: the real adapters now exist (the original precondition); the envelope still wants a deliberate review against live model behavior (`TODO(review)` in dsh-session).
|
||||
- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`<context source="…">…</context>`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. Live-adapter review has validated the tagged-envelope rendering against current DeepSeek behavior; provider-specific mismatches belong in that adapter.
|
||||
|
||||
Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`.
|
||||
|
||||
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
|
||||
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage, seed boundary) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
|
||||
|
||||
## Prompt assembly (dsh-system-prompt)
|
||||
|
||||
@@ -145,6 +150,7 @@ forever:
|
||||
session('assistant/message' {content, usage?}) log records what tool dispatch uses
|
||||
each tool-call (sequential, abort-checked between calls):
|
||||
session('tool/call'); ctx.tools.execute() ⟵ waterfall tools/execute
|
||||
tool execution may append tool-owned session events, e.g. `todo/write`
|
||||
session('tool/result')
|
||||
drain steering → session('steering/message'); emit agent/steering
|
||||
emit agent/step-end
|
||||
@@ -191,16 +197,16 @@ Every MVP feature (including the TODO-marked ones), with the mechanism that impl
|
||||
| `/loop` | on `agent/turn-end`, `send()` the next iteration; or force-continue |
|
||||
| Dynamic workflow | orchestrator plugin on `agent/turn-end` / `agent/step-end` driving `send`/`steer` (+ sub-agents later) |
|
||||
| Queued + steering messages | core `Agent.send()` / `Agent.steer()` |
|
||||
| Context compaction (auto + manual) | wrap `agent/request`: measure tokens, rewrite `req.messages`, append merged `compaction/*` session events; manual = a command plugin invoking the same routine |
|
||||
| Context compaction (auto + manual) | the `ctx.compact` seam ([dsh-compact](../packages/compact/compact)): a backend summarizes an older surface range into a single `user/message` `replace` op, bracketed by log-only `compact/*` events; auto = check token pressure at turn boundaries, manual = a `/compact` tool. See the [compaction capability-seam RFC](rfc/proposed/feature/2026-06-18-compaction-capability-seam.md) |
|
||||
| System prompt configurability | `ctx.systemPrompt.section()` with ordering |
|
||||
| AGENTS.md (root) | a section provider reading the file |
|
||||
| AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener |
|
||||
| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically. **Bash: implemented** — `dsh-bash` (seam) + `dsh-bash-local` (subprocesses) + `dsh-tool-bash` (`bash`/`bash_output`/`bash_kill`, incl. background tasks) |
|
||||
| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically. **Bash: implemented** — `dsh-bash` (seam) + `dsh-bash-local` (subprocesses) + `dsh-tool-bash` (`bash`/`bash_output`/`bash_kill`, incl. background tasks). **`todo_write`: implemented** — `dsh-tool-todo` writes the whole task list to the session log (`todo/write`), rendered as a stdio checklist / ACP `plan` |
|
||||
| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` |
|
||||
| Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute`, or implement a sandboxing `BashExecutor` (the dsh-bash seam) |
|
||||
| Permission system / AskUserQuestion | wrap `tools/execute` (veto or ask); register an ask tool |
|
||||
| Plan mode | wrap `tools/execute` (deny writes) + `agent/request` (inject mode prompt) |
|
||||
| Sub-agents (spawn / fork / steer) | TODO seam on `AgentLoop.create()`; fork = seed Session with parent events; `steer()` on the child handle |
|
||||
| Sub-agent delegation | Implemented as the `ctx.subagents` provider-registry seam: `dsh-subagent-spawn` starts a fresh in-process child, `dsh-subagent-fork` seeds a child from the parent's completed-turn prefix, `dsh-subagent-acp` drives an out-of-process child over ACP, and `dsh-tool-subagent` exposes one configured provider to the model |
|
||||
| MCP | one plugin per server: discover tools → `ctx.tools.register()` |
|
||||
| Skills | section + tool registration; `inject()` skill content on invocation |
|
||||
| Memory | section provider + tool |
|
||||
@@ -218,7 +224,7 @@ Code skeletons for the three plugin shapes (tool, hook/permission-gate, UI) and
|
||||
|
||||
Tracked here deliberately — each is designed-for but not implemented:
|
||||
|
||||
- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events.
|
||||
- **Inter-agent channels beyond delegation** (shared state, streaming child output, background/poll semantics) remain out of scope for the current `ctx.subagents` seam.
|
||||
- **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging.
|
||||
- **Parallel tool execution** (concurrency-safety hints on ToolDefinition).
|
||||
- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based forking.
|
||||
|
||||
@@ -16,7 +16,7 @@ packages/<group>/<pkg>/
|
||||
README.md # service API, events, extension points, design notes
|
||||
```
|
||||
|
||||
Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it.
|
||||
Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `compact`, `subagent`, `todo`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it.
|
||||
|
||||
package.json invariants (enforced by `pnpm run constraints` / `scripts/check-workspace-constraints.ts`): `private: true`, `version: 0.0.1`, `type: module`, `main: "lib/index.js"`, `types: "lib/types/index.d.ts"`, `exports["."].types: "./lib/types/index.d.ts"`, `exports["."].default: "./lib/index.js"`, `cordis` in BOTH peerDependencies and devDependencies (same range). Mirror every dsh peer dependency in devDependencies. `schemastery` goes in `dependencies` (it is a runtime validator), matching agent-loop. The `files` list is precise: `lib/index.js`, `lib/types/**/*.d.ts`, `lib/types/**/*.d.ts.map`, and `src`; do not publish `lib/types` JS or JS-map intermediates or stale root declaration files. CLI app packages with a package `bin` include `lib/bin.js` immediately after `lib/index.js` in `files`.
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary
|
||||
|
||||
## Events
|
||||
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 24 events across 6 scopes.
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto).
|
||||
|
||||
### `agent/*`
|
||||
|
||||
@@ -207,7 +207,7 @@ A session was created in the store.
|
||||
'session/created'(session: Session): void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:30`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:33`](../../packages/core/session/src/index.ts)
|
||||
|
||||
#### `session/event` — emit
|
||||
|
||||
@@ -219,7 +219,7 @@ An event was appended to a session log (sync, fire-and-forget). This is the per-
|
||||
|
||||
Types: [SessionEvent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:36`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:39`](../../packages/core/session/src/index.ts)
|
||||
|
||||
#### `session/flush` — parallel
|
||||
|
||||
@@ -229,7 +229,7 @@ Awaited durability checkpoint. The agent loop awaits `ctx.parallel('session/flus
|
||||
'session/flush'(session: Session): Promise<void> | void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:45`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:48`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `subagent/*`
|
||||
|
||||
@@ -301,7 +301,7 @@ Source: [`packages/core/tools/src/index.ts:43`](../../packages/core/tools/src/in
|
||||
|
||||
## Services
|
||||
|
||||
The 9 `ctx.<key>` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against.
|
||||
The `ctx.<key>` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against.
|
||||
|
||||
### `ctx.agentLoop` — `AgentLoop`
|
||||
|
||||
@@ -361,6 +361,24 @@ Types: [BashExecRequest](../core-data-structures/bash.md) · [BashExecSpec](../c
|
||||
|
||||
Source: [`packages/bash/bash/src/index.ts:59`](../../packages/bash/bash/src/index.ts)
|
||||
|
||||
### `ctx.compact` — `CompactService` (abstract seam)
|
||||
|
||||
Abstract compaction service. Subclass implement the two abstract methods, and load the subclass as a plugin — it registers as `ctx.compact` (one implementation per context; loading a second throws, which is cordis' standard duplicate-service behavior).
|
||||
|
||||
Both core methods are abstract: the contract states WHAT compaction does, while the entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
|
||||
Implementations MUST honor:
|
||||
|
||||
- **Surface contract**: a successful compaction shadows the compacted surface nodes with a SINGLE replacement node carrying the summary. Because `SurfaceEventType` is a closed union, that node is a `user/message` with `surfaceOp: { op:'replace', start, end }`; the `compact/*` events are log-only (lock + provenance).
|
||||
- **Blocking**: no compaction begins while another is in progress for the same session. The recommended mechanism is the log-recorded lock — append `compact/start` before the slow work and `compact/end` after (even on failure) — so the lock is visible to replay and crash recovery.
|
||||
|
||||
```ts cordis-catalog
|
||||
abstract compactIfNeeded( session: Session, systemPrompt?: string, model?: string, signal?: AbortSignal, ): Promise<CompactionResult | null>
|
||||
abstract compactRegion( session: Session, start: number, end: number, model: string, signal?: AbortSignal, ): Promise<CompactionResult>
|
||||
```
|
||||
|
||||
Source: [`packages/compact/compact/src/index.ts:57`](../../packages/compact/compact/src/index.ts)
|
||||
|
||||
### `ctx.llm` — `LlmService`
|
||||
|
||||
The abstract `llm` service: an adapter registry plus a streaming model-call surface, interceptable via the `llm/stream` waterfall.
|
||||
@@ -412,7 +430,7 @@ get(id: SessionId): Session | undefined
|
||||
list(): Session[]
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:229`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:321`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `ctx.subagents` — `SubagentService`
|
||||
|
||||
|
||||
46
docs/core-data-structures/compaction.md
Normal file
46
docs/core-data-structures/compaction.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Compaction
|
||||
|
||||
The compaction seam — a [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) split like bash: interface ([dsh-compact](../../packages/compact/compact), `ctx.compact`), implementation (a backend such as `dsh-compact-basic`, deferred), and consumer (a `/compact` tool, deferred). Compaction is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A tokenizer- or template-based backend is a sibling package implementing the same interface. Unlike bash, the interface necessarily depends on `dsh-session` and `dsh-llm`: its verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary (see the [compaction capability-seam RFC](../rfc/proposed/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
|
||||
Source: [`packages/compact/compact/src/types.ts`](../../packages/compact/compact/src/types.ts)
|
||||
|
||||
## The `compact/*` session events
|
||||
|
||||
Compaction extends [`SessionEventMap`](session.md) with three event types via declaration merging. All three are **log-only** — they record the compaction lock and its provenance, and never join the surface. `SurfaceEventType` is deliberately NOT extended (only message-producing events reach the model), so the summary itself rides on a separate `user/message` with `surfaceOp: { op: 'replace', start, end }` — the only surface mutation. See the RFC for why reusing `user/message` is honest rather than a workaround.
|
||||
|
||||
| Event | Payload | Role |
|
||||
|---|---|---|
|
||||
| `compact/start` | `{ turn }` | acquires the log-recorded lock |
|
||||
| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount }` | provenance: the summary blocks, the shadowed seq range, and the estimated token count |
|
||||
| `compact/end` | `{ turn, error? }` | releases the lock (`error` set when summarization threw) |
|
||||
|
||||
The lock brackets the **whole** operation: `compact/start` is appended first, then summarization, the `compact/summary` provenance record, and the `user/message` replacement all land, and only then `compact/end`. Releasing the lock last turns a crash mid-operation into a detectable orphaned lock (a `compact/start` with no matching `compact/end`) rather than a `compact/end` that falsely claims compaction finished.
|
||||
|
||||
These variants are merged inside a `declare module '@deepseek-ai/dsh-session'` block, so — unlike the top-level types on the other sub-pages — they are not pasted as a drift-checked ` ```ts type-equiv ` block (the `verify-type-equiv` extractor matches only top-level declarations by name). The payload table above is the catalog entry; follow the source link for the authoritative shapes.
|
||||
|
||||
## `CompactionResult`
|
||||
|
||||
What a successful compaction returns to its caller: the seqs of the three appended `compact/*` events, the summary blocks, and the shadowed range/seqs plus the estimated token count.
|
||||
|
||||
```ts type-equiv
|
||||
interface CompactionResult {
|
||||
/** The seq of the appended `compact/start` event. */
|
||||
startSeq: number
|
||||
/** The seq of the appended `compact/summary` event. */
|
||||
summarySeq: number
|
||||
/** The seq of the appended `compact/end` event. */
|
||||
endSeq: number
|
||||
/** The summary content blocks produced by the backend. */
|
||||
summary: ContentBlock[]
|
||||
/** The seq range that was shadowed [start, end] inclusive. */
|
||||
shadowedRange: { start: number; end: number }
|
||||
/** The seq numbers of all shadowed surface nodes. */
|
||||
shadowedSeqs: number[]
|
||||
/** Estimated token count of the shadowed content. */
|
||||
shadowedTokenCount: number
|
||||
}
|
||||
```
|
||||
|
||||
## The service
|
||||
|
||||
`CompactService` (`ctx.compact`, abstract — defined in [`packages/compact/compact/src/index.ts`](../../packages/compact/compact/src/index.ts)) declares two abstract methods: `compactIfNeeded(session, systemPrompt?, model?, signal?)` checks token pressure and compacts an older range if the history is too large (returning `null` when nothing needs it), and `compactRegion(session, start, end, model, signal?)` forcibly summarizes surface nodes `[start, end]` into a single replacement node. Both take an optional `signal: AbortSignal` that a backend summarizing via `ctx.llm.stream()` must forward into the call's `GenerateOptions.signal`, so an abort or dispose tears down the in-flight summarization. The entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
@@ -20,6 +20,7 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
|
||||
| [persistence.md](persistence.md) | the durability seam: `SessionPersistence`, JSONL + SQLite backends, `session/flush`, crash recovery, `SessionHeader` |
|
||||
| [tools.md](tools.md) | `ToolDefinition` full fields, the schema DSL, `ToolExecution`/`ToolResult`, tool-presentation UI types, the `tools/execute` waterfall |
|
||||
| [bash.md](bash.md) | the bash executor seam: `BashExecRequest`/`Spec`, `BashRunResult`, background `BashTask`s |
|
||||
| [compaction.md](compaction.md) | the compaction seam: the `compact/*` session events, `CompactionResult`, the `CompactService` interface |
|
||||
| [subagent.md](subagent.md) | the subagent seam: the named-provider registry, `SubagentStartRequest`/`Result`/`Run`, the start-time-vs-runtime capability split |
|
||||
|
||||
> Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts.
|
||||
@@ -200,11 +201,20 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
/** Unix epoch milliseconds. */
|
||||
time: number
|
||||
data: SessionEventMap[K]
|
||||
}
|
||||
} & (K extends SurfaceEventType ? {
|
||||
/**
|
||||
* Seq numbers of events that are provenance sources of this event
|
||||
* (e.g. the `assistant/chunk` seqs that built an `assistant/message`,
|
||||
* or the surface nodes shadowed by a compaction marker).
|
||||
*/
|
||||
sourceEventSeqs?: number[]
|
||||
/** How this event entered the surface; absent for non-surface events. */
|
||||
surfaceOp?: SurfaceOp
|
||||
} : object)
|
||||
}[T]
|
||||
```
|
||||
|
||||
The eleven event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
The twelve event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`, `todo/write`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
|
||||
## The agent handle
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ A backend that reloads a log crashed mid-turn finds an open `turn/start` with no
|
||||
|
||||
## `SessionHeader` — metadata beside the log
|
||||
|
||||
Per-session metadata travels **separately** from the event log: format version, cwd, and lineage are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`.
|
||||
Per-session metadata travels **separately** from the event log: format version, cwd, lineage, and the seed boundary are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`.
|
||||
|
||||
Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts)
|
||||
|
||||
@@ -78,6 +78,6 @@ Replay/fork is therefore `ctx.sessions.create(id, { seed: seedEvents })`; resumi
|
||||
Both implement the same abstract `SessionPersistence` (create/append/load/list over `SessionEvent`) and pass `runPersistenceContract`, proving the seam is genuinely backend-agnostic:
|
||||
|
||||
- **[dsh-session-persistence-jsonl](../../packages/session-persistence/session-persistence-jsonl)** — an append-only JSONL log per session with crash-safe atomic writes, the interrupted-turn crash recovery above, and a read/replay path.
|
||||
- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data)` maps 1:1 onto the event, so there is no parallel persisted schema to keep in sync.
|
||||
- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto the event, including optional surface metadata, so there is no parallel persisted schema to keep in sync.
|
||||
|
||||
Multiple backends sharing one on-disk session coordinate writes through the [shared persistence write-coordinator](../rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
@@ -6,7 +6,7 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t
|
||||
|
||||
## `SessionEventMap` — the event vocabulary
|
||||
|
||||
The append-only event types. Merge-extensible: a plugin (e.g. compaction) declares extra event types via declaration merging.
|
||||
The append-only event types. Merge-extensible: a plugin declares extra event types via declaration merging — e.g. the [compaction seam](compaction.md) adds `compact/start` / `compact/summary` / `compact/end`.
|
||||
|
||||
```ts type-equiv
|
||||
interface SessionEventMap {
|
||||
@@ -35,6 +35,31 @@ interface SessionEventMap {
|
||||
'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string } }
|
||||
/** Steering content injected between steps of a running turn. */
|
||||
'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource }
|
||||
/**
|
||||
* The agent's whole todo list, carried as a full snapshot and replaced
|
||||
* wholesale on each write — the current list is the most recent `todo/write`
|
||||
* (last-write-wins on replay, no fold). Appended by an owning agent via
|
||||
* `session.append('todo/write', { todos })`.
|
||||
*
|
||||
* NOT a {@link SurfaceEventType}: it produces no LLM message and never reaches
|
||||
* `deriveMessages()`, so it carries no `surfaceOp` and stays off the surface —
|
||||
* it is durable, replayable UI state, distinct from the conversation history.
|
||||
* It is a `SessionEventMap` member riding the existing `session/event` emit,
|
||||
* not a first-class Cordis `interface Events` notification, so it has no
|
||||
* cordis-catalog row.
|
||||
*/
|
||||
'todo/write': { todos: TodoItem[] }
|
||||
}
|
||||
```
|
||||
|
||||
### `TodoItem` — one todo-list entry
|
||||
|
||||
The unit of the `todo/write` event's whole-list snapshot. Deliberately minimal — a `content` line and a three-state `status` (no id, priority, or `activeForm`): the list is replaced wholesale on every write, so entries need no stable identity, and the status triple is exactly the ACP `PlanEntryStatus`, so a UI bridge can map a todo list onto an ACP `plan` 1:1 (synthesizing the priority ACP additionally requires). See the [todo_write RFC](../rfc/implemented/feature/2026-06-29-todo-write-tool.md).
|
||||
|
||||
```ts type-equiv
|
||||
export interface TodoItem {
|
||||
content: string
|
||||
status: 'pending' | 'in_progress' | 'completed'
|
||||
}
|
||||
```
|
||||
|
||||
@@ -51,12 +76,67 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
/** Unix epoch milliseconds. */
|
||||
time: number
|
||||
data: SessionEventMap[K]
|
||||
}
|
||||
} & (K extends SurfaceEventType ? {
|
||||
/**
|
||||
* Seq numbers of events that are provenance sources of this event
|
||||
* (e.g. the `assistant/chunk` seqs that built an `assistant/message`,
|
||||
* or the surface nodes shadowed by a compaction marker).
|
||||
*/
|
||||
sourceEventSeqs?: number[]
|
||||
/** How this event entered the surface; absent for non-surface events. */
|
||||
surfaceOp?: SurfaceOp
|
||||
} : object)
|
||||
}[T]
|
||||
```
|
||||
|
||||
`SessionEventType = keyof SessionEventMap`. Because `SessionEventMap` is merge-extensible, switches over `SessionEvent` must NOT use `assertNever` — a plugin-added variant is a valid unknown value; handle the known cases and fall through `default`.
|
||||
|
||||
## Surface types
|
||||
|
||||
The five message-producing types (`SurfaceEventType` — `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`) carry surface metadata declaring how they join the derived surface linked list. See the [session surface RFC](../rfc/implemented/architecture/2026-06-18-session-surface.md).
|
||||
|
||||
### `SurfaceEventType` — the message-producing subset of event types
|
||||
|
||||
```ts type-equiv
|
||||
export type SurfaceEventType =
|
||||
| 'user/message'
|
||||
| 'assistant/message'
|
||||
| 'tool/result'
|
||||
| 'context/message'
|
||||
| 'steering/message'
|
||||
```
|
||||
|
||||
### `SurfaceOp` — how an event entered the surface
|
||||
|
||||
```ts type-equiv
|
||||
export type SurfaceOp =
|
||||
| 'append'
|
||||
| { op: 'replace'; start: number; end: number }
|
||||
```
|
||||
|
||||
`'append'` is the normal tail-append path. `replace` shadows surface nodes from `start` through `end` inclusive (both must be valid surface node seqs; `start === end` replaces a single node) and inserts the new node in their place.
|
||||
|
||||
### `SurfaceIntent` — the parameter to `session.append()`
|
||||
|
||||
```ts type-equiv
|
||||
export interface SurfaceIntent {
|
||||
surfaceOp: SurfaceOp
|
||||
sourceEventSeqs?: number[]
|
||||
}
|
||||
```
|
||||
|
||||
Required for `SurfaceEventType` events — every message-producing event must declare how it joins the surface, the sole source of derived history. Non-surface types reject it at compile time.
|
||||
|
||||
### `SurfaceNode` — a node in the surface linked list
|
||||
|
||||
```ts type-equiv
|
||||
export interface SurfaceNode {
|
||||
seq: number
|
||||
prev: number | null
|
||||
next: number | null
|
||||
}
|
||||
```
|
||||
|
||||
## Derived history: `deriveMessages()`
|
||||
|
||||
`Session.deriveMessages()` projects the event log into the `Message[]` the model sees. The projection rules:
|
||||
|
||||
@@ -61,7 +61,7 @@ lefthook is configured in `lefthook.yml` as an early local checkpoint before rev
|
||||
|
||||
The vendor manifest guard checks that changes under `vendor/*/src` are staged with the matching `vendor/README.md` manifest update. See `vendor/README.md` before editing vendored code.
|
||||
|
||||
These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs an echo-agent smoke test and exercises the matrix on Node 24 and 26.
|
||||
These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs echo-agent and built-bin smoke tests and exercises the matrix on Node 24 and 26.
|
||||
|
||||
## CI gates
|
||||
|
||||
@@ -78,6 +78,7 @@ The GitHub workflow runs these gates on each pull request:
|
||||
- `pnpm run build`
|
||||
- `pnpm run hygiene`
|
||||
- an echo-agent smoke test that checks the demo's tool call, tool result, and JSONL output
|
||||
- built-bin smoke tests that run the published `lib/bin.js` entrypoints under plain `node`
|
||||
|
||||
`pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types`; CI also runs `pnpm run constraints` as an earlier fail-fast step, then runs the full hygiene script after `pnpm run build`.
|
||||
|
||||
@@ -121,6 +122,12 @@ The coding-agent demo uses the real DeepSeek adapter and needs `DEEPSEEK_API_KEY
|
||||
pnpm run demo:coding
|
||||
```
|
||||
|
||||
The ACP server demo exposes the same coding agent over JSON-RPC stdio and also needs `DEEPSEEK_API_KEY`:
|
||||
|
||||
```sh
|
||||
pnpm run demo:acp
|
||||
```
|
||||
|
||||
## TODO markers
|
||||
|
||||
Use one of three comment tags to flag known issues in the code, ordered by urgency:
|
||||
|
||||
@@ -18,6 +18,8 @@ graph TD
|
||||
agent --> brand
|
||||
agent --> llm
|
||||
agent --> session
|
||||
compact --> llm
|
||||
compact --> session
|
||||
llm-replay --> llm
|
||||
llm-replay --> session
|
||||
session-persistence --> session
|
||||
@@ -52,6 +54,9 @@ graph TD
|
||||
tool-bash --> bash
|
||||
tool-bash --> llm
|
||||
tool-bash --> tools
|
||||
tool-todo --> agent
|
||||
tool-todo --> session
|
||||
tool-todo --> tools
|
||||
agent-core --> agent
|
||||
agent-core --> agent-loop
|
||||
agent-core --> invariants
|
||||
@@ -101,6 +106,7 @@ graph TD
|
||||
| `session` | `brand`, `llm` |
|
||||
| `system-prompt` | `llm` |
|
||||
| `agent` | `brand`, `llm`, `session` |
|
||||
| `compact` | `llm`, `session` |
|
||||
| `llm-replay` | `llm`, `session` |
|
||||
| `session-persistence` | `session` |
|
||||
| `invariants` | `agent`, `llm`, `session` |
|
||||
@@ -112,6 +118,7 @@ graph TD
|
||||
| `agent-loop` | `agent`, `llm`, `session`, `session-persistence`, `system-prompt`, `tools` |
|
||||
| `subagent` | `agent`, `llm`, `tools` |
|
||||
| `tool-bash` | `agent`, `bash`, `llm`, `tools` |
|
||||
| `tool-todo` | `agent`, `session`, `tools` |
|
||||
| `agent-core` | `agent`, `agent-loop`, `invariants`, `llm`, `session`, `system-prompt`, `tool-bash`, `tools` |
|
||||
| `subagent-acp` | `agent`, `llm`, `subagent` |
|
||||
| `subagent-inprocess` | `agent`, `llm`, `session`, `subagent` |
|
||||
|
||||
@@ -44,6 +44,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Agent Client Protocol (ACP) support for external editors](proposed/feature/2026-06-14-acp-agent-client-protocol.md) | 2026-06-14 |
|
||||
| [Multiplex concurrent ACP sessions over one connection](proposed/feature/2026-06-14-acp-multi-session.md) | 2026-06-14 |
|
||||
| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/feature/2026-06-15-optional-code-mode.md) | 2026-06-15 |
|
||||
| [Compaction as a capability seam (abstract contract + basic backend)](proposed/feature/2026-06-18-compaction-capability-seam.md) | 2026-06-18 |
|
||||
|
||||
### Simplification
|
||||
|
||||
@@ -84,6 +85,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 |
|
||||
| [Subagent capability seam](implemented/feature/2026-06-21-subagent-capability-seam.md) | 2026-06-21 |
|
||||
| [ACP subagent backend (out-of-process delegation)](implemented/feature/2026-06-22-acp-subagent-backend.md) | 2026-06-22 |
|
||||
| [The `todo_write` tool — model task list as event-sourced session state](implemented/feature/2026-06-29-todo-write-tool.md) | 2026-06-29 |
|
||||
|
||||
### Simplification
|
||||
|
||||
@@ -114,6 +116,7 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Every session event is enclosed in a turn](implemented/architecture/2026-06-15-turn-enclosure-invariant.md) | 2026-06-15 |
|
||||
| [Shared persistence write coordinator](implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md) | 2026-06-18 |
|
||||
| [Agent lifecycle and ownership seams](implemented/architecture/2026-06-18-agent-lifecycle-and-ownership-seams.md) | 2026-06-18 |
|
||||
| [Session surface — a linked list over the event log for LLM message derivation](implemented/architecture/2026-06-18-session-surface.md) | 2026-06-18 |
|
||||
| [Reorganize packages into a modular hierarchy](implemented/architecture/2026-06-20-package-hierarchy.md) | 2026-06-20 |
|
||||
| [Branded IDs everywhere they belong](implemented/architecture/2026-06-20-branded-ids.md) | 2026-06-20 |
|
||||
| [Extract example apps into packages](implemented/architecture/2026-06-20-extract-example-app-packages.md) | 2026-06-20 |
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# RFC: Session surface — a linked list over the event log for LLM message derivation
|
||||
|
||||
Status: implemented (accepted 2026-06-18)
|
||||
|
||||
## Context
|
||||
|
||||
The `Session` event log is the single source of truth ([event-sourced sessions](2026-06-11-event-sourced-sessions.md)), but the only view over it was `deriveMessages()` — a linear scan that filtered and transformed raw events into `Message[]`. This creates problems for session-history-manipulating plugins (compaction, tool-call result pruning, etc.). Without a central mechanism, each plugin would need to wrap `agent/request` to rewrite the message list — a pattern that suffers from listener-ordering fragility, provides no durable record of what was changed, and forces repeated changes to the core `deriveMessages()` whenever a new manipulation is added. A central hub in the `session` package, with a provenance-recording mechanism and enough flexibility for future plugins to manipulate session history through a stable API, lays a solid foundation for plugin development.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a **surface** — a derived, cached linked list of "surface nodes" (the subset of events that produce LLM messages) — maintained by `surfaceOp` markers in the event log.
|
||||
|
||||
### Two new top-level fields on `SessionEvent`
|
||||
|
||||
Every `SessionEvent` gains two optional fields (structural metadata, like `seq`/`time`):
|
||||
|
||||
- **`sourceEventSeqs?: number[]`** — seq numbers of events that are provenance sources (e.g., the `assistant/chunk` seqs that built an `assistant/message`, or the surface nodes shadowed by a compaction marker). Provenance is a core design principle; without it, the replace-range operation cannot be validated on replay.
|
||||
- **`surfaceOp?: SurfaceOp`** — how this event entered the surface. Absent for non-surface events.
|
||||
|
||||
### SurfaceOp: two operations
|
||||
|
||||
```ts
|
||||
export type SurfaceOp =
|
||||
| 'append' // normal tail append
|
||||
| { op: 'replace'; start: number; end: number } // shadow [start, end] inclusive
|
||||
```
|
||||
|
||||
1. **Append** — add a new node to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`. The loop passes `surfaceOp: 'append'` on all such appends, and `sourceEventSeqs` where applicable (e.g., `assistant/message` records its `assistant/chunk` sources; `tool/result` records its `tool/call` source).
|
||||
|
||||
2. **Replace** — remove nodes from `start` through `end` (both inclusive) and insert a new node in their place. Both `start` and `end` must be valid surface node seqs in the current surface; `start === end` replaces a single node. The node's `sourceEventSeqs` must contain every shadowed surface node. The shadowed events remain in the log but are no longer on the surface.
|
||||
|
||||
The both-ends-inclusive design was chosen over half-open `[start, endExclusive)` because the surface is a doubly-linked list — both ends are naturally named by node seqs, and single-node replacement (`start === end`) is a common case that reads naturally with inclusive semantics.
|
||||
|
||||
### SurfaceManager: delta-based, not full rebuild
|
||||
|
||||
A `SurfaceManager` class (private to `Session`) maintains the cached linked list. It tracks `_lastProcessedSeq` and processes only the **delta** (new events since the last access) rather than rescanning the entire log. Because the log is append-only, prior events never change — full rebuild is only needed after a wholesale log replacement (e.g., seeding).
|
||||
|
||||
Why delta processing? The naive approach (a dirty flag + full rebuild on every access) would be O(N²) over a session's lifetime — every single-event append triggers a complete scan of all prior events. Delta processing is O(1) when no new events and O(new events) when new events arrive.
|
||||
|
||||
`deriveMessages()` uses the surface when surface markers exist, falling back to the existing linear scan for sessions without markers (backward compatibility).
|
||||
|
||||
### Persistence
|
||||
|
||||
The new fields are serialized as top-level JSON properties. The JSONL backend requires zero changes — `JSON.stringify`/`JSON.parse` preserve everything transparently. The SQLite backend's `events` table carries two nullable TEXT columns (`source_event_seqs`, `surface_op`). The on-disk `SCHEMA_VERSION` is bumped to reflect the column set, and — per the pre-release bump-and-reject policy — a database written by any other build is REJECTED on open rather than migrated (there is no persisted user data to upgrade). The session format `version` is pinned at `SESSION_FORMAT_VERSION = 0` (the "unstable / pre-release" stance): the optional surface fields are absorbed without bumping it.
|
||||
|
||||
### Crash recovery
|
||||
|
||||
The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls after a crash. These closers carry `surfaceOp: 'append'` and `sourceEventSeqs` pointing to the orphaned `tool/call` event, so the rehydrated surface is valid.
|
||||
|
||||
### Invariants
|
||||
|
||||
The dev-mode invariants plugin validates: `sourceEventSeqs` references (non-empty, no duplicates, references earlier events, references known seqs) and `surfaceOp` (replace `start ≤ end`, both endpoints are on the tracked surface, the range is non-reversed in surface position, and `sourceEventSeqs` includes every node the range shadows).
|
||||
|
||||
Because the surface is the SOLE derivation path, a surface-eligible event that carries no `surfaceOp` marker is invisible to `deriveMessages()` — it would land in the log yet silently drop from history on resume/fork. `append`'s typed overload makes the marker mandatory for `SurfaceEventType` events at compile time, but only when the type argument is a SPECIFIC literal; when it widens to the `SessionEventType` union (a caller iterating raw events, e.g. `for (const e of log) append(e.type, e.data)`) the conditional rest collapses to optional and the compiler stops enforcing it. The marker requirement is therefore ALSO checked at runtime in two places: `append` itself throws on a marker-less surface-eligible event (covering the union-widening loophole), and the `Session` seed constructor re-checks the same invariant (alongside its seq-contiguity and JSON-serializability checks) so a seed/load/fork — which arrives as raw `SessionEvent[]`, bypassing `append` — is REJECTED rather than constructing a session that resumes with missing history. (No backward-compat path for surface-less logs: per the pre-release stance there is no persisted user data to preserve, so such a log is rejected, not upgraded.)
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`packages/core/session`**: New `surface.ts` (`SurfaceManager`), new types (`SurfaceOp`, `SurfaceIntent`), new fields on `SessionEvent`, modified `append()` (third required `SurfaceIntent` param), refactored `deriveMessages()` (walks the surface as the sole derivation path), surface-aware `repair.ts`. The seed constructor rejects a surface-eligible seed event missing its `surfaceOp` marker (see § Invariants).
|
||||
- **`packages/core/agent-loop`**: All surface-capable appends pass surface opts. Chunk seqs are collected for `assistant/message` provenance; `tool/call` seqs are captured for `tool/result` provenance.
|
||||
- **`packages/session-persistence/session-persistence-sqlite`**: Two new nullable TEXT columns (`source_event_seqs`, `surface_op`) on the `events` table; `SCHEMA_VERSION` bumped (bump-and-reject, no migration).
|
||||
- **`packages/support/invariants`**: Surface-related validation rules.
|
||||
- **`packages/session-persistence/session-persistence-jsonl`**: No changes required.
|
||||
- **`packages/session-persistence/session-persistence`**: Abstract interface unchanged.
|
||||
|
||||
The surface is the foundation for future history manipulation. A compaction or tool-result-prune plugin appends one of the existing message-producing event types (a `user/message` carrying the summary, say) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed nodes — the new node takes the range's place on the surface while the plugin's own trace events (e.g. `compaction/start`, `compaction/end`) stay off it. Replay preserves the decision deterministically.
|
||||
58
docs/rfc/implemented/feature/2026-06-29-todo-write-tool.md
Normal file
58
docs/rfc/implemented/feature/2026-06-29-todo-write-tool.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# RFC: The `todo_write` tool — model task list as event-sourced session state
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness gives the model bash and subagent tools but no way to record a structured task list. A todo list serves two co-equal purposes: it steers the model to plan multi-step work and keep the active task unambiguous (at most one active, exactly one while work remains), and it gives the human a live progress checklist. The ACP protocol has a native `plan` sessionUpdate that editors (Zed) already render, but the bridge never emitted one. Every reference coding agent surveyed (claude-code, opencode, codex, oh-my-pi, pi) ships some form of this; the harness had nothing.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a model-facing `todo_write(todos: [{ content, status }])` tool whose whole-list state lives on the event-sourced session log as a new `todo/write` `SessionEventMap` variant. Both the stdio UI and the ACP bridge render off the existing `session/event` — the ACP bridge maps the list to a `plan` sessionUpdate.
|
||||
|
||||
### Whole-list replace, three-state status
|
||||
|
||||
The model sends the ENTIRE list every call; the new list replaces the old (last-write-wins on replay). This is the shape claude-code V1, opencode, and codex `update_plan` all use, and the shape the model is most trained on — no per-item ids, no delta protocol. `status` is exactly `pending | in_progress | completed`: the same triple as codex `update_plan` and, crucially, **identical to the ACP `PlanEntryStatus`**, so the bridge maps it 1:1 with no lossy translation.
|
||||
|
||||
### State on the session log, not a service
|
||||
|
||||
The list is appended as a `todo/write` event carrying the full `{ todos }` snapshot. The harness is event-sourced — the LLM history, tool calls, and turn structure all live on the log — so the todo list lives there too. This buys durability, replay, and `session/load` reconstruction for free: a reopened session re-derives the current list (the last `todo/write`) and the ACP bridge re-emits the `plan` on load, with no separate persistence backend, no in-memory service to rehydrate, and no extra wiring. An in-memory `ctx.todos` service would have had to reinvent all of that.
|
||||
|
||||
### NOT a surface event
|
||||
|
||||
`todo/write` is deliberately excluded from `SurfaceEventType`. The surface is the projection that produces the LLM message history (`deriveMessages()`); a todo write produces no conversation message. So it carries no `surfaceOp`, never joins the surface linked list, and never reaches `deriveMessages()` — it is durable, replayable *UI* state that travels alongside the conversation without being part of it. (The dev-mode invariants still require it to sit inside an open turn, which it always does: it is appended mid-step during a tool call.)
|
||||
|
||||
### Priority synthesized only at the ACP boundary
|
||||
|
||||
ACP's `PlanEntry` requires `content` + `priority` + `status`, but a `TodoItem` has no priority — the model never reasons about it. Rather than burden the schema with a field the model must always supply, the bridge synthesizes a constant `priority: 'medium'` on every entry when it builds the `plan`. Priority is an ACP wire requirement, not a harness concept, so it lives at exactly the boundary that needs it.
|
||||
|
||||
### Dropped vs claude-code V1: `activeForm`, id, priority
|
||||
|
||||
claude-code V1's item is `{ content, status, activeForm }`; later (V2) it grew ids, dependencies, and ownership — but only to support agent *swarms* (disk-backed, lock-guarded, per-item mutation). This tool keeps the item at the minimum: `{ content, status }`. No `activeForm` (the present-continuous label) — the UI shows `content`; no id — whole-list replace needs no stable identity; no priority — see above. Each dropped field is one less thing the model must produce on every call.
|
||||
|
||||
### Single owner — no swarm machinery (YAGNI)
|
||||
|
||||
The list belongs to the ONE agent session that called the tool (`exec.agent.session`); a non-agent caller is rejected. There is deliberately no shared/multi-owner scope, no capability seam (interface/impl/consumer), no scope resolver, and no delta protocol. The harness does have subagents, and a shared cross-agent list is conceivable — but building that now means designing for a form the product does not yet have. The whole-list-replace + single-owner shape is what claude-code V1, opencode, and codex all ship; if a shared list is ever needed, the on-log representation would change to per-item deltas (so concurrent writers can't clobber each other) and a scope resolver would choose the target log. That is a future RFC, not speculative scaffolding today.
|
||||
|
||||
### Validation: the cheap middle
|
||||
|
||||
The schema enforces type/required/enum. Beyond that, `execute` rejects empty or duplicate `content` and more than one `in_progress` task. claude-code leaves single-in-progress to the prompt; oh-my-pi enforces it in code. We take the middle: enforce the cheap invariants that make a plan *coherent* (no blank tasks, no dupes, at most one active), but leave ordering and the discipline of keeping the list current to the model via the tool description. A rejected write returns an `isError` result so the model self-corrects.
|
||||
|
||||
## Why no cordis-catalog entry / no `@mode`
|
||||
|
||||
`todo/write` is a member of `SessionEventMap`, not a first-class cordis `interface Events` event. The catalog generator (`scripts/gen-cordis-catalog.ts`) scans `interface Events` declarations; a `SessionEventMap` variant rides the existing `session/event` emit and produces no new catalog row. So it carries no `@mode` tag (which the generator requires only on `interface Events` members) — adding one would be meaningless.
|
||||
|
||||
## Testing
|
||||
|
||||
Four tiers, designed up front:
|
||||
- **Unit** — the session event (append/snapshot-clone/last-write-wins/not-on-surface); the tool (schema shape, arg validation via the real `ctx.tools.execute`, value validation, the event append + replacement, no-agent rejection, `presentCall`, HMR-safety); the ACP `todosToPlan` mapping; the stdio render arm.
|
||||
- **Real-Loader path** — the plugin run through `Loader.unwrapExports`, asserting the namespace export shape survives (it HAS `inject`, so a stray default would crash at load — postmortem/0001).
|
||||
- **Full-loop integration** — a scripted mock model calls `todo_write` through the real agent loop; the `todo/write` event lands and a second call replaces it.
|
||||
- **`session/load` replay** — a persisted `todo/write` re-emits the `plan` update when a fresh ACP bridge loads the session.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot golden gains the `plan` notification and the log event.
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
- **In-memory `ctx.todos` service** — would reinvent durability, replay, and `session/load` reconstruction the log gives for free.
|
||||
- **Per-item delta protocol** — only needed for a shared multi-owner list, which is out of scope; whole-list replace is simpler and matches the references.
|
||||
- **Tool in `core/`** — `todo_write` is an extension tool registering on `ctx.tools`, not part of the spine; it lives in its own `packages/todo/` group like other tool families.
|
||||
@@ -27,7 +27,7 @@ Record where a session's **inherited** prefix ends, persist it, and have the rep
|
||||
- **JSONL**: a `seedLength` field on the header line (`toHeaderLine`/`fromHeaderLine`).
|
||||
- **SQLite**: a `seed_length` column on the `sessions` table.
|
||||
|
||||
The SQLite change is a breaking table-layout change, so `SCHEMA_VERSION` bumps **2 → 3**. Per the repo's pre-release stance (§ "Pre-release stance" in AGENTS.md) the backend **rejects** a non-current `user_version` on open rather than migrating it — there is no persisted user data to preserve, so no migration code is written (the existing reject-not-migrate path at `openDatabase` already enforces this; v1 and now v2 are both rejected).
|
||||
The SQLite change is a breaking table-layout change, so `SCHEMA_VERSION` bumps. This branch added `seed_length` under version **3**; it later merged with the session-surface branch, which had independently shipped its OWN version-3 layout (the `source_event_seqs`/`surface_op` columns). Because an on-disk `3` is ambiguous between the two sibling layouts, the merged build is version **4** (every column), and an on-disk `3` is rejected like any other non-current version. Per the repo's pre-release stance (§ "Pre-release stance" in AGENTS.md) the backend **rejects** a non-current `user_version` on open rather than migrating it — there is no persisted user data to preserve, so no migration code is written (the existing reject-not-migrate path at `openDatabase` already enforces this; v1, v2, and the collided v3 are all rejected).
|
||||
|
||||
### 3. Replay derives a child script after the boundary
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# RFC: Compaction as a capability seam (abstract contract + basic backend)
|
||||
|
||||
Status: proposed (2026-06-18)
|
||||
|
||||
## Context
|
||||
|
||||
A long-running agent conversation grows without bound. As the event log accumulates turns, the derived message history eventually approaches the model's context window — the model then truncates mid-response (`max-tokens`) or degrades. **Compaction** is the mitigation: replace a run of older history with a concise summary, keeping recent context intact.
|
||||
|
||||
The [session surface](../../implemented/architecture/2026-06-18-session-surface.md) was built as the foundation for exactly this — a linked list over the event log with a `surfaceOp: { op: 'replace', start, end }` operation purpose-built to shadow a range of nodes and insert a replacement, with `sourceEventSeqs` recording provenance so the decision replays deterministically. What remained was the plugin that *decides what to compact and produces the summary*.
|
||||
|
||||
Two forces shape the design. First, compaction is **swappable**: token counting can be a char/4 heuristic or a real tokenizer, and summarization can be a model call, a template, or a remote service — these vary independently of *when* and *which range* to compact. Second, a later commit (`ce43c25`) closed `SurfaceEventType` to five event types (`user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`); only those may carry `surfaceOp`. A bespoke `compaction/*` event therefore **cannot** itself appear on the surface — the compiler rejects `surfaceOp` on it and the invariants plugin rejects it at runtime.
|
||||
|
||||
## Decision
|
||||
|
||||
### Compaction is a capability seam, split interface / implementation
|
||||
|
||||
Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently:
|
||||
|
||||
1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
|
||||
2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that owns the entire algorithm — token estimation (char/4 + per-block overhead), the tail→head retention walk, summarization via `ctx.llm.generate()`, the surface replacement, the lock, and the `agent/request` auto-compaction listener. A tokenizer-based or template-based backend is a sibling package (or a subclass overriding the two protected estimation/summarization hooks).
|
||||
3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
|
||||
|
||||
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
|
||||
|
||||
The capability-seams RFC states the interface package "depends only on cordis" (true of `dsh-bash`, whose vocabulary is self-contained). Compaction **cannot** honor that: its verbs are defined *over* a `Session` (`compactRegion(session, start, end)`) and its output *is* the content vocabulary (`CompactionResult.summary: ContentBlock[]`). There is no way to express the contract without naming `Session`/`SessionEvent` (from `dsh-session`) and `ContentBlock` (from `dsh-llm`).
|
||||
|
||||
This is not a coupling smell — it is the contract's domain. The "only cordis" guidance was always shorthand for "the interface depends only on what the contract genuinely names, and never on an implementation." `dsh-session` and `dsh-llm` are themselves interface/vocabulary packages, not implementations; `dsh-compact` still imports no backend. The seam's real invariant — *consumers and implementations evolve independently behind an abstract service* — holds intact. We record the deviation here so a future reader doesn't mistake it for an accident or "fix" it by smuggling `Session` behind an opaque handle.
|
||||
|
||||
### Abstract `compactIfNeeded` / `compactRegion`, algorithm in the backend
|
||||
|
||||
An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface, with only `estimateContentTokens()` and `summarize()` abstract. That recouples the contract to one strategy: a backend that wants a different retention policy (e.g. turn-count instead of token-budget) or a different event-sequencing would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend, where it belongs, and keeps the interface a pure statement of *what*. The backend remains internally factored — `estimateContentTokens()` and `summarize()` are `protected` hooks a sub-backend can override without reimplementing the walk — but that factoring is the backend's private concern, not the contract's.
|
||||
|
||||
### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary
|
||||
|
||||
Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the summary `ContentBlock[]` and whose `sourceEventSeqs` covers the shadowed nodes *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance), never on the surface. The surface mutation sits **inside** the lock — `compact/end` is the last event appended:
|
||||
|
||||
```
|
||||
compact/start → log-only. Acquires the lock.
|
||||
[summarize older range via the backend]
|
||||
compact/summary → log-only. Provenance: summary, range, shadowed seqs, token count.
|
||||
user/message → surfaceOp { op:'replace', start, end }. THE surface mutation.
|
||||
deriveMessages() renders it as a user-role message.
|
||||
compact/end → log-only. Releases the lock.
|
||||
```
|
||||
|
||||
Ordering the surface mutation **before** `compact/end` is deliberate: `session.append()` commits one event at a time, so there is no multi-event transaction to make the sequence atomic. Releasing the lock last converts the crash window from *silent corruption* (a `compact/end` that claims compaction finished while the surface was never shadowed) into a *detectable orphaned lock* (a `compact/start` with no matching `compact/end`), which a persistence backend already detects on reload. A `session/event` listener on `compact/end` likewise never sees the lock free before the replacement has landed.
|
||||
|
||||
`deriveMessages()` then yields `[summary_as_user_message, ...retained_nodes]`. An alternative — extending `SurfaceEventType` to admit a `compact/*` type — was rejected: the closed union is a deliberate safety boundary (only message-producing events reach the model), and a summary genuinely *is* user-role context, so reusing `user/message` is honest rather than a workaround.
|
||||
|
||||
### Blocking via a log-recorded lock, not a mutex
|
||||
|
||||
Compaction must be serialized: no second compaction starts before the first finishes, and no ordinary events interleave the slow summarization. Rather than an in-memory mutex (invisible to replay, lost on crash), the lock **is** the log: `compactRegion` refuses to start if the last `compact/start` has no matching `compact/end` after it. `compact/start` is appended first (fast, synchronous), the slow model call runs, then the `compact/summary` and `user/message` replacement land, and only then is `compact/end` appended — in a `catch` that records the error, so a failed summarization can never wedge the lock. Because the backend runs compaction synchronously inside the `agent/request` waterfall, the loop is single-threaded for that window; the lock additionally gives observability and lets a persistence backend detect an orphaned `compact/start` on reload.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **New packages**: `packages/compact/compact` (interface) and a sibling `compact-basic` (backend) under `packages/compact/`, wired into the three root tsconfigs. The consumer tier is deferred.
|
||||
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
|
||||
- **No changes** to `dsh-session`, `dsh-invariants`, or `dsh-agent-loop`: the surface replace op, the surface-metadata runtime guard, and the `agent/request` waterfall all already exist. Compaction is a pure plugin on documented seams.
|
||||
- The capability-seams convention gains a second reference beyond bash, and a documented case where "interface depends only on cordis" relaxes to "depends only on interface/vocabulary packages the contract genuinely names." On acceptance, [AGENTS.md](../../../../AGENTS.md) § Conventions and [architecture.md](../../../architecture.md) § "Capability seams" should note this relaxation.
|
||||
Reference in New Issue
Block a user