Merge remote-tracking branch 'origin/master' into fs-tool-clean
# Conflicts: # docs/architecture.md # docs/cordis-catalog/events-and-services.md # docs/module-graph.md # packages/README.md
This commit is contained in:
Tianyi Cui
@@ -27,6 +27,7 @@ For a catalog of the **data structures** this architecture moves around — the
|
||||
│ @deepseek-ai/dsh-fs-local (filesystem impl) │
|
||||
│ @deepseek-ai/dsh-file-context (filesystem policy gate) │
|
||||
│ @deepseek-ai/dsh-tool-fs (filesystem tools+executor)│
|
||||
│ @deepseek-ai/dsh-subagent-* (subagent providers) │
|
||||
│ @deepseek-ai/dsh-session-persistence-jsonl (persistence impl)│
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ @deepseek-ai/dsh-agent (vocabulary + registry) │
|
||||
@@ -37,6 +38,8 @@ For a catalog of the **data structures** this architecture moves around — the
|
||||
│ @deepseek-ai/dsh-llm (abstract model service) │
|
||||
│ @deepseek-ai/dsh-bash (abstract bash executor) │
|
||||
│ @deepseek-ai/dsh-fs (filesystem provider seam) │
|
||||
│ @deepseek-ai/dsh-compact (abstract compaction seam) │
|
||||
│ @deepseek-ai/dsh-subagent (provider registry seam) │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ vendor/: cordis, loader, include, group, timer, hmr, │
|
||||
│ logger-console, cosmokit, schemastery │
|
||||
@@ -60,6 +63,7 @@ Dependency rule: **extension** plugins depend on interface packages, never on `d
|
||||
| `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks |
|
||||
| `ctx.fs` | `FileSystem` (abstract) | dsh-fs | filesystem provider seam: path resolution, stat, text read/stream, atomic writes/edits (optional version guard); owns the `fs/*` policy events |
|
||||
| `ctx.compact` | `CompactService` (abstract) | dsh-compact | compaction seam: decide when history is too large, summarize an older range into a single surface node |
|
||||
| `ctx.subagents` | `SubagentService` | dsh-subagent | named provider registry for delegating a task to child agents |
|
||||
|
||||
All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically.
|
||||
|
||||
@@ -94,11 +98,11 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source
|
||||
- `user/message` → user message
|
||||
- `assistant/message` → assistant message (raw `assistant/chunk` events are replay/UI data and are skipped in derivation; an empty-content `assistant/message`, which exists only to host a max-tokens step's `usage`, is skipped too)
|
||||
- `tool/result` → user message carrying a `tool-result` block
|
||||
- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`<context source="…">…</context>`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. **TODO(review)**: the real adapters now exist (the original precondition); the envelope still wants a deliberate review against live model behavior (`TODO(review)` in dsh-session).
|
||||
- `context/message`, `steering/message` → user-role messages wrapped in a tagged envelope (`<context source="…">…</context>`) at their chronological position — the "system-reminder" pattern; models distinguish them from real user prompts by the envelope. Live-adapter review has validated the tagged-envelope rendering against current DeepSeek behavior; provider-specific mismatches belong in that adapter.
|
||||
|
||||
Replay/fork = `ctx.sessions.create(id, { seed: seedEvents })`. Trace/telemetry = listen to `session/event`.
|
||||
|
||||
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
|
||||
**Durability seam**: `session/event` is a synchronous notification; persistence plugins buffer (write-behind) and drain at the awaited `session/flush` checkpoint the loop fires at every turn end. The durable backend is a real **capability seam**: the abstract `SessionPersistence` service (`dsh-session-persistence`, `ctx.sessionPersistence`) defines create/append/load/list over the existing `SessionEvent` (no parallel persisted type), and `dsh-session-persistence-jsonl` is the first implementation — an append-only JSONL log per session with crash-safe atomic writes, crash recovery that PRESERVES an interrupted turn (closing it with a synthetic `turn/end {interrupted}` rather than truncating — a turn can be huge), and a read/replay path. Session metadata (format version, cwd, lineage, seed boundary) travels separately as `SessionHeader`, attached to a `Session` via `session.header`. Resuming a persisted session into a live agent is `ctx.agents.resume({ resumeSessionId })`. A second backend, `dsh-session-persistence-sqlite` (`node:sqlite`, one row per `SessionEvent` — the row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto it), passes the same `runPersistenceContract` suite, proving the seam is genuinely backend-agnostic.
|
||||
|
||||
## Prompt assembly (dsh-system-prompt)
|
||||
|
||||
@@ -141,10 +145,11 @@ forever:
|
||||
drain queued → 'turn/start' → session('user/message'…) → emit agent/turn-start
|
||||
STEP loop:
|
||||
drain steering (late steering from previous step's listeners)
|
||||
session('step/start'); emit agent/step-start
|
||||
assembly = ctx.systemPrompt.assemble() ⟵ waterfall system-prompt/assemble
|
||||
await ctx.serial('agent/pre-step') ⟵ surface mutation (compaction) OUTSIDE the step
|
||||
session('step/start'); emit agent/step-start
|
||||
req = {model, system, tools, messages: session.deriveMessages(), signal}
|
||||
req = waterfall agent/request ⟵ hooks, compaction, model switch
|
||||
req = waterfall agent/request ⟵ hooks, model switch
|
||||
stream ctx.llm.stream(req) ⟵ waterfall llm/stream (raw chunks)
|
||||
session('assistant/chunk'); emit agent/stream-chunk
|
||||
if assembler.finish is error/aborted: throw ⟵ adapter's in-band error path →
|
||||
@@ -154,6 +159,7 @@ forever:
|
||||
session('assistant/message' {content, usage?}) log records what tool dispatch uses
|
||||
each tool-call (sequential, abort-checked between calls):
|
||||
session('tool/call'); ctx.tools.execute() ⟵ waterfall tools/execute
|
||||
tool execution may append tool-owned session events, e.g. `todo/write`
|
||||
session('tool/result')
|
||||
drain steering → session('steering/message'); emit agent/steering
|
||||
emit agent/step-end
|
||||
@@ -200,16 +206,16 @@ Every MVP feature (including the TODO-marked ones), with the mechanism that impl
|
||||
| `/loop` | on `agent/turn-end`, `send()` the next iteration; or force-continue |
|
||||
| Dynamic workflow | orchestrator plugin on `agent/turn-end` / `agent/step-end` driving `send`/`steer` (+ sub-agents later) |
|
||||
| Queued + steering messages | core `Agent.send()` / `Agent.steer()` |
|
||||
| Context compaction (auto + manual) | the `ctx.compact` seam ([dsh-compact](../packages/compact/compact)): a backend summarizes an older surface range into a single `user/message` `replace` op, bracketed by log-only `compact/*` events; auto = check token pressure at turn boundaries, manual = a `/compact` tool. See the [compaction capability-seam RFC](rfc/proposed/feature/2026-06-18-compaction-capability-seam.md) |
|
||||
| Context compaction (auto + manual) | the `dsh-compact` seam (`ctx.compact`) + a backend (`dsh-compact-basic`) on the serial `agent/pre-step` seam: a backend summarizes an older surface range into a single `user/message` `replace` op, bracketed by log-only `compact/*` events; auto = check token pressure before each step — runaway-turn survival, manual = a (deferred) `/compact` tool invoking the same `ctx.compact` routine. See the [compaction capability-seam RFC](rfc/implemented/feature/2026-06-18-compaction-capability-seam.md) |
|
||||
| System prompt configurability | `ctx.systemPrompt.section()` with ordering |
|
||||
| AGENTS.md (root) | a section provider reading the file |
|
||||
| AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener |
|
||||
| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically. **Bash: implemented** — `dsh-bash` (seam) + `dsh-bash-local` (subprocesses) + `dsh-tool-bash` (`bash`/`bash_output`/`bash_kill`, incl. background tasks) |
|
||||
| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically. **Bash: implemented** — `dsh-bash` (seam) + `dsh-bash-local` (subprocesses) + `dsh-tool-bash` (`bash`/`bash_output`/`bash_kill`, incl. background tasks). **`todo_write`: implemented** — `dsh-tool-todo` writes the whole task list to the session log (`todo/write`), rendered as a stdio checklist / ACP `plan` |
|
||||
| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` |
|
||||
| Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute`, or implement a sandboxing `BashExecutor` (the dsh-bash seam) |
|
||||
| Permission system / AskUserQuestion | wrap `tools/execute` (veto or ask); register an ask tool |
|
||||
| Plan mode | wrap `tools/execute` (deny writes) + `agent/request` (inject mode prompt) |
|
||||
| Sub-agents (spawn / fork / steer) | TODO seam on `AgentLoop.create()`; fork = seed Session with parent events; `steer()` on the child handle |
|
||||
| Sub-agent delegation | Implemented as the `ctx.subagents` provider-registry seam: `dsh-subagent-spawn` starts a fresh in-process child, `dsh-subagent-fork` seeds a child from the parent's completed-turn prefix, `dsh-subagent-acp` drives an out-of-process child over ACP, and `dsh-tool-subagent` exposes one configured provider to the model |
|
||||
| MCP | one plugin per server: discover tools → `ctx.tools.register()` |
|
||||
| Skills | section + tool registration; `inject()` skill content on invocation |
|
||||
| Memory | section provider + tool |
|
||||
@@ -227,7 +233,7 @@ Code skeletons for the three plugin shapes (tool, hook/permission-gate, UI) and
|
||||
|
||||
Tracked here deliberately — each is designed-for but not implemented:
|
||||
|
||||
- **Sub-agent spawn/fork semantics** (seam: `AgentLoop.create()`); inter-agent channels beyond `send`/`steer`/events.
|
||||
- **Compaction implementation** (auto thresholds, summarization prompts) on the `agent/request` seam, with its session-event types added by declaration merging.
|
||||
- **Inter-agent channels beyond delegation** (shared state, streaming child output, background/poll semantics) remain out of scope for the current `ctx.subagents` seam.
|
||||
- **Compaction** — the `dsh-compact` seam (`ctx.compact`) and the `dsh-compact-basic` backend exist (auto thresholds, summarization on the serial `agent/pre-step` seam, `compact/*` session events via declaration merging). The model-facing `/compact` consumer tool is still deferred. See [the compaction capability-seam RFC](rfc/implemented/feature/2026-06-18-compaction-capability-seam.md).
|
||||
- **Parallel tool execution** (concurrency-safety hints on ToolDefinition).
|
||||
- **Session branching/tree** (pi-style entry tree) if needed beyond seed-based forking.
|
||||
|
||||
@@ -16,7 +16,7 @@ packages/<group>/<pkg>/
|
||||
README.md # service API, events, extension points, design notes
|
||||
```
|
||||
|
||||
Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it.
|
||||
Choose an existing group when one matches the package's role (`core`, `llm`, `bash`, `compact`, `subagent`, `todo`, `session-persistence`, `ui`, `util`, or `support`). A new group is allowed, but it is a pure container: no `package.json`, no source files, and packages still sit exactly one level below it.
|
||||
|
||||
package.json invariants (enforced by `pnpm run constraints` / `scripts/check-workspace-constraints.ts`): `private: true`, `version: 0.0.1`, `type: module`, `main: "lib/index.js"`, `types: "lib/types/index.d.ts"`, `exports["."].types: "./lib/types/index.d.ts"`, `exports["."].default: "./lib/index.js"`, `cordis` in BOTH peerDependencies and devDependencies (same range). Mirror every dsh peer dependency in devDependencies. `schemastery` goes in `dependencies` (it is a runtime validator), matching agent-loop. The `files` list is precise: `lib/index.js`, `lib/types/**/*.d.ts`, `lib/types/**/*.d.ts.map`, and `src`; do not publish `lib/types` JS or JS-map intermediates or stale root declaration files. CLI app packages with a package `bin` include `lib/bin.js` immediately after `lib/index.js` in `files`.
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ The **harness tier** below (the `@deepseek-ai/dsh-*` packages) is the vocabulary
|
||||
|
||||
## Events
|
||||
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out, no veto). The harness declares 27 events across 7 scopes.
|
||||
Dispatch modes: **emit** (fire-and-forget), **waterfall** (each listener gets `next()` and may transform or veto — see [waterfall semantics](../architecture.md#cordis-waterfall-semantics-important)), **parallel** (awaited fan-out; all listeners run), **serial** (awaited in registration order until one returns a bail value — anything other than `null`, `false`, or `undefined`).
|
||||
|
||||
### `agent/*`
|
||||
|
||||
@@ -49,7 +49,21 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:220`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:254`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/pre-step` — serial
|
||||
|
||||
Awaited pre-step surface-mutation checkpoint, fired once per step AFTER `turn/start` (and after the prior step closed) but BEFORE this step's `step/start` — so anything a listener appends lands OUTSIDE the step, between `turn/start`/`step/end` and the upcoming `step/start`. `step` is the number of the step about to start. The loop awaits `ctx.serial('agent/pre-step', …)` after assembling the system prompt, then opens the step and derives the request history ONCE from whatever the surface now holds. This is where compaction belongs: it mutates the session surface in place (shadowing an older range with a summary node) with its log-only `compact/*` records cleanly outside any step, and the single subsequent derive reflects the mutation — so there is no double-derive and no listener can see (or be expected to act on) an assembled `messages` array that does not exist yet.
|
||||
|
||||
Serial (awaited in registration order), not a waterfall: a listener mutates the surface as a side effect; there is nothing to transform, but the loop must wait for the mutation to complete before opening the step and deriving. Cordis `serial` bails early if a listener returns a bail value; this event is typed and documented as `void`, so listeners must not return a semantic veto value. `fullSystemPrompt` is the assembled prompt a listener needs to measure pressure (the system prompt counts toward the budget). `signal` cancels any in-flight work a listener starts (e.g. a summarization model call).
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/pre-step'(agent: Agent, turn: number, step: number, fullSystemPrompt: string, signal: AbortSignal): Promise<void> | void
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:214`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/queued` — emit
|
||||
|
||||
@@ -65,7 +79,7 @@ Source: [`packages/core/agent/src/types.ts:156`](../../packages/core/agent/src/t
|
||||
|
||||
#### `agent/request` — waterfall
|
||||
|
||||
Waterfall: mutate the fully-assembled GenerateOptions before the model call (hooks, compaction, model switching, tool filtering, …). Call `next()` to delegate, or return without it to short-circuit.
|
||||
Waterfall: mutate the fully-assembled GenerateOptions before the model call (hooks, model switching, tool filtering, …). Call `next()` to delegate, or return without it to short-circuit. For surface mutation that must precede history derivation (compaction), use agent/pre-step instead — by the time this fires, `options.messages` is already derived.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/request'(agent: Agent, turn: number, step: number, options: GenerateOptions, next: () => Promise<GenerateOptions>): Promise<GenerateOptions>
|
||||
@@ -73,7 +87,7 @@ Waterfall: mutate the fully-assembled GenerateOptions before the model call (hoo
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:189`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:223`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/status` — emit
|
||||
|
||||
@@ -97,7 +111,7 @@ Steering content was injected into a running turn.
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:214`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:248`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/step-end` — emit
|
||||
|
||||
@@ -121,7 +135,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:195`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:229`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/step-start` — emit
|
||||
|
||||
@@ -145,7 +159,7 @@ A raw StreamChunk arrived from the model (token-level UI/log feed).
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:209`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:243`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/turn-continuation` — waterfall
|
||||
|
||||
@@ -157,7 +171,7 @@ Waterfall: override the turn-continuation decision. The default (computed by the
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:202`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:236`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
#### `agent/turn-end` — emit
|
||||
|
||||
@@ -245,7 +259,7 @@ A session was created in the store.
|
||||
'session/created'(session: Session): void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:33`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:34`](../../packages/core/session/src/index.ts)
|
||||
|
||||
#### `session/event` — emit
|
||||
|
||||
@@ -257,7 +271,7 @@ An event was appended to a session log (sync, fire-and-forget). This is the per-
|
||||
|
||||
Types: [SessionEvent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:39`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:40`](../../packages/core/session/src/index.ts)
|
||||
|
||||
#### `session/flush` — parallel
|
||||
|
||||
@@ -267,7 +281,7 @@ Awaited durability checkpoint. The agent loop awaits `ctx.parallel('session/flus
|
||||
'session/flush'(session: Session): Promise<void> | void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:48`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:49`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `subagent/*`
|
||||
|
||||
@@ -339,7 +353,7 @@ Source: [`packages/core/tools/src/index.ts:43`](../../packages/core/tools/src/in
|
||||
|
||||
## Services
|
||||
|
||||
The 11 `ctx.<key>` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against.
|
||||
The `ctx.<key>` services the harness provides. An abstract seam (e.g. `ctx.bash`) is implemented by a separate package; the interface is what consumers code against.
|
||||
|
||||
### `ctx.agentLoop` — `AgentLoop`
|
||||
|
||||
@@ -411,11 +425,11 @@ Implementations MUST honor:
|
||||
- **Blocking**: no compaction begins while another is in progress for the same session. The recommended mechanism is the log-recorded lock — append `compact/start` before the slow work and `compact/end` after (even on failure) — so the lock is visible to replay and crash recovery.
|
||||
|
||||
```ts cordis-catalog
|
||||
abstract compactIfNeeded( session: Session, systemPrompt?: string, model?: string, signal?: AbortSignal, ): Promise<CompactionResult | null>
|
||||
abstract compactRegion( session: Session, start: number, end: number, model: string, signal?: AbortSignal, ): Promise<CompactionResult>
|
||||
abstract compactIfNeeded( agent: CompactAgentContext, turn: number, step: number, fullSystemPrompt: string, signal: AbortSignal, ): Promise<CompactionResult | null>
|
||||
abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, turn: number, step: number, signal?: AbortSignal, ): Promise<CompactionResult>
|
||||
```
|
||||
|
||||
Source: [`packages/compact/compact/src/index.ts:57`](../../packages/compact/compact/src/index.ts)
|
||||
Source: [`packages/compact/compact/src/index.ts:63`](../../packages/compact/compact/src/index.ts)
|
||||
|
||||
### `ctx.fs` — `FileSystem` (abstract seam)
|
||||
|
||||
@@ -493,7 +507,7 @@ get(id: SessionId): Session | undefined
|
||||
list(): Session[]
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:321`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:322`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `ctx.subagents` — `SubagentService`
|
||||
|
||||
@@ -560,7 +574,7 @@ The framework surface every plugin inherits, beyond the harness vocabulary above
|
||||
### Inherited `ctx` members
|
||||
|
||||
- `ctx.on / ctx.once` — Register an event listener (disposable). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts))
|
||||
- `ctx.emit / ctx.parallel / ctx.serial / ctx.bail / ctx.waterfall` — Dispatch an event (sync / awaited / first-non-nullish / veto-chain). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts))
|
||||
- `ctx.emit / ctx.parallel / ctx.serial / ctx.bail / ctx.waterfall` — Dispatch an event (sync / awaited / first-bail / veto-chain). ([`vendor/cordis/src/events.ts:29`](../../vendor/cordis/src/events.ts))
|
||||
- `ctx.plugin / ctx.inject` — Load a plugin / declare required services. ([`vendor/cordis/src/registry.ts:144`](../../vendor/cordis/src/registry.ts))
|
||||
- `ctx.effect` — Register a disposable side effect tied to the fiber. ([`vendor/cordis/src/fiber.ts:9`](../../vendor/cordis/src/fiber.ts))
|
||||
- `ctx.get / ctx.set / ctx.provide / ctx.accessor / ctx.mixin` — Low-level service-store access and binding. ([`vendor/cordis/src/reflect.ts:7`](../../vendor/cordis/src/reflect.ts))
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Compaction
|
||||
|
||||
The compaction seam — a [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) split like bash: interface ([dsh-compact](../../packages/compact/compact), `ctx.compact`), implementation (a backend such as `dsh-compact-basic`, deferred), and consumer (a `/compact` tool, deferred). Compaction is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A tokenizer- or template-based backend is a sibling package implementing the same interface. Unlike bash, the interface necessarily depends on `dsh-session` and `dsh-llm`: its verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary (see the [compaction capability-seam RFC](../rfc/proposed/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
The compaction seam — a [capability seam](../rfc/implemented/architecture/2026-06-13-capability-seams.md) split like bash: interface ([dsh-compact](../../packages/compact/compact), `ctx.compact`), implementation (a backend such as [dsh-compact-basic](../../packages/compact/compact-basic)), and consumer (a `/compact` tool, deferred). Compaction is **one optional capability**, not part of the agent-loop spine — so its vocabulary lives here, not in [core.md](core.md). A tokenizer- or template-based backend is a sibling package implementing the same interface. Unlike bash, the interface necessarily depends on `dsh-session` and `dsh-llm`: its verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary (see the [compaction capability-seam RFC](../rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)).
|
||||
|
||||
Source: [`packages/compact/compact/src/types.ts`](../../packages/compact/compact/src/types.ts)
|
||||
|
||||
@@ -11,7 +11,7 @@ Compaction extends [`SessionEventMap`](session.md) with three event types via de
|
||||
| Event | Payload | Role |
|
||||
|---|---|---|
|
||||
| `compact/start` | `{ turn }` | acquires the log-recorded lock |
|
||||
| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount }` | provenance: the summary blocks, the shadowed seq range, and the estimated token count |
|
||||
| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount }` | provenance: the summary blocks, the shadowed surface-boundary pair (`start`/`end` seqs — a position span, not a numeric interval), the shadowed seqs in surface order, and the estimated token count |
|
||||
| `compact/end` | `{ turn, error? }` | releases the lock (`error` set when summarization threw) |
|
||||
|
||||
The lock brackets the **whole** operation: `compact/start` is appended first, then summarization, the `compact/summary` provenance record, and the `user/message` replacement all land, and only then `compact/end`. Releasing the lock last turns a crash mid-operation into a detectable orphaned lock (a `compact/start` with no matching `compact/end`) rather than a `compact/end` that falsely claims compaction finished.
|
||||
@@ -32,9 +32,16 @@ interface CompactionResult {
|
||||
endSeq: number
|
||||
/** The summary content blocks produced by the backend. */
|
||||
summary: ContentBlock[]
|
||||
/** The seq range that was shadowed [start, end] inclusive. */
|
||||
/**
|
||||
* The surface-boundary pair that was shadowed: the seqs of the first
|
||||
* (`start`) and last (`end`) surface nodes of the replaced range. A
|
||||
* surface-POSITION span, not a numeric seq interval — after a prior replace
|
||||
* lands a fresh high-seq summary node at an older range's position, `start`
|
||||
* can be GREATER than `end`. {@link CompactionResult.shadowedSeqs} is the
|
||||
* authoritative set of shadowed nodes, in surface order.
|
||||
*/
|
||||
shadowedRange: { start: number; end: number }
|
||||
/** The seq numbers of all shadowed surface nodes. */
|
||||
/** The seqs of all shadowed surface nodes, in surface order. */
|
||||
shadowedSeqs: number[]
|
||||
/** Estimated token count of the shadowed content. */
|
||||
shadowedTokenCount: number
|
||||
@@ -43,4 +50,6 @@ interface CompactionResult {
|
||||
|
||||
## The service
|
||||
|
||||
`CompactService` (`ctx.compact`, abstract — defined in [`packages/compact/compact/src/index.ts`](../../packages/compact/compact/src/index.ts)) declares two abstract methods: `compactIfNeeded(session, systemPrompt?, model?, signal?)` checks token pressure and compacts an older range if the history is too large (returning `null` when nothing needs it), and `compactRegion(session, start, end, model, signal?)` forcibly summarizes surface nodes `[start, end]` into a single replacement node. Both take an optional `signal: AbortSignal` that a backend summarizing via `ctx.llm.stream()` must forward into the call's `GenerateOptions.signal`, so an abort or dispose tears down the in-flight summarization. The entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
`CompactService` (`ctx.compact`, abstract — defined in [`packages/compact/compact/src/index.ts`](../../packages/compact/compact/src/index.ts)) declares two abstract methods: `compactIfNeeded(agent, turn, step, fullSystemPrompt, signal)` checks token pressure and compacts an older range if the history is too large (returning `null` when nothing needs it), and `compactRegion(session, start, end, agent, turn, step, signal?)` forcibly summarizes surface nodes `[start, end]` into a single replacement node. `compactIfNeeded`'s parameters are all required — the loop's `agent/pre-step` checkpoint supplies the agent, lifecycle context, assembled `fullSystemPrompt`, and turn `signal`. A backend summarizing via `ctx.llm.stream()` must forward `signal` into the call's `GenerateOptions.signal`, so an abort or dispose tears down the in-flight summarization. The entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
|
||||
Auto-compaction runs on the serial `agent/pre-step` loop seam (fired once per step, after `turn/start` and BEFORE the step opens and its request history is derived), not the `agent/request` waterfall: compaction mutates the session surface in place — with its log-only `compact/*` records landing cleanly outside any step — and the loop derives the request from the already-compacted surface. Retention is turn-agnostic — the only structural guard is tool-pairing balance (a compacted region's edges are balanced cuts on the surface, so it never splits a step's tool-calls from their results), so a single runaway turn that alone exceeds the window compacts its own early closed steps rather than being retained verbatim. The backend that ships this (`dsh-compact-basic`) documents the retention walk, summary shrink validation, bounded re-compaction, and the crash/recoverable failure taxonomy.
|
||||
|
||||
@@ -206,7 +206,7 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
/**
|
||||
* Seq numbers of events that are provenance sources of this event
|
||||
* (e.g. the `assistant/chunk` seqs that built an `assistant/message`,
|
||||
* or the surface nodes shadowed by a compaction marker).
|
||||
* or the surface nodes shadowed by a compaction replace node).
|
||||
*/
|
||||
sourceEventSeqs?: number[]
|
||||
/** How this event entered the surface; absent for non-surface events. */
|
||||
@@ -215,7 +215,7 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
}[T]
|
||||
```
|
||||
|
||||
The eleven event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
The twelve event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`, `todo/write`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
|
||||
## The agent handle
|
||||
|
||||
@@ -307,7 +307,7 @@ interface Agent {
|
||||
}
|
||||
```
|
||||
|
||||
`AgentStatus` is `'idle' | 'running' | 'disposed'`. `AgentId` is a branded string. `AgentOptions` (`model?`, `systemPrompt?`) is merge-extensible — plugins add creation options by declaration merging. The `agent/*` event taxonomy (lifecycle, turn/step boundaries, the `agent/request`/`agent/step-result`/`agent/turn-continuation` waterfalls) is in [architecture.md § Event taxonomy](../architecture.md#event-taxonomy).
|
||||
`AgentStatus` is `'idle' | 'running' | 'disposed'`. `AgentId` is a branded string. `AgentOptions` (`model?`, `systemPrompt?`) is merge-extensible — plugins add creation options by declaration merging. The `agent/*` event taxonomy (lifecycle, turn/step boundaries, the serial `agent/pre-step` surface-mutation seam, and the `agent/request`/`agent/step-result`/`agent/turn-continuation` waterfalls) is in [architecture.md § Event taxonomy](../architecture.md#event-taxonomy).
|
||||
|
||||
## `ToolDefinition`
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ A backend that reloads a log crashed mid-turn finds an open `turn/start` with no
|
||||
|
||||
## `SessionHeader` — metadata beside the log
|
||||
|
||||
Per-session metadata travels **separately** from the event log: format version, cwd, and lineage are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`.
|
||||
Per-session metadata travels **separately** from the event log: format version, cwd, lineage, and the seed boundary are storage concerns, not conversation events, so they stay out of `SessionEventMap` and never reach `deriveMessages()`. The header is attached to a `Session` via `session.header`.
|
||||
|
||||
Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts)
|
||||
|
||||
@@ -78,6 +78,6 @@ Replay/fork is therefore `ctx.sessions.create(id, { seed: seedEvents })`; resumi
|
||||
Both implement the same abstract `SessionPersistence` (create/append/load/list over `SessionEvent`) and pass `runPersistenceContract`, proving the seam is genuinely backend-agnostic:
|
||||
|
||||
- **[dsh-session-persistence-jsonl](../../packages/session-persistence/session-persistence-jsonl)** — an append-only JSONL log per session with crash-safe atomic writes, the interrupted-turn crash recovery above, and a read/replay path.
|
||||
- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data)` maps 1:1 onto the event, so there is no parallel persisted schema to keep in sync.
|
||||
- **[dsh-session-persistence-sqlite](../../packages/session-persistence/session-persistence-sqlite)** — `node:sqlite`, one row per `SessionEvent`. The row shape `(session_id, seq, type, time, data, source_event_seqs, surface_op)` maps 1:1 onto the event, including optional surface metadata, so there is no parallel persisted schema to keep in sync.
|
||||
|
||||
Multiple backends sharing one on-disk session coordinate writes through the [shared persistence write-coordinator](../rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
@@ -35,6 +35,31 @@ interface SessionEventMap {
|
||||
'tool/result': { turn: number; step: number; callId: CallId; content: ContentBlock[]; isError: boolean; error?: { name: string; code: string } }
|
||||
/** Steering content injected between steps of a running turn. */
|
||||
'steering/message': { turn: number; content: ContentBlock[]; source: MessageSource }
|
||||
/**
|
||||
* The agent's whole todo list, carried as a full snapshot and replaced
|
||||
* wholesale on each write — the current list is the most recent `todo/write`
|
||||
* (last-write-wins on replay, no fold). Appended by an owning agent via
|
||||
* `session.append('todo/write', { todos })`.
|
||||
*
|
||||
* NOT a {@link SurfaceEventType}: it produces no LLM message and never reaches
|
||||
* `deriveMessages()`, so it carries no `surfaceOp` and stays off the surface —
|
||||
* it is durable, replayable UI state, distinct from the conversation history.
|
||||
* It is a `SessionEventMap` member riding the existing `session/event` emit,
|
||||
* not a first-class Cordis `interface Events` notification, so it has no
|
||||
* cordis-catalog row.
|
||||
*/
|
||||
'todo/write': { todos: TodoItem[] }
|
||||
}
|
||||
```
|
||||
|
||||
### `TodoItem` — one todo-list entry
|
||||
|
||||
The unit of the `todo/write` event's whole-list snapshot. Deliberately minimal — a `content` line and a three-state `status` (no id, priority, or `activeForm`): the list is replaced wholesale on every write, so entries need no stable identity, and the status triple is exactly the ACP `PlanEntryStatus`, so a UI bridge can map a todo list onto an ACP `plan` 1:1 (synthesizing the priority ACP additionally requires). See the [todo_write RFC](../rfc/implemented/feature/2026-06-29-todo-write-tool.md).
|
||||
|
||||
```ts type-equiv
|
||||
export interface TodoItem {
|
||||
content: string
|
||||
status: 'pending' | 'in_progress' | 'completed'
|
||||
}
|
||||
```
|
||||
|
||||
@@ -55,7 +80,7 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
/**
|
||||
* Seq numbers of events that are provenance sources of this event
|
||||
* (e.g. the `assistant/chunk` seqs that built an `assistant/message`,
|
||||
* or the surface nodes shadowed by a compaction marker).
|
||||
* or the surface nodes shadowed by a compaction replace node).
|
||||
*/
|
||||
sourceEventSeqs?: number[]
|
||||
/** How this event entered the surface; absent for non-surface events. */
|
||||
|
||||
@@ -61,7 +61,7 @@ lefthook is configured in `lefthook.yml` as an early local checkpoint before rev
|
||||
|
||||
The vendor manifest guard checks that changes under `vendor/*/src` are staged with the matching `vendor/README.md` manifest update. See `vendor/README.md` before editing vendored code.
|
||||
|
||||
These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs an echo-agent smoke test and exercises the matrix on Node 24 and 26.
|
||||
These hooks do not exactly mirror CI. Notably, `pre-push` runs unit tests without coverage, while CI runs `pnpm run test:coverage`; CI also runs echo-agent and built-bin smoke tests and exercises the matrix on Node 24 and 26.
|
||||
|
||||
## CI gates
|
||||
|
||||
@@ -78,6 +78,7 @@ The GitHub workflow runs these gates on each pull request:
|
||||
- `pnpm run build`
|
||||
- `pnpm run hygiene`
|
||||
- an echo-agent smoke test that checks the demo's tool call, tool result, and JSONL output
|
||||
- built-bin smoke tests that run the published `lib/bin.js` entrypoints under plain `node`
|
||||
|
||||
`pnpm run hygiene` is the local shorthand for `pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-node-next-types`; CI also runs `pnpm run constraints` as an earlier fail-fast step, then runs the full hygiene script after `pnpm run build`.
|
||||
|
||||
@@ -121,6 +122,12 @@ The coding-agent demo uses the real DeepSeek adapter and needs `DEEPSEEK_API_KEY
|
||||
pnpm run demo:coding
|
||||
```
|
||||
|
||||
The ACP server demo exposes the same coding agent over JSON-RPC stdio and also needs `DEEPSEEK_API_KEY`:
|
||||
|
||||
```sh
|
||||
pnpm run demo:acp
|
||||
```
|
||||
|
||||
## TODO markers
|
||||
|
||||
Use one of three comment tags to flag known issues in the code, ordered by urgency:
|
||||
|
||||
113
docs/i18n/terminology.md
Normal file
113
docs/i18n/terminology.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# Terminology
|
||||
|
||||
本表约定本仓库的中英术语统一译法。
|
||||
|
||||
| English | 中文 | 备注 |
|
||||
|---|---|---|
|
||||
| ACP | ACP | 首次出现可写：ACP（Agent Client Protocol） |
|
||||
| AI | AI | 首次出现可写：人工智能（AI） |
|
||||
| API | API | |
|
||||
| CLI | CLI | 首次出现可写：命令行界面（CLI） |
|
||||
| Cordis | Cordis | 保留英文 |
|
||||
| Function Calling | Function Calling | 首次出现可写：Function Calling（函数调用） |
|
||||
| HMR | HMR | 首次出现可写：热模块替换（HMR） |
|
||||
| JSON Schema | JSON Schema | |
|
||||
| JSONL | JSONL | |
|
||||
| lint | lint | |
|
||||
| loader | loader | |
|
||||
| LLM | LLM | 首次出现可写：大语言模型（LLM） |
|
||||
| MCP | MCP | |
|
||||
| RAG | RAG | 首次出现可写：检索增强生成（RAG） |
|
||||
| SDK | SDK | |
|
||||
| SSE | SSE | 首次出现可写：SSE（Server-Sent Events） |
|
||||
| agent | agent | 首次出现可写：agent（智能体） |
|
||||
| agent loop | agent loop | |
|
||||
| fiber | fiber | 首次出现可写：fiber（插件运行时） |
|
||||
| fixture | fixture | 指测试前置数据或环境 |
|
||||
| fork | fork | 保留英文 |
|
||||
| harness | harness | 保留英文 |
|
||||
| manifest | manifest | 描述模块或工具元数据的文件 |
|
||||
| schema DSL | schema DSL | |
|
||||
| schema | schema | 保留英文 |
|
||||
| seam | seam | 首次出现可写：seam（扩展点） |
|
||||
| skill | skill | 首次出现可写：skill（技能） |
|
||||
| spawn | spawn | 保留英文 |
|
||||
| steering | steering | 首次出现可写：steering（中途引导） |
|
||||
| subagent | subagent | 首次出现可写：subagent（子 agent） |
|
||||
| transcript | transcript | 首次出现可写：transcript（文本记录）；指会话渲染给用户或编辑器的完整文本，区别于事件日志（event log） |
|
||||
| waterfall | waterfall | 首次出现可写：waterfall（瀑布式事件） |
|
||||
| wire format | 协议格式 | 首次出现可写：协议格式（wire format） |
|
||||
| adapter contract | 适配器契约 | 首次出现可写：适配器契约（adapter contract） |
|
||||
| adapter | 适配器 | |
|
||||
| append-only | 仅追加 | |
|
||||
| artifact | 产物 | |
|
||||
| block | 块 | |
|
||||
| background task | 后台任务 | |
|
||||
| backend | 后端 | |
|
||||
| capability | 能力 | |
|
||||
| cancel | 取消 | |
|
||||
| checkpoint | 检查点 | |
|
||||
| chunk | 分片 | |
|
||||
| compaction | compaction | 首次出现可写：compaction（上下文压缩）；正文优先保留英文 |
|
||||
| consumer | 消费方 | |
|
||||
| content block | 内容块 | |
|
||||
| config | 配置 | |
|
||||
| context | 上下文 | |
|
||||
| context compaction | 上下文压缩 | 首次出现可写：上下文压缩（context compaction） |
|
||||
| coverage | 覆盖率 | |
|
||||
| crash recovery | 崩溃恢复 | |
|
||||
| dispose | dispose | 首次出现可写：dispose（释放资源）；正文优先保留英文 |
|
||||
| durability | 持久性 | |
|
||||
| event log | 事件日志 | |
|
||||
| event | 事件 | |
|
||||
| event stream | 事件流 | |
|
||||
| executor | 执行器 | |
|
||||
| extension | 扩展 | |
|
||||
| finish reason | 结束原因 | |
|
||||
| foreground run | 前台运行 | |
|
||||
| hook | 钩子 | |
|
||||
| implementation | 实现 | |
|
||||
| inference | 推理（inference） | 每次提及时保留英文括注，避免与 reasoning 混淆 |
|
||||
| injection | 注入 | |
|
||||
| interface | 接口 | |
|
||||
| integration | 集成 | |
|
||||
| memory | memory / 记忆 / 内存 | 按上下文区分：agent memory 译为“记忆”；resource/memory usage 译为“内存” |
|
||||
| message | 消息 | |
|
||||
| mod | 模组 | 区别于 module（模块）；plugin 译作「插件」 |
|
||||
| model provider | 模型提供方 | |
|
||||
| module | 模块 | |
|
||||
| permission | 权限 | |
|
||||
| persistence | 持久化 | |
|
||||
| pipeline | 流水线 | |
|
||||
| plugin | 插件 | mod 对应“模组” |
|
||||
| prompt | 提示词 | |
|
||||
| provider | 提供方 | |
|
||||
| provider-neutral | 提供方无关 | |
|
||||
| quality gate | 质量门禁 | |
|
||||
| registry | 注册表 | |
|
||||
| reasoning | 推理（reasoning） | 需要和 inference 区分时保留英文括注；`reasoning_content` 译为“思考内容” |
|
||||
| replay | 回放 | |
|
||||
| resume | 恢复 | |
|
||||
| runtime | 运行时 | |
|
||||
| sandbox | 沙箱 | |
|
||||
| service | 服务 | |
|
||||
| session | 会话 | |
|
||||
| session event | 会话事件 | |
|
||||
| snapshot | 快照 | |
|
||||
| spine | 主干 | |
|
||||
| step | 步骤 | |
|
||||
| stream | 流 | |
|
||||
| streaming | 流式输出 | |
|
||||
| system prompt | 系统提示词 | |
|
||||
| taxonomy | 分类体系 | |
|
||||
| token usage | token 用量 | |
|
||||
| thinking | thinking | API 字段保留；模型模式译为“思考” |
|
||||
| tool | 工具 | |
|
||||
| tool call | 工具调用 | |
|
||||
| tool result | 工具结果 | |
|
||||
| tool schema | 工具 schema | |
|
||||
| toolkit | 工具包 | |
|
||||
| turn | 轮次 | |
|
||||
| typecheck | 类型检查 | |
|
||||
| vocabulary | 词汇 | |
|
||||
| workflow | 工作流 | |
|
||||
@@ -27,6 +27,10 @@ graph TD
|
||||
llm-replay --> llm
|
||||
llm-replay --> session
|
||||
session-persistence --> session
|
||||
compact-basic --> agent
|
||||
compact-basic --> compact
|
||||
compact-basic --> llm
|
||||
compact-basic --> session
|
||||
invariants --> agent
|
||||
invariants --> llm
|
||||
invariants --> session
|
||||
@@ -62,6 +66,9 @@ graph TD
|
||||
tool-fs --> llm
|
||||
tool-fs --> system-prompt
|
||||
tool-fs --> tools
|
||||
tool-todo --> agent
|
||||
tool-todo --> session
|
||||
tool-todo --> tools
|
||||
agent-core --> agent
|
||||
agent-core --> agent-loop
|
||||
agent-core --> invariants
|
||||
@@ -117,6 +124,7 @@ graph TD
|
||||
| `fs-local` | `fs` |
|
||||
| `llm-replay` | `llm`, `session` |
|
||||
| `session-persistence` | `session` |
|
||||
| `compact-basic` | `agent`, `compact`, `llm`, `session` |
|
||||
| `invariants` | `agent`, `llm`, `session` |
|
||||
| `session-persistence-jsonl` | `session`, `session-persistence` |
|
||||
| `session-persistence-sqlite` | `session`, `session-persistence` |
|
||||
@@ -127,6 +135,7 @@ graph TD
|
||||
| `subagent` | `agent`, `llm`, `tools` |
|
||||
| `tool-bash` | `agent`, `bash`, `llm`, `tools` |
|
||||
| `tool-fs` | `fs`, `llm`, `system-prompt`, `tools` |
|
||||
| `tool-todo` | `agent`, `session`, `tools` |
|
||||
| `agent-core` | `agent`, `agent-loop`, `invariants`, `llm`, `session`, `system-prompt`, `tool-bash`, `tools` |
|
||||
| `subagent-acp` | `agent`, `llm`, `subagent` |
|
||||
| `subagent-inprocess` | `agent`, `llm`, `session`, `subagent` |
|
||||
|
||||
@@ -44,7 +44,6 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
| [Agent Client Protocol (ACP) support for external editors](proposed/feature/2026-06-14-acp-agent-client-protocol.md) | 2026-06-14 |
|
||||
| [Multiplex concurrent ACP sessions over one connection](proposed/feature/2026-06-14-acp-multi-session.md) | 2026-06-14 |
|
||||
| [Optional Code Mode — model writes TypeScript against an SDK of all tools](proposed/feature/2026-06-15-optional-code-mode.md) | 2026-06-15 |
|
||||
| [Compaction as a capability seam (abstract contract + basic backend)](proposed/feature/2026-06-18-compaction-capability-seam.md) | 2026-06-18 |
|
||||
|
||||
### Simplification
|
||||
|
||||
@@ -84,8 +83,10 @@ Do NOT write one for a mechanical or local choice (a variable name, a one-file r
|
||||
|---|---|
|
||||
| [Filesystem tool schemas — model-facing read/write/edit shapes](implemented/feature/2026-06-17-filesystem-tool-schemas.md) | 2026-06-17 |
|
||||
| [Rich ACP bash rendering — the terminal card (`_meta`) and command classification](implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md) | 2026-06-18 |
|
||||
| [Compaction as a capability seam (abstract contract + basic backend)](implemented/feature/2026-06-18-compaction-capability-seam.md) | 2026-06-18 |
|
||||
| [Subagent capability seam](implemented/feature/2026-06-21-subagent-capability-seam.md) | 2026-06-21 |
|
||||
| [ACP subagent backend (out-of-process delegation)](implemented/feature/2026-06-22-acp-subagent-backend.md) | 2026-06-22 |
|
||||
| [The `todo_write` tool — model task list as event-sourced session state](implemented/feature/2026-06-29-todo-write-tool.md) | 2026-06-29 |
|
||||
|
||||
### Simplification
|
||||
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# RFC: Compaction as a capability seam (abstract contract + basic backend)
|
||||
|
||||
Status: implemented (2026-06-18; retention/seam reform 2026-06-26)
|
||||
|
||||
## Context
|
||||
|
||||
A long-running agent conversation grows without bound. As the event log accumulates turns, the derived message history eventually approaches the model's context window — the model then truncates mid-response (`max-tokens`) or degrades. **Compaction** is the mitigation: replace a run of older history with a concise summary, keeping recent context intact.
|
||||
|
||||
The [session surface](../../implemented/architecture/2026-06-18-session-surface.md) was built as the foundation for exactly this — a linked list over the event log with a `surfaceOp: { op: 'replace', start, end }` operation purpose-built to shadow a range of nodes and insert a replacement, with `sourceEventSeqs` recording provenance so the decision replays deterministically. What remained was the plugin that *decides what to compact and produces the summary*.
|
||||
|
||||
Two forces shape the design. First, compaction is **swappable**: token counting can be a char/4 heuristic or a real tokenizer, and summarization can be a model call, a template, or a remote service — these vary independently of *when* and *which range* to compact. Second, a later commit (`ce43c25`) closed `SurfaceEventType` to five event types (`user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`); only those may carry `surfaceOp`. A bespoke `compaction/*` event therefore **cannot** itself appear on the surface — the compiler rejects `surfaceOp` on it and the invariants plugin rejects it at runtime.
|
||||
|
||||
## Decision
|
||||
|
||||
### Compaction is a capability seam, split interface / implementation
|
||||
|
||||
Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently:
|
||||
|
||||
1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
|
||||
2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that owns the entire algorithm — token estimation (char/4 + per-block overhead), the tail→head retention walk, summarization via `ctx.llm.stream()`, the surface replacement, the lock, and the `agent/pre-step` auto-compaction listener. A tokenizer-based or template-based backend is a sibling package (or a subclass overriding the two protected estimation/summarization hooks).
|
||||
3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
|
||||
|
||||
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
|
||||
|
||||
The capability-seams RFC states the interface package "depends only on cordis" (true of `dsh-bash`, whose vocabulary is self-contained). Compaction **cannot** honor that: its verbs are defined *over* a `Session` (`compactRegion(session, start, end)`) and its output *is* the content vocabulary (`CompactionResult.summary: ContentBlock[]`). There is no way to express the contract without naming `Session`/`SessionEvent` (from `dsh-session`) and `ContentBlock` (from `dsh-llm`).
|
||||
|
||||
This is not a coupling smell — it is the contract's domain. The "only cordis" guidance was always shorthand for "the interface depends only on what the contract genuinely names, and never on an implementation." `dsh-session` and `dsh-llm` are themselves interface/vocabulary packages, not implementations; `dsh-compact` still imports no backend. The seam's real invariant — *consumers and implementations evolve independently behind an abstract service* — holds intact.
|
||||
|
||||
### Abstract `compactIfNeeded` / `compactRegion`, algorithm in the backend
|
||||
|
||||
An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface, with only `estimateContentTokens()` and `summarize()` abstract. That recouples the contract to one strategy: a backend that wants a different retention policy or a different event-sequencing would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend, where it belongs, and keeps the interface a pure statement of *what*. The backend remains internally factored — `estimateContentTokens()` and `summarize()` are `protected` hooks a sub-backend can override without reimplementing the walk — but that factoring is the backend's private concern, not the contract's.
|
||||
|
||||
`compactIfNeeded(agent, turn, step, fullSystemPrompt, signal)` takes **required** parameters (not the original all-optional shape). The auto-compaction seam (below) always supplies the agent, lifecycle context, assembled system prompt (counted toward the estimate), and the turn's abort signal, so optionality would only invite a hidden default at the seam. The session being compacted comes from the agent context. `compactRegion(session, start, end, agent, turn, step, signal?)` keeps an optional signal (a manual caller may omit it). Passing lifecycle context rather than a concrete model keeps router agents honest: the backend's summarization request can run through `agent/request`, where model-routing plugins already choose the actual model.
|
||||
|
||||
### Auto-compaction runs on `agent/pre-step`, a dedicated surface-mutation seam
|
||||
|
||||
Compaction is a **surface mutation**, not a request transform — and that distinction is the seam it belongs on. The loop's request lifecycle, per step, is: assemble the system prompt → open the step → derive the message history from the surface → run the `agent/request` waterfall → call the model. An earlier cut wedged compaction into the `agent/request` waterfall, which forced two problems: (1) the loop had already derived `messages` from the *stale* surface, so the listener had to mutate the surface and then *re-derive* and overwrite `request.messages` — a double-derive whose only purpose was to undo the premature first derive; and (2) `agent/request` also carries downstream-injected context a listener might have added to `request.messages`, which compaction cannot act on (it can only compact the surface), inviting the confusion of measuring tokens compaction can't shed.
|
||||
|
||||
The fix is a dedicated loop seam, **`agent/pre-step`** (`@mode serial`), fired by the loop *after* system assembly and *before* the step opens (`step/start`):
|
||||
|
||||
```
|
||||
assembly = ctx.systemPrompt.assemble()
|
||||
await ctx.serial('agent/pre-step', agent, turn, step, system, signal) ⟵ compaction mutates the surface here
|
||||
session('step/start') ⟵ the step opens AFTER the seam
|
||||
messages = session.deriveMessages() ⟵ single derive, reflects the compaction
|
||||
request = waterfall agent/request ⟵ pure request transform (hooks, model switch)
|
||||
```
|
||||
|
||||
This makes the layering correct *by construction*: compaction mutates the surface, the loop derives **once** from the result (no double-derive), and at `pre-step` the assembled `messages` do not yet exist — so a listener structurally *cannot* see or be expected to act on downstream-injected context. `agent/request` reverts to a pure request transformer. Firing the seam **before** `step/start` (not inside the open step) is load-bearing for crash-safety: compaction's log-only `compact/*` records and its replacement node land *outside* any step, so the honest log structure a crash leaves (a dangling `compact/start` sitting before the synthetic `turn/end` that turn-repair appends) holds without a half-open step to reconcile. The seam is `serial` (awaited, in registration order), not `parallel`: a listener mutates the surface as a side effect — there is nothing to transform or return — and serial isolates listeners from each other so two surface-mutating listeners can never interleave their `session.append`s. Cordis `serial` does bail early if a listener returns a bail value, so `agent/pre-step` listeners are typed/documented to return `void` and must not use that bail channel as a semantic veto surface.
|
||||
|
||||
This **amends** the original RFC's claim of "NO changes to `dsh-agent-loop`; compaction is a pure plugin." That claim was load-bearing for a wrong design — reusing `agent/request` was the mistake. Per the pre-release "foundation over blast radius" stance, adding the correct seam (one event declaration in `dsh-agent`, one awaited emit in the loop) beats preserving a no-change boast that locked in the double-derive.
|
||||
|
||||
### Retention is turn-agnostic; tool-pairing balance is the only structural guard
|
||||
|
||||
Auto-compaction fires before **every** step, not once per turn. This is **load-bearing for runaway-turn survival**: a tool-heavy ReAct turn appends an `assistant/message` + a `tool/result` per step, so the surface grows *within* a turn. A single turn can grow past the window on its own (a "runaway turn") — and the only moment to rescue it before the next model call overflows is the next step's `pre-step` checkpoint. Gating compaction to a turn's first step (or, worse, retaining the whole in-flight turn verbatim) re-opens exactly the hole compaction exists to close: the harness would die when compaction is most needed.
|
||||
|
||||
So retention does **not** protect the in-flight turn, and turn boundaries play no role in it. `compactIfNeeded` walks the surface nodes tail→head, summing per-node token estimates, and retains the smallest tail-run of **whole units** whose total reaches `retainTokens`; everything older is compacted (head-anchored — see below). A *unit* is either a whole closed step (its `assistant/message` plus its `tool/result`s) or a single no-step node (a pre-step `user/message`, inter-step `steering/message`, or injection `context/message`). The walk rounds toward retaining *more*: when the raw token cutoff lands mid-step, it extends the retained side head-ward until the cut before the retained node is **tool-pairing balanced**. The single structural guard is therefore **tool-pairing balance** — a region's edges are balanced cuts on the *surface* (no unanswered `tool-call` crosses either edge), so a compacted region never splits a step's tool-calls from their `tool/result`s (which would produce a transcript every provider rejects). The check is decided over the surface linked list, **not** the log's `step/*` markers: a compaction lands a replacement node at a high log seq whose surface position is the head, so a log-position scan mis-reads its neighbours — `dsh-session` exports `isToolPairingBalanced(nodes, events, beforeSeq)` for the surface-anchored check. `compactRegion` enforces it strictly, throwing on a boundary that would split a step.
|
||||
|
||||
A runaway turn thus compacts exactly like any other history: its early *closed* steps get summarized while its recent steps stay verbatim. When the only compactable content left is an un-splittable open tail step (its tool-calls have no results yet), compaction declines (`null`) and retries once that step closes.
|
||||
|
||||
**Single-unit overflow is out of scope, by design.** If a single retained unit — one closed step, or a large free node such as a pasted `user/message` — *alone* exceeds the budget, compaction cannot help and the next model call may go out over-budget. Bounding an individual unit's size is a separate concern (output truncation), handled elsewhere; compaction makes no promise about it, and the harness without such a mechanism can still break on a single oversized unit. This is named honestly rather than papered over.
|
||||
|
||||
### Head-anchoring: one auto checkpoint, always at the head
|
||||
|
||||
`compactIfNeeded` always anchors the compacted range at the surface **head** (`nodes[0]`). After a first compaction lands a summary node at the head, the *second* compaction's range starts at that summary node and re-summarizes it together with the steps accumulated since — so the surface holds **at most one** auto-generated checkpoint, always at the head, re-consolidated each cycle (the backend's checkpoint-merge prompt makes this a cheap incremental merge — see below). This is *why* `CompactionResult.shadowedRange` is a **surface-position span, not a numeric seq interval**: after a replace lands a fresh high-seq summary node at an older range's position, `start` can be numerically **greater** than `end`. The range is resolved positionally (index into the ordered node list and slice), and `shadowedSeqs` is the authoritative set in surface order. (Manual `compactRegion` may target any aligned mid-range and so *can* leave several checkpoints; the checkpoint framing does not claim everything after it is recent.)
|
||||
|
||||
### Approximate convergence invariant
|
||||
|
||||
`resolveConfig` validates numeric knobs but does NOT reject based on a pretend summary-length invariant. Convergence is dynamic: provider output caps can be spent on hidden or surfaced reasoning tokens, and the model may emit a summary of unpredictable size. `maxTokens` is only the provider-side generation cap for the summarization call; reasoning blocks are stripped before the checkpoint is stored. If a compacted surface is still over threshold, `compactIfNeeded()` re-compacts the head checkpoint up to `compactionRetries` extra times, but each committed summary must be smaller than the content it shadows. The sole residual is the single-unit-overflow case above (a backward-rounded oversized step can push the retained tail over budget) — which is exactly the out-of-scope concern, not a thrash bug.
|
||||
|
||||
### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary
|
||||
|
||||
Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the (framed) summary and whose `sourceEventSeqs` covers the shadowed nodes *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance). The surface mutation sits **inside** the lock — `compact/end` is the last event appended:
|
||||
|
||||
```
|
||||
compact/start → log-only. Acquires the lock.
|
||||
[summarize older range via the backend]
|
||||
compact/summary → log-only. Provenance: raw summary, range, shadowed seqs, token count.
|
||||
user/message → surfaceOp { op:'replace', start, end }. THE surface mutation (framed summary).
|
||||
deriveMessages() renders it as a user-role message.
|
||||
compact/end → log-only. Releases the lock (carries `error` on a recoverable failure).
|
||||
```
|
||||
|
||||
`deriveMessages()` then yields `[summary_as_user_message, ...retained_nodes]`. Reusing `user/message` is honest rather than a workaround: a summary genuinely *is* user-role context.
|
||||
|
||||
### Checkpoint framing + incremental merge (backend-private)
|
||||
|
||||
The landed `user/message` is not the raw summary: the backend wraps it in a checkpoint preamble (so a resuming model reads it as established background, not a fresh request) and `<compacted-summary>…</compacted-summary>` tags. The tags make a prior checkpoint detectable on the next cycle, and the summarization prompt then instructs the model to *merge it in place* (preserve still-true facts, drop stale) rather than re-summarize verbatim — a cheap incremental merge that needs no extra log/event machinery. The raw, unframed summary stays on the `compact/summary` provenance event. This framing is entirely a **backend HOW decision** — the contract only promises "a single replace `user/message` carries the (possibly framed) summary; the raw summary lives on `compact/summary`." A template or remote backend may frame differently or not at all.
|
||||
|
||||
### Blocking via a log-recorded lock, plus a crash/recoverable failure taxonomy
|
||||
|
||||
The `compact/start … compact/end` bracket is justified, in order of what now does the work:
|
||||
|
||||
1. **Crash-detectable orphan + provenance** (primary). Summarization is a slow model call persisted *after* `compact/start`. A crash mid-summarization leaves a `compact/start` with no matching `compact/end` — a detectable orphan. Releasing the lock last (rather than first) converts the crash window from *silent corruption* into that detectable orphan.
|
||||
2. **Prevents concurrent compaction.** `compactRegion` refuses to start if the current turn holds an unmatched `compact/start`. (The loop is single-threaded across the awaited `pre-step`, so this is also a re-entry tripwire — a thrown "already in progress" signals a real bug.)
|
||||
|
||||
Two failure paths, both documented:
|
||||
|
||||
- **Crash** (the loop dies mid-summarization): a dangling `compact/start`, no closer. Because `compact/*` are **log-only**, the orphan is **inert** — the surface replacement never landed, so the full, uncompacted history derives correctly. Generic turn-repair (`interruptedTurnClosers`) closes the turn with a synthetic `turn/end`; the orphan sits *before* that `turn/end`, so the turn-scoped in-progress check never sees it and a crash can't wedge future compaction. Compaction simply re-attempts at the next `pre-step`.
|
||||
- **Recoverable** (summarization throws but the loop survives): the backend appends `compact/end` with its **`error`** field set, leaving the surface untouched, and the model call proceeds with full history.
|
||||
|
||||
`compact/end` keeps its `error?` field (mirroring `tool/result`'s self-contained error — one event tells success from failure without correlating a sibling). There is no separate `compact/error` event.
|
||||
|
||||
**Core session repair stays compaction-agnostic — deliberately.** `interruptedTurnClosers` is never taught about `compact/*`. Teaching it would force every future `xxx/start … xxx/end` plugin pair to patch a core module — exactly the coupling the capability-seam architecture exists to avoid. Because the log-only orphan is inert, no special repair is needed: generic turn-repair plus the inertness of an un-landed surface mutation is sufficient.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **New packages**: `packages/compact/compact` (interface) and a sibling `compact-basic` (backend) under `packages/compact/`, wired into the root tsconfigs. The consumer tier is deferred.
|
||||
- **New loop seam**: `agent/pre-step` (`@mode serial`) declared in `dsh-agent` and emitted by `dsh-agent-loop` after system assembly and before `step/start`. This is a documented change to the loop — `docs/architecture.md` records it and the generated cordis catalog carries its signature.
|
||||
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
|
||||
- **`dsh-session`** gains the tool-pairing balance predicate (`isToolPairingBalanced`, in `tool-pairing.ts`, exported from the package index) that `compactRegion`/`compactIfNeeded` use to keep a collapsed region from splitting a step's tool-call/result pair. The surface `replace` op and the surface-metadata runtime guard already existed and are reused.
|
||||
- **`dsh-invariants`** drops its `surface replace: start must be <= end` assertion: a head-anchored compaction lands a high-seq replacement node at an older range's *position*, so `start > end` numerically is normal and valid (the range is positional, validated by the surface's `indexOf` checks that remain). The turn-enclosure invariant is reused unchanged.
|
||||
- **Wiring**: `dsh-compact-basic` is loaded in `examples/coding-agent`'s `cordis.yml`, so the seam ships in the real demo (it was previously loaded nowhere).
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit** (`dsh-compact-basic`): the whole-unit retention walk, the convergence-invariant throw, both failure paths (`compact/end` with/without `error`), head-anchoring producing a non-monotonic `shadowedRange`, decline-on-open-tail, crash-orphan inertness, and the **runaway-turn regression** — a single oversized open turn compacts its early closed steps (proven to fail on the layer-2 protection it replaced). Driven through the real `dsh-invariants` plugin and the real Loader/inject path.
|
||||
- **Loop** (`dsh-agent-loop`): `agent/pre-step` fires once per step, after `turn/start` and before `step/start`, awaited; a surface mutation in a `pre-step` listener lands outside the step and is reflected in the single derived request.
|
||||
- **With-key e2e** (`examples/coding-agent`): a real model + real bash session with a lowered `contextWindow`/`retainTokens` triggers compaction mid-session; the test verifies the WORLD (a `compact/start…end` pair landed, the surface shrank, the agent still completed the task after compaction). This is compaction's first real-world exercise and the runaway-survival net.
|
||||
- **Snapshot (deferred, named gap)**: a full-transcript snapshot of a runaway-turn compaction is NOT yet possible — `dsh-llm-replay` derives one model call per `(turn, step)` from `assistant/chunk` events, but the summarization call records no `assistant/chunk`s and carries no `sessionId` (it binds to the anonymous cursor and claims a non-existent extra script). Covering it needs net-new replay infrastructure (record/replay an interleaved summarization call) and is scheduled as a follow-up rather than discovered mid-build.
|
||||
58
docs/rfc/implemented/feature/2026-06-29-todo-write-tool.md
Normal file
58
docs/rfc/implemented/feature/2026-06-29-todo-write-tool.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# RFC: The `todo_write` tool — model task list as event-sourced session state
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness gives the model bash and subagent tools but no way to record a structured task list. A todo list serves two co-equal purposes: it steers the model to plan multi-step work and keep the active task unambiguous (at most one active, exactly one while work remains), and it gives the human a live progress checklist. The ACP protocol has a native `plan` sessionUpdate that editors (Zed) already render, but the bridge never emitted one. Every reference coding agent surveyed (claude-code, opencode, codex, oh-my-pi, pi) ships some form of this; the harness had nothing.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a model-facing `todo_write(todos: [{ content, status }])` tool whose whole-list state lives on the event-sourced session log as a new `todo/write` `SessionEventMap` variant. Both the stdio UI and the ACP bridge render off the existing `session/event` — the ACP bridge maps the list to a `plan` sessionUpdate.
|
||||
|
||||
### Whole-list replace, three-state status
|
||||
|
||||
The model sends the ENTIRE list every call; the new list replaces the old (last-write-wins on replay). This is the shape claude-code V1, opencode, and codex `update_plan` all use, and the shape the model is most trained on — no per-item ids, no delta protocol. `status` is exactly `pending | in_progress | completed`: the same triple as codex `update_plan` and, crucially, **identical to the ACP `PlanEntryStatus`**, so the bridge maps it 1:1 with no lossy translation.
|
||||
|
||||
### State on the session log, not a service
|
||||
|
||||
The list is appended as a `todo/write` event carrying the full `{ todos }` snapshot. The harness is event-sourced — the LLM history, tool calls, and turn structure all live on the log — so the todo list lives there too. This buys durability, replay, and `session/load` reconstruction for free: a reopened session re-derives the current list (the last `todo/write`) and the ACP bridge re-emits the `plan` on load, with no separate persistence backend, no in-memory service to rehydrate, and no extra wiring. An in-memory `ctx.todos` service would have had to reinvent all of that.
|
||||
|
||||
### NOT a surface event
|
||||
|
||||
`todo/write` is deliberately excluded from `SurfaceEventType`. The surface is the projection that produces the LLM message history (`deriveMessages()`); a todo write produces no conversation message. So it carries no `surfaceOp`, never joins the surface linked list, and never reaches `deriveMessages()` — it is durable, replayable *UI* state that travels alongside the conversation without being part of it. (The dev-mode invariants still require it to sit inside an open turn, which it always does: it is appended mid-step during a tool call.)
|
||||
|
||||
### Priority synthesized only at the ACP boundary
|
||||
|
||||
ACP's `PlanEntry` requires `content` + `priority` + `status`, but a `TodoItem` has no priority — the model never reasons about it. Rather than burden the schema with a field the model must always supply, the bridge synthesizes a constant `priority: 'medium'` on every entry when it builds the `plan`. Priority is an ACP wire requirement, not a harness concept, so it lives at exactly the boundary that needs it.
|
||||
|
||||
### Dropped vs claude-code V1: `activeForm`, id, priority
|
||||
|
||||
claude-code V1's item is `{ content, status, activeForm }`; later (V2) it grew ids, dependencies, and ownership — but only to support agent *swarms* (disk-backed, lock-guarded, per-item mutation). This tool keeps the item at the minimum: `{ content, status }`. No `activeForm` (the present-continuous label) — the UI shows `content`; no id — whole-list replace needs no stable identity; no priority — see above. Each dropped field is one less thing the model must produce on every call.
|
||||
|
||||
### Single owner — no swarm machinery (YAGNI)
|
||||
|
||||
The list belongs to the ONE agent session that called the tool (`exec.agent.session`); a non-agent caller is rejected. There is deliberately no shared/multi-owner scope, no capability seam (interface/impl/consumer), no scope resolver, and no delta protocol. The harness does have subagents, and a shared cross-agent list is conceivable — but building that now means designing for a form the product does not yet have. The whole-list-replace + single-owner shape is what claude-code V1, opencode, and codex all ship; if a shared list is ever needed, the on-log representation would change to per-item deltas (so concurrent writers can't clobber each other) and a scope resolver would choose the target log. That is a future RFC, not speculative scaffolding today.
|
||||
|
||||
### Validation: the cheap middle
|
||||
|
||||
The schema enforces type/required/enum. Beyond that, `execute` rejects empty or duplicate `content` and more than one `in_progress` task. claude-code leaves single-in-progress to the prompt; oh-my-pi enforces it in code. We take the middle: enforce the cheap invariants that make a plan *coherent* (no blank tasks, no dupes, at most one active), but leave ordering and the discipline of keeping the list current to the model via the tool description. A rejected write returns an `isError` result so the model self-corrects.
|
||||
|
||||
## Why no cordis-catalog entry / no `@mode`
|
||||
|
||||
`todo/write` is a member of `SessionEventMap`, not a first-class cordis `interface Events` event. The catalog generator (`scripts/gen-cordis-catalog.ts`) scans `interface Events` declarations; a `SessionEventMap` variant rides the existing `session/event` emit and produces no new catalog row. So it carries no `@mode` tag (which the generator requires only on `interface Events` members) — adding one would be meaningless.
|
||||
|
||||
## Testing
|
||||
|
||||
Four tiers, designed up front:
|
||||
- **Unit** — the session event (append/snapshot-clone/last-write-wins/not-on-surface); the tool (schema shape, arg validation via the real `ctx.tools.execute`, value validation, the event append + replacement, no-agent rejection, `presentCall`, HMR-safety); the ACP `todosToPlan` mapping; the stdio render arm.
|
||||
- **Real-Loader path** — the plugin run through `Loader.unwrapExports`, asserting the namespace export shape survives (it HAS `inject`, so a stray default would crash at load — postmortem/0001).
|
||||
- **Full-loop integration** — a scripted mock model calls `todo_write` through the real agent loop; the `todo/write` event lands and a second call replaces it.
|
||||
- **`session/load` replay** — a persisted `todo/write` re-emits the `plan` update when a fresh ACP bridge loads the session.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot golden gains the `plan` notification and the log event.
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
- **In-memory `ctx.todos` service** — would reinvent durability, replay, and `session/load` reconstruction the log gives for free.
|
||||
- **Per-item delta protocol** — only needed for a shared multi-owner list, which is out of scope; whole-list replace is simpler and matches the references.
|
||||
- **Tool in `core/`** — `todo_write` is an extension tool registering on `ctx.tools`, not part of the spine; it lives in its own `packages/todo/` group like other tool families.
|
||||
@@ -20,7 +20,7 @@ Pure generation is correct here because the codebase is disciplined enough that
|
||||
|
||||
Specific choices:
|
||||
|
||||
- **`@mode` tag, cross-checked.** Each harness event's JSDoc carries an explicit `@mode emit|waterfall|parallel` tag; the generator hard-errors on a missing tag. Where the signature shape is conclusive — a trailing `next: () => …` parameter is structurally a waterfall — it asserts the tag agrees and hard-errors on a contradiction. The emit-vs-parallel distinction is not structurally visible (`session/flush` returns `Promise<void> | void` with no `next`), so it is trusted from the tag. The authoring rule lives in [AGENTS.md](../../../../AGENTS.md).
|
||||
- **`@mode` tag, cross-checked.** Each harness event's JSDoc carries an explicit `@mode emit|waterfall|parallel|serial` tag; the generator hard-errors on a missing tag. Where the signature shape is conclusive — a trailing `next: () => …` parameter is structurally a waterfall — it asserts the tag agrees and hard-errors on a contradiction. The emit/parallel/serial distinction is not structurally visible (`session/flush` returns `Promise<void> | void` with no `next`, as does the ordered `agent/pre-step` checkpoint), so it is trusted from the tag. The authoring rule lives in [AGENTS.md](../../../../AGENTS.md).
|
||||
- **Tiered scope.** The harness tier (the 8 `@deepseek-ai/dsh-*` services + their events) is rendered in full from source. The inherited tier (cordis-core `ctx.on/emit/effect/provide/…` + the `internal/*` events + loader/hmr/timer) is pinned vendor source a plugin also sees; it is rendered tersely (name + one-line + source pointer) from a curated table in the generator, NOT walked from the vendor AST — the cordis-core `Context` mixes true ctx members with non-service fields (`root`, `baseUrl`, `logger`), and the vendor surface changes only on a deliberate vendor sync.
|
||||
- **Cross-links to the data-structure catalog.** A type name in a signature (`GenerateOptions`, `StreamChunk`, `ToolDefinition`, …) links to the core-data-structures page that documents it. The map is a small hand-curated const in the generator — NOT `type-equiv.manifest.json`, which documents the `…Map` symbols while signatures reference the derived union names, and lists a few symbols on two pages.
|
||||
- **A dedicated fence.** Signature blocks use a ` ```ts cordis-catalog ` info string that `doc-typecheck` recognizes and skips (a bare signature fragment is not standalone-compilable), excluded from the opt-out ratio — the same treatment `type-equiv` blocks get.
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
# RFC: Compaction as a capability seam (abstract contract + basic backend)
|
||||
|
||||
Status: proposed (2026-06-18)
|
||||
|
||||
## Context
|
||||
|
||||
A long-running agent conversation grows without bound. As the event log accumulates turns, the derived message history eventually approaches the model's context window — the model then truncates mid-response (`max-tokens`) or degrades. **Compaction** is the mitigation: replace a run of older history with a concise summary, keeping recent context intact.
|
||||
|
||||
The [session surface](../../implemented/architecture/2026-06-18-session-surface.md) was built as the foundation for exactly this — a linked list over the event log with a `surfaceOp: { op: 'replace', start, end }` operation purpose-built to shadow a range of nodes and insert a replacement, with `sourceEventSeqs` recording provenance so the decision replays deterministically. What remained was the plugin that *decides what to compact and produces the summary*.
|
||||
|
||||
Two forces shape the design. First, compaction is **swappable**: token counting can be a char/4 heuristic or a real tokenizer, and summarization can be a model call, a template, or a remote service — these vary independently of *when* and *which range* to compact. Second, a later commit (`ce43c25`) closed `SurfaceEventType` to five event types (`user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`); only those may carry `surfaceOp`. A bespoke `compaction/*` event therefore **cannot** itself appear on the surface — the compiler rejects `surfaceOp` on it and the invariants plugin rejects it at runtime.
|
||||
|
||||
## Decision
|
||||
|
||||
### Compaction is a capability seam, split interface / implementation
|
||||
|
||||
Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently:
|
||||
|
||||
1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
|
||||
2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that owns the entire algorithm — token estimation (char/4 + per-block overhead), the tail→head retention walk, summarization via `ctx.llm.generate()`, the surface replacement, the lock, and the `agent/request` auto-compaction listener. A tokenizer-based or template-based backend is a sibling package (or a subclass overriding the two protected estimation/summarization hooks).
|
||||
3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
|
||||
|
||||
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
|
||||
|
||||
The capability-seams RFC states the interface package "depends only on cordis" (true of `dsh-bash`, whose vocabulary is self-contained). Compaction **cannot** honor that: its verbs are defined *over* a `Session` (`compactRegion(session, start, end)`) and its output *is* the content vocabulary (`CompactionResult.summary: ContentBlock[]`). There is no way to express the contract without naming `Session`/`SessionEvent` (from `dsh-session`) and `ContentBlock` (from `dsh-llm`).
|
||||
|
||||
This is not a coupling smell — it is the contract's domain. The "only cordis" guidance was always shorthand for "the interface depends only on what the contract genuinely names, and never on an implementation." `dsh-session` and `dsh-llm` are themselves interface/vocabulary packages, not implementations; `dsh-compact` still imports no backend. The seam's real invariant — *consumers and implementations evolve independently behind an abstract service* — holds intact. We record the deviation here so a future reader doesn't mistake it for an accident or "fix" it by smuggling `Session` behind an opaque handle.
|
||||
|
||||
### Abstract `compactIfNeeded` / `compactRegion`, algorithm in the backend
|
||||
|
||||
An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface, with only `estimateContentTokens()` and `summarize()` abstract. That recouples the contract to one strategy: a backend that wants a different retention policy (e.g. turn-count instead of token-budget) or a different event-sequencing would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend, where it belongs, and keeps the interface a pure statement of *what*. The backend remains internally factored — `estimateContentTokens()` and `summarize()` are `protected` hooks a sub-backend can override without reimplementing the walk — but that factoring is the backend's private concern, not the contract's.
|
||||
|
||||
### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary
|
||||
|
||||
Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the summary `ContentBlock[]` and whose `sourceEventSeqs` covers the shadowed nodes *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance), never on the surface. The surface mutation sits **inside** the lock — `compact/end` is the last event appended:
|
||||
|
||||
```
|
||||
compact/start → log-only. Acquires the lock.
|
||||
[summarize older range via the backend]
|
||||
compact/summary → log-only. Provenance: summary, range, shadowed seqs, token count.
|
||||
user/message → surfaceOp { op:'replace', start, end }. THE surface mutation.
|
||||
deriveMessages() renders it as a user-role message.
|
||||
compact/end → log-only. Releases the lock.
|
||||
```
|
||||
|
||||
Ordering the surface mutation **before** `compact/end` is deliberate: `session.append()` commits one event at a time, so there is no multi-event transaction to make the sequence atomic. Releasing the lock last converts the crash window from *silent corruption* (a `compact/end` that claims compaction finished while the surface was never shadowed) into a *detectable orphaned lock* (a `compact/start` with no matching `compact/end`), which a persistence backend already detects on reload. A `session/event` listener on `compact/end` likewise never sees the lock free before the replacement has landed.
|
||||
|
||||
`deriveMessages()` then yields `[summary_as_user_message, ...retained_nodes]`. An alternative — extending `SurfaceEventType` to admit a `compact/*` type — was rejected: the closed union is a deliberate safety boundary (only message-producing events reach the model), and a summary genuinely *is* user-role context, so reusing `user/message` is honest rather than a workaround.
|
||||
|
||||
### Blocking via a log-recorded lock, not a mutex
|
||||
|
||||
Compaction must be serialized: no second compaction starts before the first finishes, and no ordinary events interleave the slow summarization. Rather than an in-memory mutex (invisible to replay, lost on crash), the lock **is** the log: `compactRegion` refuses to start if the last `compact/start` has no matching `compact/end` after it. `compact/start` is appended first (fast, synchronous), the slow model call runs, then the `compact/summary` and `user/message` replacement land, and only then is `compact/end` appended — in a `catch` that records the error, so a failed summarization can never wedge the lock. Because the backend runs compaction synchronously inside the `agent/request` waterfall, the loop is single-threaded for that window; the lock additionally gives observability and lets a persistence backend detect an orphaned `compact/start` on reload.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **New packages**: `packages/compact/compact` (interface) and a sibling `compact-basic` (backend) under `packages/compact/`, wired into the three root tsconfigs. The consumer tier is deferred.
|
||||
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
|
||||
- **No changes** to `dsh-session`, `dsh-invariants`, or `dsh-agent-loop`: the surface replace op, the surface-metadata runtime guard, and the `agent/request` waterfall all already exist. Compaction is a pure plugin on documented seams.
|
||||
- The capability-seams convention gains a second reference beyond bash, and a documented case where "interface depends only on cordis" relaxes to "depends only on interface/vocabulary packages the contract genuinely names." On acceptance, [AGENTS.md](../../../../AGENTS.md) § Conventions and [architecture.md](../../../architecture.md) § "Capability seams" should note this relaxation.
|
||||
Reference in New Issue
Block a user