Merge remote-tracking branch 'origin/master' into codex/skill-system
# Conflicts: # docs/event-producer-consumer.md
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# DeepSeek Harness Architecture
|
||||
|
||||
The **DeepSeek Harness SDK** is an SDK for building agent harnesses using the Cordis framework. The governing principle is simple: **everything is a plugin**. For example, the shipped agent loop is just one plugin in the default bundle, not a privileged kernel.
|
||||
The **DeepSeek Harness SDK** builds agent harnesses on Cordis. The governing principle is simple: **everything is a plugin**. The shipped agent loop is just one plugin in the default bundle, not a privileged kernel.
|
||||
|
||||
Read this page as the system map before changing `packages/`. It explains how the runtime is shaped, how the default loop moves work, where state lives, and where extensions attach. Type shapes live in [core-data-structures/](core-data-structures/core.md); exact event and service signatures live in the generated [events](cordis-catalog/events.md) and [services](cordis-catalog/services.md) catalogs; package contracts live in the [package map](../packages/README.md); rationale lives in the [RFCs](rfc/README.md). If Cordis itself is new to you, start with the [Cordis primer](cordis-primer.md).
|
||||
|
||||
@@ -42,7 +42,7 @@ Events are the harness extension API. Each service owns the vocabulary for the b
|
||||
Use the event domain to decide where new behavior belongs:
|
||||
|
||||
- **Session events** are durable, replayable facts. Turn and step boundaries, user input, assistant output, tool calls, tool results, steering, compaction records, and tool-owned durable facts append to the session log and flow through `session/event`.
|
||||
- **Agent events** are live runtime surfaces. They carry the live `Agent` handle for status, diagnostics, prompt admission, request mutation, result validation, and continuation policy.
|
||||
- **Agent events** are live runtime surfaces. They carry the live `Agent` handle for status, diagnostics, prompt admission, call-config shaping, result validation, and continuation policy.
|
||||
- **Capability events** belong to the seam that owns the action. `tools/*`, `llm/*`, `system-prompt/*`, `fs/*`, and `subagent/*` let policy and adapters attach without importing the loop.
|
||||
|
||||
### Interception Semantics
|
||||
@@ -51,7 +51,7 @@ Waterfall events behave like around-middleware: a listener delegates by calling
|
||||
|
||||
## Default Loop Lifecycle
|
||||
|
||||
The shipped loop drains queued work, assembles a request, streams a model answer, executes tools, decides whether to continue, and checkpoints durable state. The important architecture is where it pauses: each pause is a documented service call or event seam that another plugin can program against.
|
||||
The shipped loop drains queued work, assembles a request, streams a model answer, executes tools, decides whether to continue, and checkpoints durable state. Its architecture is where it pauses: each pause is a documented service call or event seam another plugin can program against.
|
||||
|
||||
A **session** is one agent's append-only event log. A **turn** drains one queued batch and runs until the model stops asking for tools and no plugin requests continuation. A **step** is one model request plus the tool executions caused by that response. In the flow below ([sequence companion](agent-lifecycle.md)), quoted names are durable session events and event names are extension seams.
|
||||
|
||||
@@ -72,8 +72,8 @@ forever:
|
||||
assemble system prompt and tool schemas
|
||||
agent/pre-step
|
||||
'step/start'
|
||||
derive messages from the session log
|
||||
agent/request -> llm/stream
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
agent/request (config only) -> log request/header -> llm/stream (frozen)
|
||||
'assistant/chunk'
|
||||
agent/step-result
|
||||
'assistant/message'
|
||||
@@ -89,7 +89,7 @@ forever:
|
||||
checkpoint persistence and notify idle/running status
|
||||
```
|
||||
|
||||
Prompt assembly is single-path: `renderPrompt(assemble({ agent }))` IS the system prompt sent to the model. Plugins contribute ordered sections (static or computed from the per-call `AssembleContext`), tool schemas, and named variables interpolated as `{{name}}` at render — strictly, so an unknown or valueless reference fails the turn instead of shipping a hole. `dsh-system-prompt` itself owns the openers — the static `harness:identity` section (order −100) and the deployment's persona (order 0, from its `persona` config, shared by every agent in the context) — while the shipped loop registers the `model`/`cwd` variables; prompt-fact ownership is pinned by the [prompt-variables RFC](rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md).
|
||||
Prompt assembly is single-path: `renderPrompt(assemble({ agent }))` IS the system prompt sent to the model. Plugins contribute ordered sections (static or computed from the per-call `AssembleContext`), tool schemas, and named variables interpolated as `{{name}}` at render — strictly, so an unknown or valueless reference fails the turn instead of shipping a hole. `dsh-system-prompt` owns the openers — the static `harness:identity` section (order −100) and the deployment's persona (order 0, from its `persona` config, shared by every agent) — while the shipped loop registers the `model`/`cwd` variables; prompt-fact ownership is pinned by the [prompt-variables RFC](rfc/implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md).
|
||||
|
||||
Post-tool context lands after all tool results so tool-call/result adjacency stays stable. Steering drains between steps; leftover steering after a turn is re-queued as ordinary input.
|
||||
|
||||
@@ -109,6 +109,8 @@ Every session event is turn-enclosed. Reloading a crashed session preserves the
|
||||
|
||||
The session log is the source of truth. `deriveMessages()` projects session events into the `Message[]` sent to the model; raw `assistant/chunk` events stay in the log for replay and UI fidelity. Replay, fork, resume, transcript rendering, telemetry, and persistence all derive from the same event stream.
|
||||
|
||||
**Model-visible ⟺ logged**: the log reconstructs every conversation request byte-for-byte — messages by derivation at the `step/start` boundary, the header (system prompt, tools, model + sampling) by folding `request/header` events — asserted per request by the dev invariant ([reconstructability RFC](rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)).
|
||||
|
||||
Durability is a plugin concern. Persistence backends buffer synchronous `session/event` notifications and the loop awaits a turn-end checkpoint before moving on. The `SessionPersistence` seam stores `SessionEvent` directly, with metadata in `SessionHeader`; JSONL and SQLite share one contract suite.
|
||||
|
||||
### Model Content
|
||||
|
||||
@@ -99,10 +99,10 @@ Every product feature maps to a listener on a documented extension seam — the
|
||||
| AGENTS.md (root) | a section provider reading the file |
|
||||
| AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener |
|
||||
| Built-in tools | `ctx.tools.register()`; schemas flow into the assembly automatically — the `dsh-tool-*` families (bash, fs, web, subagent, todo) are the shipped examples |
|
||||
| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` |
|
||||
| ToolSearch / progressive disclosure | filter tools at `system-prompt/assemble` (the assembly carries the schemas; the loop logs the result as the request header, so disclosure stays reconstructable) |
|
||||
| Tool sandbox (landlock / sandbox-exec) | `tools/pre-execute` (deny), or a sandboxing `BashExecutor` on the `dsh-bash` seam |
|
||||
| Permission system / AskUserQuestion | `tools/pre-execute` (deny/ask); register an ask tool |
|
||||
| Plan mode | `tools/pre-execute` (deny writes) + `agent/request` (inject mode prompt) |
|
||||
| Plan mode | `tools/pre-execute` (deny writes) + a mode prompt section via `ctx.systemPrompt.section()` or `agent.inject()` (model-visible ⟺ logged: `agent/request` shapes call config only) |
|
||||
| Sub-agent delegation | the `ctx.subagents` provider registry (`dsh-subagent-spawn`/`-fork`/`-acp`) + `dsh-tool-subagent` exposing one configured provider to the model |
|
||||
| MCP | one plugin per server: discover tools → `ctx.tools.register()` |
|
||||
| Skills | section + tool registration; `inject()` skill content on invocation |
|
||||
|
||||
@@ -47,7 +47,7 @@ A step or turn errored. The loop reports a failure here (plus the logger) even w
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:394`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:404`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/pre-step` — serial
|
||||
|
||||
@@ -89,15 +89,15 @@ Source: [`packages/core/agent/src/types.ts:273`](../../packages/core/agent/src/t
|
||||
|
||||
### `agent/request` — waterfall
|
||||
|
||||
Waterfall: mutate the fully-assembled GenerateOptions before the model call (hooks, model switching, tool filtering, …). Call `next()` to delegate, or return without it to short-circuit. For surface mutation that must precede history derivation (compaction), use agent/pre-step instead — by the time this fires, `options.messages` is already derived.
|
||||
Waterfall: shape the step's call configuration — model switching, sampling overrides — by returning a replacement LlmCallConfig (the frozen seed is the config the loop would otherwise use). Config is ALL a listener shapes here: every request is a pure function of the session log (the reconstructability RFC), so model-visible content flows through the log channels — `inject()`, steering, prompt-submit `additionalContext`, prompt sections via `system-prompt/assemble` — never through request mutation, and the loop records whatever config the request actually uses as a `request/header*` event before dispatch. The step's messages are already snapshotted when this fires (the `step/start` boundary): an `inject()` from a listener here lands in the log but joins the NEXT request. For surface mutation that must precede the snapshot (compaction), use agent/pre-step. Call `next()` to delegate, or return an LlmCallConfig without it to short-circuit.
|
||||
|
||||
```ts cordis-catalog
|
||||
'agent/request'(agent: Agent, turn: number, step: number, options: GenerateOptions, next: () => Promise<GenerateOptions>): Promise<GenerateOptions>
|
||||
'agent/request'(agent: Agent, turn: number, step: number, config: LlmCallConfig, next: () => Promise<LlmCallConfig>): Promise<LlmCallConfig>
|
||||
```
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [GenerateOptions](../core-data-structures/core.md)
|
||||
Types: [Agent](../core-data-structures/core.md) · [LlmCallConfig](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:359`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:369`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/session-start` — emit
|
||||
|
||||
@@ -133,7 +133,7 @@ Waterfall: post-process the assembled assistant Message before tool dispatch (va
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md) · [Message](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:369`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:379`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
### `agent/turn-continuation` — waterfall
|
||||
|
||||
@@ -145,7 +145,7 @@ Waterfall: override the turn-continuation decision via a typed ContinuationDecis
|
||||
|
||||
Types: [Agent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/agent/src/types.ts:382`](../../packages/core/agent/src/types.ts)
|
||||
Source: [`packages/core/agent/src/types.ts:392`](../../packages/core/agent/src/types.ts)
|
||||
|
||||
## `fs/*`
|
||||
|
||||
@@ -189,7 +189,7 @@ Source: [`packages/fs/fs/src/index.ts:109`](../../packages/fs/fs/src/index.ts)
|
||||
|
||||
### `llm/stream` — waterfall
|
||||
|
||||
Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
Waterfall around every streaming model call (retry, replay, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
|
||||
@@ -197,7 +197,7 @@ Waterfall around every streaming model call (retry, caching, routing). Bound to
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:33`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:39`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `session/*`
|
||||
|
||||
@@ -209,7 +209,7 @@ A session was created in the store.
|
||||
'session/created'(session: Session): void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:36`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:39`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `session/event` — emit
|
||||
|
||||
@@ -221,7 +221,7 @@ An event was appended to a session log (sync, fire-and-forget). This is the per-
|
||||
|
||||
Types: [SessionEvent](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:44`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:47`](../../packages/core/session/src/index.ts)
|
||||
|
||||
### `session/flush` — parallel
|
||||
|
||||
@@ -231,7 +231,7 @@ Awaited durability checkpoint. The agent loop awaits `ctx.parallel('session/flus
|
||||
'session/flush'(session: Session): Promise<void> | void
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:54`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:57`](../../packages/core/session/src/index.ts)
|
||||
|
||||
## `subagent/*`
|
||||
|
||||
|
||||
@@ -79,8 +79,8 @@ Implementations MUST honor:
|
||||
- **Blocking**: no compaction begins while another is in progress for the same session. The recommended mechanism is the log-recorded lock — append `compact/start` before the slow work and `compact/end` after (even on failure) — so the lock is visible to replay and crash recovery.
|
||||
|
||||
```ts cordis-catalog
|
||||
abstract compactIfNeeded( agent: CompactAgentContext, turn: number, step: number, fullSystemPrompt: string, signal: AbortSignal, ): Promise<CompactionResult | null>
|
||||
abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, turn: number, step: number, signal?: AbortSignal, ): Promise<CompactionResult>
|
||||
abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, signal: AbortSignal, ): Promise<CompactionResult | null>
|
||||
abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise<CompactionResult>
|
||||
```
|
||||
|
||||
Source: [`packages/compact/compact/src/index.ts:63`](../../packages/compact/compact/src/index.ts)
|
||||
@@ -124,7 +124,7 @@ stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:78`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:84`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `ctx.sessionPersistence` — `SessionPersistence` (abstract seam)
|
||||
|
||||
@@ -163,7 +163,7 @@ get(id: SessionId): Session | undefined
|
||||
list(): Session[]
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/index.ts:327`](../../packages/core/session/src/index.ts)
|
||||
Source: [`packages/core/session/src/index.ts:371`](../../packages/core/session/src/index.ts)
|
||||
|
||||
## `ctx.skills` — `SkillService`
|
||||
|
||||
@@ -176,7 +176,7 @@ async get(name: string, options: SkillLookupOptions = {}): Promise<SkillDefiniti
|
||||
async renderModelListing(options: SkillLookupOptions = {}): Promise<string>
|
||||
```
|
||||
|
||||
Source: [`packages/core/skill/src/index.ts:133`](../../packages/core/skill/src/index.ts)
|
||||
Source: [`packages/core/skill/src/index.ts:134`](../../packages/core/skill/src/index.ts)
|
||||
|
||||
## `ctx.subagents` — `SubagentService`
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ Compaction extends [`SessionEventMap`](session.md) with three event types via de
|
||||
| Event | Payload | Role |
|
||||
|---|---|---|
|
||||
| `compact/start` | `{ turn }` | acquires the log-recorded lock |
|
||||
| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount }` | provenance: the summary blocks, the shadowed surface-boundary pair (`start`/`end` seqs — a position span, not a numeric interval), the shadowed seqs in surface order, and the estimated token count |
|
||||
| `compact/summary` | `{ summary, shadowedRange, shadowedSeqs, shadowedTokenCount, model, maxTokens? }` | provenance: the summary blocks, the shadowed surface-boundary pair (`start`/`end` seqs — a position span, not a numeric interval), the shadowed seqs in surface order, the estimated token count, and the summarize call's envelope (`model`, plus its generation cap when one applied) — logged so the one-shot request is reconstructable from log + code (the reconstructability RFC) |
|
||||
| `compact/end` | `{ turn, error? }` | releases the lock (`error` set when summarization threw) |
|
||||
|
||||
The lock brackets the **whole** operation: `compact/start` is appended first, then summarization, the `compact/summary` provenance record, and the `user/message` replacement all land, and only then `compact/end`. Releasing the lock last turns a crash mid-operation into a detectable orphaned lock (a `compact/start` with no matching `compact/end`) rather than a `compact/end` that falsely claims compaction finished.
|
||||
@@ -50,6 +50,6 @@ interface CompactionResult {
|
||||
|
||||
## The service
|
||||
|
||||
`CompactService` (`ctx.compact`, abstract — defined in [`packages/compact/compact/src/index.ts`](../../packages/compact/compact/src/index.ts)) declares two abstract methods: `compactIfNeeded(agent, turn, step, fullSystemPrompt, signal)` checks token pressure and compacts an older range if the history is too large (returning `null` when nothing needs it), and `compactRegion(session, start, end, agent, turn, step, signal?)` forcibly summarizes surface nodes `[start, end]` into a single replacement node. `compactIfNeeded`'s parameters are all required — the loop's `agent/pre-step` checkpoint supplies the agent, lifecycle context, assembled `fullSystemPrompt`, and turn `signal`. A backend summarizing via `ctx.llm.stream()` must forward `signal` into the call's `GenerateOptions.signal`, so an abort or dispose tears down the in-flight summarization. The entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
`CompactService` (`ctx.compact`, abstract — defined in [`packages/compact/compact/src/index.ts`](../../packages/compact/compact/src/index.ts)) declares two abstract methods: `compactIfNeeded(agent, fullSystemPrompt, signal)` checks token pressure and compacts an older range if the history is too large (returning `null` when nothing needs it), and `compactRegion(session, start, end, agent, signal?)` forcibly summarizes surface nodes `[start, end]` into a single replacement node. `compactIfNeeded`'s parameters are all required — the loop's `agent/pre-step` checkpoint supplies the agent, the assembled `fullSystemPrompt`, and the turn `signal`. A backend summarizing via `ctx.llm.stream()` must forward `signal` into the call's `GenerateOptions.signal`, so an abort or dispose tears down the in-flight summarization. The entire strategy — token estimation, retention policy, event sequencing, summarization — is a HOW decision owned by the implementation.
|
||||
|
||||
Auto-compaction runs on the serial `agent/pre-step` loop seam (fired once per step, after `turn/start` and BEFORE the step opens and its request history is derived), not the `agent/request` waterfall: compaction mutates the session surface in place — with its log-only `compact/*` records landing cleanly outside any step — and the loop derives the request from the already-compacted surface. Retention is turn-agnostic — the only structural guard is tool-pairing balance (a compacted region's edges are balanced cuts on the surface, so it never splits a step's tool-calls from their results), so a single runaway turn that alone exceeds the window compacts its own early closed steps rather than being retained verbatim. The backend that ships this (`dsh-compact-basic`) documents the retention walk, summary shrink validation, bounded re-compaction, and the crash/recoverable failure taxonomy.
|
||||
|
||||
@@ -28,6 +28,8 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
|
||||
|
||||
> Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts.
|
||||
|
||||
FIXME(catalog-verbs): the drift gate covers only the nouns (the pasted type shapes); every method surface on these pages is hand-written prose. core-data-structures should probably also generate the *verbs* — the public methods of the cataloged classes — so a signature change cannot silently outdate the catalog.
|
||||
|
||||
## The `…Map → derived-union` pattern
|
||||
|
||||
Almost every extensible sum type in the harness follows one shape: an interface keyed by a discriminant tag (the `…Map`), from which the union is derived with `keyof`. Plugins add variants by **declaration merging** — no edit to the owning package.
|
||||
@@ -184,6 +186,21 @@ interface ToolSchema {
|
||||
|
||||
The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` that produces it (schema + `execute`) is on [tools.md](tools.md).
|
||||
|
||||
### The request envelope: `LlmCallConfig` and the logged header
|
||||
|
||||
Requests are built by the loop, not shaped per call: the non-content half of a request — the `EpochHeader`: this call configuration plus the rendered system prompt and assembled tool schemas — is logged session state (`request/header` snapshot and delta events, [session.md](session.md#the-request-header-events-requestheader-and-requestheader-delta)), so every conversation request is a pure function of the session log ([reconstructability RFC](../rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). The `agent/request` waterfall receives a frozen `LlmCallConfig` seed and a listener returns a replacement to switch model or sampling — the loop logs whatever the request actually uses. Loop-built requests arrive at `llm/stream` deep-frozen; mutation throws.
|
||||
|
||||
FIXME(call-config-shape): revisit the exact definition of this type — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit here out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
|
||||
```ts type-equiv
|
||||
interface LlmCallConfig {
|
||||
model: string
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
stop?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
## Sessions
|
||||
|
||||
A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`), not stored separately. The event vocabulary derives from `SessionEventMap`:
|
||||
@@ -212,7 +229,7 @@ type SessionEvent<T extends SessionEventType = SessionEventType> = {
|
||||
}[T]
|
||||
```
|
||||
|
||||
The thirteen event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `prompt/blocked`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`, `todo/write`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
The fifteen event variants (`turn/start`, `turn/end`, `step/start`, `step/end`, `user/message`, `prompt/blocked`, `context/message`, `assistant/chunk`, `assistant/message`, `tool/call`, `tool/result`, `steering/message`, `todo/write`, `request/header`, `request/header-delta`), the `deriveMessages()` projection rules, the `TurnTrigger`/`TurnEndReason` reasons, and the turn-enclosure invariant are on **[session.md](session.md)**. How the log is made durable — the `SessionPersistence` seam, JSONL/SQLite backends, the `session/flush` checkpoint, crash recovery, and `SessionHeader` — is on **[persistence.md](persistence.md)**.
|
||||
|
||||
## The agent handle
|
||||
|
||||
|
||||
@@ -60,6 +60,30 @@ interface SessionEventMap {
|
||||
* cordis-catalog row.
|
||||
*/
|
||||
'todo/write': { todos: TodoItem[] }
|
||||
/**
|
||||
* Full snapshot of the {@link EpochHeader} the NEXT request is built under,
|
||||
* with the {@link RequestHeaderReason} it was recorded whole. Appended by
|
||||
* the loop inside the step, before dispatch, on a loop instance's first
|
||||
* request-building step (`'initial'`/`'resume'`) or when a delta failed its
|
||||
* round-trip guard (`'fallback'`); always records what the request actually
|
||||
* used, post-`agent/request`. Anchors the header fold: reconstruction reads
|
||||
* the latest snapshot and applies the deltas after it. NOT a
|
||||
* {@link SurfaceEventType}: it produces no LLM message — it is the request
|
||||
* envelope, logged so every request is a pure function of the session log
|
||||
* (the reconstructability RFC).
|
||||
*/
|
||||
'request/header': { header: EpochHeader; reason: RequestHeaderReason }
|
||||
/**
|
||||
* Amendment to the folded {@link EpochHeader}: at least one of a
|
||||
* {@link SystemDelta}, a {@link ToolsDelta}, or a whole replacement
|
||||
* {@link LlmCallConfig} (four scalars — not worth diffing). Appended by the
|
||||
* loop inside the step, before dispatch, when the header for this request
|
||||
* differs from the fold of the log so far; the writer verifies
|
||||
* `applyHeaderDelta(previous, delta)` reproduces the new header exactly and
|
||||
* falls back to a `'fallback'` `request/header` snapshot when it cannot, so
|
||||
* a logged delta ALWAYS round-trips. NOT a {@link SurfaceEventType}.
|
||||
*/
|
||||
'request/header-delta': { system?: SystemDelta; tools?: ToolsDelta; config?: LlmCallConfig }
|
||||
}
|
||||
```
|
||||
|
||||
@@ -74,6 +98,23 @@ export interface TodoItem {
|
||||
}
|
||||
```
|
||||
|
||||
### The request header events: `request/header` and `request/header-delta`
|
||||
|
||||
The request envelope — the `EpochHeader` (call config + rendered system prompt + assembled tool schemas) — is logged session state, so every conversation request is a pure function of the log (the reconstructability RFC). A `request/header` snapshot (reason `'initial' | 'resume' | 'fallback'`) anchors the fold at conversation birth, process boundaries, and delta-encoding fallbacks; `request/header-delta` events amend it mid-run. `foldRequestHeader(events)` reconstructs the header any request was built under; the writer round-trip-verifies every delta before logging it, so a well-formed log always folds. Neither is a `SurfaceEventType` — they produce no LLM message.
|
||||
|
||||
```ts type-equiv
|
||||
export interface EpochHeader {
|
||||
/** The conversation's call configuration (model + sampling scalars). */
|
||||
config: LlmCallConfig
|
||||
/** Rendered system prompt text; absent for a system-less request. */
|
||||
system?: string
|
||||
/** Assembled tool schemas; absent for a tool-less request. */
|
||||
tools?: ToolSchema[]
|
||||
}
|
||||
```
|
||||
|
||||
Canonical form: an empty system prompt and an empty tool list are ABSENT fields, matching how requests are built. The delta payloads (`SystemDelta` — a common-prefix/suffix line trim; `ToolsDelta` — name-keyed added/removed/changed) live beside the events in [`packages/core/session/src/types.ts`](../../packages/core/session/src/types.ts).
|
||||
|
||||
## `SessionEvent<T>` — one log entry
|
||||
|
||||
A proper discriminated union over `type` (not independent `type`/`data` unions), so `switch (event.type)` narrows `event.data` without casts. `seq` is the monotonic position in the log (`seq = log.length`); `time` is epoch ms.
|
||||
@@ -148,9 +189,9 @@ export interface SurfaceNode {
|
||||
}
|
||||
```
|
||||
|
||||
## Derived history: `deriveMessages()`
|
||||
## Derived history: `deriveMessages()` and `deriveEventMessage()`
|
||||
|
||||
`Session.deriveMessages()` projects the event log into the `Message[]` the model sees. The projection rules:
|
||||
`Session.deriveMessages()` projects the event log into the `Message[]` the model sees — cached (each surface node projected once, when first seen; a surface rewrite rebuilds) and frozen (a fresh array per call over shared, deep-frozen messages, so mutating logged history through a projection is unrepresentable). `deriveEventMessage(event)` is the per-node pure function the fold applies — public so external reconstructors and the dev invariant project a log prefix with exactly the same rules and cannot disagree with the cache. The projection rules:
|
||||
|
||||
- `user/message` → a user message.
|
||||
- `assistant/message` → an assistant message. Raw `assistant/chunk` events are replay/UI data and are **skipped** in derivation (the assembled message is authoritative). An **empty-content** `assistant/message` is also skipped — a max-tokens step cut off with no content still records an `assistant/message` to host its `usage`, but a content-less assistant turn must not enter the provider transcript.
|
||||
|
||||
@@ -86,6 +86,6 @@ interface Config {
|
||||
|
||||
## Prompt and tool contract
|
||||
|
||||
`ctx.skills.renderModelListing()` returns a `## Skills` fragment wrapped in `<available_skills>`. Descriptions and `whenToUse` are whitespace-normalized, length-capped, and XML-escaped before rendering. The listing is appended to the same `GenerateOptions.system` string by the `agent/request` waterfall, after the base system prompt is assembled.
|
||||
`ctx.skills.renderModelListing()` returns a `## Skills` fragment wrapped in `<available_skills>`. Descriptions and `whenToUse` are whitespace-normalized, length-capped, and XML-escaped before rendering. The listing is appended as a late `system-prompt/assemble` section for the calling agent, so it remains cwd-sensitive while still flowing through the reconstructable system-prompt path.
|
||||
|
||||
The model-facing `skill({ name })` tool validates the kebab-case name, loads the complete definition for the calling agent cwd, rejects unknown or `disableModelInvocation` skills, and returns a `<skill_content name="...">` block with the body plus base-directory and relative-path guidance. The tool result is the only v1 path that exposes full skill instructions to the model.
|
||||
|
||||
@@ -9,27 +9,27 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `agent/created` | `emit` | [`packages/core/agent/src/types.ts:248`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/disposed` | `emit` | [`packages/core/agent/src/types.ts:255`](../packages/core/agent/src/types.ts) | [`agent`](../packages/core/agent) (`emit`) | [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:394`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
|
||||
| `agent/error` | `emit` | [`packages/core/agent/src/types.ts:404`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
|
||||
| `agent/pre-step` | `serial` | [`packages/core/agent/src/types.ts:333`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`serial`) | [`compact-basic`](../packages/compact/compact-basic) |
|
||||
| `agent/prompt-submit` | `waterfall` | [`packages/core/agent/src/types.ts:346`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/queued` | `emit` | [`packages/core/agent/src/types.ts:273`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | - |
|
||||
| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:359`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`), [`compact-basic`](../packages/compact/compact-basic) (`waterfall`) | [`skill`](../packages/core/skill) |
|
||||
| `agent/request` | `waterfall` | [`packages/core/agent/src/types.ts:369`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
|
||||
| `agent/session-start` | `emit` | [`packages/core/agent/src/types.ts:288`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/status` | `emit` | [`packages/core/agent/src/types.ts:264`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:369`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
|
||||
| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:382`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `agent/step-result` | `waterfall` | [`packages/core/agent/src/types.ts:379`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | - |
|
||||
| `agent/turn-continuation` | `waterfall` | [`packages/core/agent/src/types.ts:392`](../packages/core/agent/src/types.ts) | [`agent-loop`](../packages/core/agent-loop) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:138`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:109`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:33`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`llm-replay`](../packages/support/llm-replay) |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:36`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
| `session/event` | `emit` | [`packages/core/session/src/index.ts:44`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `session/flush` | `parallel` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:39`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:39`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
| `session/event` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `session/flush` | `parallel` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
| `subagent/end` | `emit` | [`packages/subagent/subagent/src/index.ts:98`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) |
|
||||
| `subagent/provider-added` | `emit` | [`packages/subagent/subagent/src/index.ts:72`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`emit`) | [`tool-subagent`](../packages/subagent/tool-subagent) |
|
||||
| `subagent/provider-removed` | `emit` | [`packages/subagent/subagent/src/index.ts:83`](../packages/subagent/subagent/src/index.ts) | - | [`tool-subagent`](../packages/subagent/tool-subagent) |
|
||||
| `subagent/start` | `emit` | [`packages/subagent/subagent/src/index.ts:91`](../packages/subagent/subagent/src/index.ts) | [`subagent`](../packages/subagent/subagent) (`events.dispatch`) | [`hooks-claude`](../packages/hooks/hooks-claude) |
|
||||
| `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:38`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | - |
|
||||
| `system-prompt/assemble` | `waterfall` | [`packages/core/system-prompt/src/index.ts:38`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`waterfall`) | [`skill`](../packages/core/skill) |
|
||||
| `system-prompt/change` | `emit` | [`packages/core/system-prompt/src/index.ts:44`](../packages/core/system-prompt/src/index.ts) | [`system-prompt`](../packages/core/system-prompt) (`emit`) | - |
|
||||
| `tools/change` | `emit` | [`packages/core/tools/src/index.ts:87`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`emit`) | - |
|
||||
| `tools/post-execute` | `waterfall` | [`packages/core/tools/src/index.ts:82`](../packages/core/tools/src/index.ts) | [`tools`](../packages/core/tools) (`waterfall`) | [`hooks-claude`](../packages/hooks/hooks-claude), [`hooks-codex`](../packages/hooks/hooks-codex) |
|
||||
|
||||
@@ -111,6 +111,7 @@ flowchart TD
|
||||
pkg_skill --> pkg_agent
|
||||
pkg_skill --> pkg_fs
|
||||
pkg_skill --> pkg_llm
|
||||
pkg_skill --> pkg_system_prompt
|
||||
pkg_tools --> pkg_agent
|
||||
pkg_tools --> pkg_llm
|
||||
pkg_tools --> pkg_system_prompt
|
||||
@@ -237,7 +238,7 @@ flowchart TD
|
||||
| [`hook-protocol`](../packages/hooks/hook-protocol) | `hooks` | [`bash`](../packages/bash/bash), [`session`](../packages/core/session) |
|
||||
| [`session-persistence`](../packages/session-persistence/session-persistence) | `session-persistence` | [`session`](../packages/core/session) |
|
||||
| [`llm-replay`](../packages/support/llm-replay) | `support` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
| [`skill`](../packages/core/skill) | `core` | [`agent`](../packages/core/agent), [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm) |
|
||||
| [`skill`](../packages/core/skill) | `core` | [`agent`](../packages/core/agent), [`fs`](../packages/fs/fs), [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`tools`](../packages/core/tools) | `core` | [`agent`](../packages/core/agent), [`llm`](../packages/llm/llm), [`system-prompt`](../packages/core/system-prompt) |
|
||||
| [`compact-basic`](../packages/compact/compact-basic) | `compact` | [`agent`](../packages/core/agent), [`compact`](../packages/compact/compact), [`llm`](../packages/llm/llm), [`session`](../packages/core/session) |
|
||||
| [`session-persistence-jsonl`](../packages/session-persistence/session-persistence-jsonl) | `session-persistence` | [`session`](../packages/core/session), [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
|
||||
@@ -23,7 +23,7 @@ Raw stream chunk — token-level replay fidelity.
|
||||
|
||||
Types: [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:237`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:298`](../../packages/core/session/src/types.ts)
|
||||
|
||||
#### `assistant/message` — surface
|
||||
|
||||
@@ -35,7 +35,7 @@ Assembled assistant message for one step (derived history uses this). Carries th
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [TokenUsage](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:244`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:305`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `compact/*`
|
||||
|
||||
@@ -47,7 +47,7 @@ Marks the end of a compaction — log-only, releases the lock. `error` set if su
|
||||
'compact/end': { turn: number; error?: string }
|
||||
```
|
||||
|
||||
Source: [`packages/compact/compact/src/types.ts:37`](../../packages/compact/compact/src/types.ts)
|
||||
Source: [`packages/compact/compact/src/types.ts:46`](../../packages/compact/compact/src/types.ts)
|
||||
|
||||
#### `compact/start` — log-only
|
||||
|
||||
@@ -64,7 +64,7 @@ Source: [`packages/compact/compact/src/types.ts:23`](../../packages/compact/comp
|
||||
Provenance record of a completed summarization — log-only, no surfaceOp. The summary content is in `data.summary`; the actual surface replacement is performed by a subsequent `user/message` event that shadows the compacted range.
|
||||
|
||||
```ts persistence-catalog
|
||||
'compact/summary': { summary: ContentBlock[]; shadowedRange: { start: number; end: number }; shadowedSeqs: number[]; shadowedTokenCount: number }
|
||||
'compact/summary': { summary: ContentBlock[]; shadowedRange: { start: number; end: number }; shadowedSeqs: number[]; shadowedTokenCount: number; model: string; maxTokens?: number }
|
||||
```
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md)
|
||||
@@ -83,7 +83,7 @@ In-session context injection (file-change notices, subdir AGENTS.md, skill conte
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:235`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:296`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `hook/*`
|
||||
|
||||
@@ -119,7 +119,29 @@ A queued prompt an `agent/prompt-submit` listener VETOED — the durable record
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:229`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:290`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `request/*`
|
||||
|
||||
#### `request/header` — log-only
|
||||
|
||||
Full snapshot of the EpochHeader the NEXT request is built under, with the RequestHeaderReason it was recorded whole. Appended by the loop inside the step, before dispatch, on a loop instance's first request-building step (`'initial'`/`'resume'`) or when a delta failed its round-trip guard (`'fallback'`); always records what the request actually used, post-`agent/request`. Anchors the header fold: reconstruction reads the latest snapshot and applies the deltas after it. NOT a SurfaceEventType: it produces no LLM message — it is the request envelope, logged so every request is a pure function of the session log (the reconstructability RFC).
|
||||
|
||||
```ts persistence-catalog
|
||||
'request/header': { header: EpochHeader; reason: RequestHeaderReason }
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:350`](../../packages/core/session/src/types.ts)
|
||||
|
||||
#### `request/header-delta` — log-only
|
||||
|
||||
Amendment to the folded EpochHeader: at least one of a SystemDelta, a ToolsDelta, or a whole replacement LlmCallConfig (four scalars — not worth diffing). Appended by the loop inside the step, before dispatch, when the header for this request differs from the fold of the log so far; the writer verifies `applyHeaderDelta(previous, delta)` reproduces the new header exactly and falls back to a `'fallback'` `request/header` snapshot when it cannot, so a logged delta ALWAYS round-trips. NOT a SurfaceEventType.
|
||||
|
||||
```ts persistence-catalog
|
||||
'request/header-delta': { system?: SystemDelta; tools?: ToolsDelta; config?: LlmCallConfig }
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:361`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `steering/*`
|
||||
|
||||
@@ -133,7 +155,7 @@ Steering content injected between steps of a running turn.
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:262`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:323`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `step/*`
|
||||
|
||||
@@ -145,7 +167,7 @@ Closes step `step` of turn `turn`.
|
||||
'step/end': { turn: number; step: number }
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:216`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:277`](../../packages/core/session/src/types.ts)
|
||||
|
||||
#### `step/start` — log-only
|
||||
|
||||
@@ -155,7 +177,7 @@ Opens step `step` of turn `turn` — one model call plus the tool executions it
|
||||
'step/start': { turn: number; step: number }
|
||||
```
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:214`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:275`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `todo/*`
|
||||
|
||||
@@ -171,7 +193,7 @@ NOT a SurfaceEventType: it produces no LLM message and never reaches `deriveMess
|
||||
|
||||
Types: [TodoItem](../core-data-structures/session.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:276`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:337`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `tool/*`
|
||||
|
||||
@@ -185,7 +207,7 @@ The model requested one tool invocation: `name` with the raw `arguments` JSON st
|
||||
|
||||
Types: [CallId](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:250`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:311`](../../packages/core/session/src/types.ts)
|
||||
|
||||
#### `tool/result` — surface
|
||||
|
||||
@@ -197,7 +219,7 @@ A completed tool call's model-facing result, plus an optional tool-private `meta
|
||||
|
||||
Types: [CallId](../core-data-structures/core.md) · [ContentBlock](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:260`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:321`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `turn/*`
|
||||
|
||||
@@ -211,7 +233,7 @@ Closes turn `turn` with the TurnEndReason that ended it. The loop fires the awai
|
||||
|
||||
Types: [TurnEndReason](../core-data-structures/session.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:212`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:273`](../../packages/core/session/src/types.ts)
|
||||
|
||||
#### `turn/start` — log-only
|
||||
|
||||
@@ -223,7 +245,7 @@ Opens turn `turn`. `trigger` records what started it — a drained message batch
|
||||
|
||||
Types: [TurnTrigger](../core-data-structures/session.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:206`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:267`](../../packages/core/session/src/types.ts)
|
||||
|
||||
### `user/*`
|
||||
|
||||
@@ -237,4 +259,4 @@ A user-visible prompt (queued message drained at turn start).
|
||||
|
||||
Types: [ContentBlock](../core-data-structures/core.md) · [MessageSource](../core-data-structures/core.md)
|
||||
|
||||
Source: [`packages/core/session/src/types.ts:218`](../../packages/core/session/src/types.ts)
|
||||
Source: [`packages/core/session/src/types.ts:279`](../../packages/core/session/src/types.ts)
|
||||
|
||||
@@ -118,6 +118,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [Tagged render-intent union for tool-call presentation](implemented/architecture/2026-07-02-tool-render-intent-union.md) | 2026-07-02 |
|
||||
| [Add direct directory listing to the filesystem seam](implemented/architecture/2026-07-03-filesystem-directory-listing-seam.md) | 2026-07-03 |
|
||||
| [Prompt variables and tool-guidance ownership](implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md) | 2026-07-05 |
|
||||
| [Every LLM request is reconstructable from the session log](implemented/architecture/2026-07-05-reconstructable-requests.md) | 2026-07-05 |
|
||||
| [Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`](implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.md) | 2026-07-05 |
|
||||
|
||||
### Process
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# RFC: Every LLM request is reconstructable from the session log
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Two gaps shared one root. First, provider KV caching (DeepSeek context caching) is prefix-based — a request pays full price only for the tokens after the longest stored prefix it matches — yet nothing in the request pipeline stated, checked, or measured prefix stability: every registered [`PromptSection`](../../../../packages/core/system-prompt/src/index.ts) happened to be static, the tool set happened not to change mid-session, no listener happened to rewrite requests. A single time-interpolating section would have silently multiplied context cost, and no test or metric would have moved. Second, and deeper: the session log — the system's single source of truth — could not actually answer *what the model saw*. It recorded every message but never the system prompt, the tool schemas, or even which model; the mutable `agent/request` waterfall handed listeners the whole `GenerateOptions` to rewrite per call; replay equivalence was therefore a property of the plugin population, not of the design.
|
||||
|
||||
The reference shape for the happy path is MiniCode's `LLMClient`: a stateful conversation client, appended to — never rebuilt — as the conversation advances, resetting only when the system prompt, tool set, or compaction genuinely changes what the model must see. The design question this RFC answers is how to get that discipline without giving up event-sourcing.
|
||||
|
||||
## Decision
|
||||
|
||||
### The principle
|
||||
|
||||
**Model-visible ⟺ logged.** Anything that reaches a model request must be recorded in the session log. The checkable consequence: **every conversation request the loop sends is a pure function of the session log** — anyone holding the log reconstructs it byte-for-byte. Scope, stated precisely: the guarantee covers the loop-built `GenerateOptions`; provider wire bytes follow from it because both adapters' serialization is a pure per-message function at a pinned code version; direct one-shots (compaction's summarize call) log their envelope scalars (`compact/summary.{model, maxTokens}`) and their input is deterministic code over the logged region — reconstructable from log + code, outside the invariant by the unfrozen-request marker.
|
||||
|
||||
Prefix-cache stability is corollary #1, not the headline: an append-only log projected by a per-node pure function yields requests that are append-extensions of their predecessors whenever the header is unchanged — stability is emergent, not managed. Byte-exact audit/replay is corollary #2; resume and fork with *attributable* drift is corollary #3.
|
||||
|
||||
### The mechanism
|
||||
|
||||
**Messages.** `Session.deriveMessages()` is cached: each surface node is projected exactly once, when first seen, through the public per-node function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree.
|
||||
|
||||
**The header.** The request's non-content half — `EpochHeader`: call config (`LlmCallConfig`: model + sampling scalars), rendered system prompt, assembled tool schemas — is logged session state, in canonical form (empty system/tools ≡ absent). Two log-only, turn-enclosed events in dsh-session carry it: `request/header`, a full snapshot with reason `'initial' | 'resume' | 'fallback'`, and `request/header-delta`, an amendment (`SystemDelta`: a common-prefix/suffix line trim; `ToolsDelta`: name-keyed added/removed/changed; `config`: replaced whole). The pure trio `foldRequestHeader` / `diffHeader` / `applyHeaderDelta` reconstructs; the live session tracks the fold with the same lazy cursor as the message cache. Snapshots anchor the fold where a fold needs anchors — conversation birth and process boundaries — and each loop instance appends one on its first request (`'initial'` when the log has none, `'resume'` otherwise, even when nothing changed: the boundary itself is a recorded fact, and cross-restart drift becomes attributable while an unchanged header resumes byte-identical). Deltas are an encoding optimization with a safety valve, never a correctness dependency: the writer verifies `applyHeaderDelta(prev, delta)` reproduces the new header exactly and records a `'fallback'` snapshot when the encoding cannot express a change (a pure tool reordering), so a well-formed log always folds.
|
||||
|
||||
**The loop, transmission-stateless.** Per step: render assembly (every step — value comparison needs no change-signal discipline, and a section that varies per step surfaces as a *logged* header event per step instead of a silent bust) → `agent/pre-step` (compaction's surface mutations land before derivation) → **messages snapshot, then `step/start` appended as the next operation in the same synchronous frame** → seed the call config (first request of the instance: from `AgentOptions`, so explicit options always beat the logged baseline — fork model-overrides and resume reconfiguration stay correct; afterwards: from the folded header) → the `agent/request` waterfall, re-typed `(agent, turn, step, config: LlmCallConfig, next) → LlmCallConfig` — a frozen seed and a returned replacement are ALL a listener shapes; content flows through the log channels (`inject()`, steering, prompt-submit `additionalContext`, sections via `system-prompt/assemble`) — → the header event the request owes the log → build `GenerateOptions` from the snapshot + header, deep-freeze (`deepFreeze` exempts the `AbortSignal`, the one live control channel — freezing one breaks `AbortController.abort()`), dispatch. The loop's only in-process bookkeeping is one boolean: whether this instance has logged its anchoring snapshot.
|
||||
|
||||
**The reconstruction boundary is `step/start`, unconditionally.** A step's messages are the derivation over `events[0..stepStartSeq)`. Because the snapshot precedes the `step/start` append in the same synchronous frame, nothing can enter this request past the boundary: an `agent.inject()` from an `agent/request` listener (or any concurrent task, or a `session/event` listener firing on `step/start` itself) lands in the log after the boundary and joins the NEXT request. For waterfall-window appends this matches the prior loop (it also derived before its waterfall); for a synchronous `step/start` listener it is a deliberate change — such a listener could previously reach the current request — and `agent/pre-step` is the sanctioned seam for content that must affect the CURRENT request. A step's header for reconstruction is the fold after its own `request/header*` event (which sits between its `step/start` and first response event) or the fold carried forward.
|
||||
|
||||
**Enforcement.** Dev-mode ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)), on `llm/stream`: a frozen request with a live `sessionId` — the loop-built marker; hand-built one-shots are unfrozen and skipped — must carry messages deep-equal to the boundary derivation, rebuilt through a FRESH `Session` over `events[0..stepStartSeq)` so the live cache cannot vouch for itself, and header fields equal to `foldRequestHeader` over the log. There is no divergence allowance and nothing to allow: no seam can put unlogged content into a request. `prepend: true` only defends against the replay adapter's short-circuit (an append-registered listener); two prepended listeners have no defined mutual order in cordis, so correctness rests on the seq-bounded fold, never on listener timing. Measurement stays lean: the with-key e2e ([request-cache.e2e.ts](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts)) proves `usage.cacheReadTokens > 0` on every request after the first against the live API, and per-step usage in the log is the production observable — a header event or compaction shows up as a cache-read collapse on the next step.
|
||||
|
||||
### The MiniCode shape: adopted, with the provenance arrow inverted
|
||||
|
||||
What survives from `LLMClient`: the conversation is maintained, not rebuilt — one projection per message, ever; requests advance append-only; resets happen only for a system-prompt/tool change, a config change, or compaction, each now a *logged* fact. What is deliberately inverted: MiniCode's client is the source of truth and its event stream derives from client appends (`on_event(MessageAdded)`), which suits an advisory event stream. Here the log is contractual — persistence, crash recovery, fork seeding, transcript rendering, and the snapshot harness all replay it — and it carries strictly more than a message list (turn/step boundaries, raw chunk streams, tool-call pairing, provenance, log-only records), so a message-list client cannot generate it. The arrow therefore points log → client: the conversation state IS the log plus two cached folds inside `Session` (messages, header), and the "client" the loop talks to is the session itself. What the inversion buys over the original: the reconstruction is *checkable* against an independent record on every request — MiniCode's client has nothing to check itself against.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Client as source of truth** (literal MiniCode): a second operative truth beside the log — the two drift and nothing notices; see the section above.
|
||||
- **A stateful transmission client mirroring the log** (a `PromptPrefix` class holding committed/open message zones with an append/editTail/reset vocabulary, the log pushed into it per event): behaviorally equivalent on the happy path, but it duplicates conversation state outside the session, needs transactional rollback around listener seams, keeps an unlogged content-shaping surface (`editTail`) whose divergence the invariant must specially allow, and still cannot answer "what header did the model see" from the log. Dissolving it into the session's own caches plus logged header events made every one of those problems unrepresentable instead of guarded. (PR #162 is the archaeology of this alternative, three designs deep.)
|
||||
- **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records.
|
||||
- **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability.
|
||||
- **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline.
|
||||
- **Narrative fields on the header events** (a `reason`/`changed` list on deltas): derivable by diffing consecutive events — one home per fact; snapshots carry a reason because an anchor's cause is NOT derivable from the data.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event.
|
||||
- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replace node), a real prompt/tool change (`request/header-delta`), a config switch (ditto), a process boundary with drift (`'resume'` snapshot differing from its predecessor). The provider's own reasoning-content exclusion is managed server-side.
|
||||
- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam.
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-node surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per conversation (system + tool schemas: the dominant term), plus deltas on real changes — small next to `assistant/chunk` volume; `SESSION_FORMAT_VERSION` stays `0` (pre-release churn is absorbed, backends reject-not-migrate).
|
||||
- Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
@@ -18,7 +18,7 @@ Each skill is either `<name>/SKILL.md` or `<name>.md` with YAML frontmatter. `na
|
||||
|
||||
Skill filesystem I/O goes through `ctx.fs` when a filesystem service is loaded: project-root lookup probes `.git` with `resolve` and `stat`, root discovery uses `listDir`, skill reads use `readText`, and system-skill installation uses `writeText`. The Node filesystem remains a fallback for minimal contexts that mount `dsh-skill` without the fs seam. Missing roots and unreadable or malformed skill files degrade to warn-and-skip so one bad local file does not make every agent request fail.
|
||||
|
||||
The service injects a request-time `## Skills` fragment through the existing `agent/request` waterfall. It appends to `GenerateOptions.system` instead of changing `systemPrompt.assemble()`, because the available project skills depend on the calling agent's cwd. The fragment contains only stable routing metadata and is sorted by skill name after first-wins collection, so equivalent workspaces produce deterministic prompt text and better prefix-cache reuse. Full skill bodies are never included in the listing.
|
||||
The service injects a request-time `## Skills` fragment through the existing `system-prompt/assemble` waterfall. It appends a late section for the calling agent instead of mutating `GenerateOptions.system` in `agent/request`, because request configuration is now reconstructable model/sampling state while model-visible content flows through system prompt assembly. The fragment contains only stable routing metadata and is sorted by skill name after first-wins collection, so equivalent workspaces produce deterministic prompt text and better prefix-cache reuse. Full skill bodies are never included in the listing.
|
||||
|
||||
The `skill({ name })` tool loads one full skill for the current agent cwd and returns a `<skill_content name="...">` block with the body plus base-directory guidance. Invalid names, unknown skills, and skills marked `disableModelInvocation` return tool errors. v1 does not additionally inject the loaded body into session context; the tool result is the model-visible disclosure path.
|
||||
|
||||
@@ -32,7 +32,7 @@ The data structures and prompt/tool contract are documented in [skills.md](../..
|
||||
|
||||
**Expose skills only as slash commands.** Rejected for v1 because model-initiated loading is the core capability; slash/ACP command advertisement can layer on later without changing discovery.
|
||||
|
||||
**Use a separate system-reminder message.** Rejected for the current loop because `agent/request` already owns the last mutation point before the adapter call and `GenerateOptions.system` is the provider-neutral system prompt surface. A later provider-specific surface can still split this fragment if needed.
|
||||
**Use a separate system-reminder message.** Rejected for the current loop because the provider-neutral system prompt surface is assembled through `system-prompt/assemble`. A later provider-specific surface can still split this fragment if needed.
|
||||
|
||||
**Recursively discover nested `**/SKILL.md`.** Rejected for v1. Flat files and one-level directory bundles cover the configured roots while keeping duplicate handling and prompt order easy to reason about.
|
||||
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
Status: proposed
|
||||
|
||||
> Premise partially stale: this proposal predates [request reconstructability](../../implemented/architecture/2026-07-05-reconstructable-requests.md) — `agent/request` now shapes call config only (no request/content mutation), so the interception points named below need re-mapping onto the log channels and `system-prompt/assemble` before implementation.
|
||||
|
||||
## Problem
|
||||
|
||||
Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` feeds its schemas into `ctx.systemPrompt`, the loop puts them on `GenerateOptions.tools`, and the adapter serializes them to the provider's function-calling wire format. The model then invokes one `tool-call` block per step, the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../architecture.md)), and **every** intermediate `tool-result` re-enters the model's context on the next request.
|
||||
|
||||
Reference in New Issue
Block a user