docs: the governing principle — every LLM request is reconstructable from the session log
The reconstructability RFC is the principle's home: model-visible ⟺ logged in both forms, the mechanism (boundary derivation + header fold), the enforcement (write-time round-trip guard, the dev invariant), the corollaries ranked (prefix-cache stability first), the MiniCode lineage with the provenance arrow inverted, and the alternatives it beat — including the stateful transmission client whose three-design archaeology lives in PR #162. Placements per the one-home-per-fact taxonomy: a standing-order line in root AGENTS.md (with displacement trims to stay inside the 1,575-word ceiling), the principle statement in architecture.md § Session Log and its Turn Flow lines (condensed to the ratcheted 1,630 ceiling), the request-envelope section in core-data-structures/core.md with the LlmCallConfig paste, both review-requested FIXMEs (FIXME(call-config-shape) beside the type, FIXME(catalog-verbs) at the catalog's drift-gate note), cookbook rows redirected off agent/request (tool filtering → system-prompt/assemble, plan-mode prompt → sections/ inject()), and the llm/stream JSDoc stating the frozen-request contract. RFC index and all generated catalogs regenerated.
This commit is contained in:
Tianyi Cui
@@ -71,8 +71,8 @@ forever:
|
||||
assemble system prompt and tool schemas
|
||||
agent/pre-step
|
||||
'step/start'
|
||||
derive messages from the session log
|
||||
agent/request -> llm/stream
|
||||
snapshot the derived messages (the reconstruction boundary)
|
||||
agent/request (config only) -> log request/header -> llm/stream (frozen)
|
||||
'assistant/chunk'
|
||||
agent/step-result
|
||||
'assistant/message'
|
||||
@@ -108,6 +108,8 @@ Every session event is turn-enclosed. Reloading a crashed session preserves the
|
||||
|
||||
The session log is the source of truth. `deriveMessages()` projects session events into the `Message[]` sent to the model; raw `assistant/chunk` events stay in the log for replay and UI fidelity. Replay, fork, resume, transcript rendering, telemetry, and persistence all derive from the same event stream.
|
||||
|
||||
**Model-visible ⟺ logged**: the log reconstructs every conversation request byte-for-byte — messages by derivation at the `step/start` boundary, the header (system prompt, tools, model + sampling) by folding `request/header` events — asserted per request by the dev invariant ([reconstructability RFC](rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)).
|
||||
|
||||
Durability is a plugin concern. Persistence backends buffer synchronous `session/event` notifications and the loop awaits a turn-end checkpoint before moving on. The `SessionPersistence` seam stores `SessionEvent` directly, with metadata in `SessionHeader`; JSONL and SQLite share one contract suite.
|
||||
|
||||
### Model Content
|
||||
|
||||
@@ -99,10 +99,10 @@ Every product feature maps to a listener on a documented extension seam — the
|
||||
| AGENTS.md (root) | a section provider reading the file |
|
||||
| AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener |
|
||||
| Built-in tools | `ctx.tools.register()`; schemas flow into the assembly automatically — the `dsh-tool-*` families (bash, fs, web, subagent, todo) are the shipped examples |
|
||||
| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` |
|
||||
| ToolSearch / progressive disclosure | filter tools at `system-prompt/assemble` (the assembly carries the schemas; the loop logs the result as the request header, so disclosure stays reconstructable) |
|
||||
| Tool sandbox (landlock / sandbox-exec) | `tools/pre-execute` (deny), or a sandboxing `BashExecutor` on the `dsh-bash` seam |
|
||||
| Permission system / AskUserQuestion | `tools/pre-execute` (deny/ask); register an ask tool |
|
||||
| Plan mode | `tools/pre-execute` (deny writes) + `agent/request` (inject mode prompt) |
|
||||
| Plan mode | `tools/pre-execute` (deny writes) + a mode prompt section via `ctx.systemPrompt.section()` or `agent.inject()` (model-visible ⟺ logged: `agent/request` shapes call config only) |
|
||||
| Sub-agent delegation | the `ctx.subagents` provider registry (`dsh-subagent-spawn`/`-fork`/`-acp`) + `dsh-tool-subagent` exposing one configured provider to the model |
|
||||
| MCP | one plugin per server: discover tools → `ctx.tools.register()` |
|
||||
| Skills | section + tool registration; `inject()` skill content on invocation |
|
||||
|
||||
@@ -189,7 +189,7 @@ Source: [`packages/fs/fs/src/index.ts:109`](../../packages/fs/fs/src/index.ts)
|
||||
|
||||
### `llm/stream` — waterfall
|
||||
|
||||
Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
Waterfall around every streaming model call (retry, replay, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit.
|
||||
|
||||
```ts cordis-catalog
|
||||
'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable<StreamChunk>): AsyncIterable<StreamChunk>
|
||||
@@ -197,7 +197,7 @@ Waterfall around every streaming model call (retry, caching, routing). Bound to
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:35`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:39`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `session/*`
|
||||
|
||||
|
||||
@@ -124,7 +124,7 @@ stream(options: GenerateOptions): AsyncIterable<StreamChunk>
|
||||
|
||||
Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md)
|
||||
|
||||
Source: [`packages/llm/llm/src/index.ts:80`](../../packages/llm/llm/src/index.ts)
|
||||
Source: [`packages/llm/llm/src/index.ts:84`](../../packages/llm/llm/src/index.ts)
|
||||
|
||||
## `ctx.sessionPersistence` — `SessionPersistence` (abstract seam)
|
||||
|
||||
|
||||
@@ -27,6 +27,8 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t
|
||||
|
||||
> Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts.
|
||||
|
||||
FIXME(catalog-verbs): the drift gate covers only the nouns (the pasted type shapes); every method surface on these pages is hand-written prose. core-data-structures should probably also generate the *verbs* — the public methods of the cataloged classes — so a signature change cannot silently outdate the catalog.
|
||||
|
||||
## The `…Map → derived-union` pattern
|
||||
|
||||
Almost every extensible sum type in the harness follows one shape: an interface keyed by a discriminant tag (the `…Map`), from which the union is derived with `keyof`. Plugins add variants by **declaration merging** — no edit to the owning package.
|
||||
@@ -183,6 +185,21 @@ interface ToolSchema {
|
||||
|
||||
The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` that produces it (schema + `execute`) is on [tools.md](tools.md).
|
||||
|
||||
### The request envelope: `LlmCallConfig` and the logged header
|
||||
|
||||
Requests are built by the loop, not shaped per call: the non-content half of a request — the `EpochHeader`: this call configuration plus the rendered system prompt and assembled tool schemas — is logged session state (`request/header` snapshot and delta events, [session.md](session.md#the-request-header-events-requestheader-and-requestheader-delta)), so every conversation request is a pure function of the session log ([reconstructability RFC](../rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). The `agent/request` waterfall receives a frozen `LlmCallConfig` seed and a listener returns a replacement to switch model or sampling — the loop logs whatever the request actually uses. Loop-built requests arrive at `llm/stream` deep-frozen; mutation throws.
|
||||
|
||||
FIXME(call-config-shape): revisit the exact definition of this type — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit here out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
|
||||
```ts type-equiv
|
||||
interface LlmCallConfig {
|
||||
model: string
|
||||
temperature?: number
|
||||
maxTokens?: number
|
||||
stop?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
## Sessions
|
||||
|
||||
A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`), not stored separately. The event vocabulary derives from `SessionEventMap`:
|
||||
|
||||
@@ -21,7 +21,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:138`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:109`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:35`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:39`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:39`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
| `session/event` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) |
|
||||
| `session/flush` | `parallel` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) |
|
||||
|
||||
@@ -117,6 +117,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand;
|
||||
| [Tagged render-intent union for tool-call presentation](implemented/architecture/2026-07-02-tool-render-intent-union.md) | 2026-07-02 |
|
||||
| [Add direct directory listing to the filesystem seam](implemented/architecture/2026-07-03-filesystem-directory-listing-seam.md) | 2026-07-03 |
|
||||
| [Prompt variables and tool-guidance ownership](implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md) | 2026-07-05 |
|
||||
| [Every LLM request is reconstructable from the session log](implemented/architecture/2026-07-05-reconstructable-requests.md) | 2026-07-05 |
|
||||
| [Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`](implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.md) | 2026-07-05 |
|
||||
|
||||
### Process
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# RFC: Every LLM request is reconstructable from the session log
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Two gaps shared one root. First, provider KV caching (DeepSeek context caching) is prefix-based — a request pays full price only for the tokens after the longest stored prefix it matches — yet nothing in the request pipeline stated, checked, or measured prefix stability: every registered [`PromptSection`](../../../../packages/core/system-prompt/src/index.ts) happened to be static, the tool set happened not to change mid-session, no listener happened to rewrite requests. A single time-interpolating section would have silently multiplied context cost, and no test or metric would have moved. Second, and deeper: the session log — the system's single source of truth — could not actually answer *what the model saw*. It recorded every message but never the system prompt, the tool schemas, or even which model; the mutable `agent/request` waterfall handed listeners the whole `GenerateOptions` to rewrite per call; replay equivalence was therefore a property of the plugin population, not of the design.
|
||||
|
||||
The reference shape for the happy path is MiniCode's `LLMClient`: a stateful conversation client, appended to — never rebuilt — as the conversation advances, resetting only when the system prompt, tool set, or compaction genuinely changes what the model must see. The design question this RFC answers is how to get that discipline without giving up event-sourcing.
|
||||
|
||||
## Decision
|
||||
|
||||
### The principle
|
||||
|
||||
**Model-visible ⟺ logged.** Anything that reaches a model request must be recorded in the session log. The checkable consequence: **every conversation request the loop sends is a pure function of the session log** — anyone holding the log reconstructs it byte-for-byte. Scope, stated precisely: the guarantee covers the loop-built `GenerateOptions`; provider wire bytes follow from it because both adapters' serialization is a pure per-message function at a pinned code version; direct one-shots (compaction's summarize call) log their envelope scalars (`compact/summary.{model, maxTokens}`) and their input is deterministic code over the logged region — reconstructable from log + code, outside the invariant by the unfrozen-request marker.
|
||||
|
||||
Prefix-cache stability is corollary #1, not the headline: an append-only log projected by a per-node pure function yields requests that are append-extensions of their predecessors whenever the header is unchanged — stability is emergent, not managed. Byte-exact audit/replay is corollary #2; resume and fork with *attributable* drift is corollary #3.
|
||||
|
||||
### The mechanism
|
||||
|
||||
**Messages.** `Session.deriveMessages()` is cached: each surface node is projected exactly once, when first seen, through the public per-node function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree.
|
||||
|
||||
**The header.** The request's non-content half — `EpochHeader`: call config (`LlmCallConfig`: model + sampling scalars), rendered system prompt, assembled tool schemas — is logged session state, in canonical form (empty system/tools ≡ absent). Two log-only, turn-enclosed events in dsh-session carry it: `request/header`, a full snapshot with reason `'initial' | 'resume' | 'fallback'`, and `request/header-delta`, an amendment (`SystemDelta`: a common-prefix/suffix line trim; `ToolsDelta`: name-keyed added/removed/changed; `config`: replaced whole). The pure trio `foldRequestHeader` / `diffHeader` / `applyHeaderDelta` reconstructs; the live session tracks the fold with the same lazy cursor as the message cache. Snapshots anchor the fold where a fold needs anchors — conversation birth and process boundaries — and each loop instance appends one on its first request (`'initial'` when the log has none, `'resume'` otherwise, even when nothing changed: the boundary itself is a recorded fact, and cross-restart drift becomes attributable while an unchanged header resumes byte-identical). Deltas are an encoding optimization with a safety valve, never a correctness dependency: the writer verifies `applyHeaderDelta(prev, delta)` reproduces the new header exactly and records a `'fallback'` snapshot when the encoding cannot express a change (a pure tool reordering), so a well-formed log always folds.
|
||||
|
||||
**The loop, transmission-stateless.** Per step: render assembly (every step — value comparison needs no change-signal discipline, and a section that varies per step surfaces as a *logged* header event per step instead of a silent bust) → `agent/pre-step` (compaction's surface mutations land before derivation) → **messages snapshot, then `step/start` appended as the next operation in the same synchronous frame** → seed the call config (first request of the instance: from `AgentOptions`, so explicit options always beat the logged baseline — fork model-overrides and resume reconfiguration stay correct; afterwards: from the folded header) → the `agent/request` waterfall, re-typed `(agent, turn, step, config: LlmCallConfig, next) → LlmCallConfig` — a frozen seed and a returned replacement are ALL a listener shapes; content flows through the log channels (`inject()`, steering, prompt-submit `additionalContext`, sections via `system-prompt/assemble`) — → the header event the request owes the log → build `GenerateOptions` from the snapshot + header, deep-freeze (`deepFreeze` exempts the `AbortSignal`, the one live control channel — freezing one breaks `AbortController.abort()`), dispatch. The loop's only in-process bookkeeping is one boolean: whether this instance has logged its anchoring snapshot.
|
||||
|
||||
**The reconstruction boundary is `step/start`, unconditionally.** A step's messages are the derivation over `events[0..stepStartSeq)`. Because the snapshot precedes the `step/start` append in the same synchronous frame, nothing can enter this request past the boundary: an `agent.inject()` from an `agent/request` listener (or any concurrent task, or a `session/event` listener firing on `step/start` itself) lands in the log after the boundary and joins the NEXT request. For waterfall-window appends this matches the prior loop (it also derived before its waterfall); for a synchronous `step/start` listener it is a deliberate change — such a listener could previously reach the current request — and `agent/pre-step` is the sanctioned seam for content that must affect the CURRENT request. A step's header for reconstruction is the fold after its own `request/header*` event (which sits between its `step/start` and first response event) or the fold carried forward.
|
||||
|
||||
**Enforcement.** Dev-mode ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)), on `llm/stream`: a frozen request with a live `sessionId` — the loop-built marker; hand-built one-shots are unfrozen and skipped — must carry messages deep-equal to the boundary derivation, rebuilt through a FRESH `Session` over `events[0..stepStartSeq)` so the live cache cannot vouch for itself, and header fields equal to `foldRequestHeader` over the log. There is no divergence allowance and nothing to allow: no seam can put unlogged content into a request. `prepend: true` only defends against the replay adapter's short-circuit (an append-registered listener); two prepended listeners have no defined mutual order in cordis, so correctness rests on the seq-bounded fold, never on listener timing. Measurement stays lean: the with-key e2e ([request-cache.e2e.ts](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts)) proves `usage.cacheReadTokens > 0` on every request after the first against the live API, and per-step usage in the log is the production observable — a header event or compaction shows up as a cache-read collapse on the next step.
|
||||
|
||||
### The MiniCode shape: adopted, with the provenance arrow inverted
|
||||
|
||||
What survives from `LLMClient`: the conversation is maintained, not rebuilt — one projection per message, ever; requests advance append-only; resets happen only for a system-prompt/tool change, a config change, or compaction, each now a *logged* fact. What is deliberately inverted: MiniCode's client is the source of truth and its event stream derives from client appends (`on_event(MessageAdded)`), which suits an advisory event stream. Here the log is contractual — persistence, crash recovery, fork seeding, transcript rendering, and the snapshot harness all replay it — and it carries strictly more than a message list (turn/step boundaries, raw chunk streams, tool-call pairing, provenance, log-only records), so a message-list client cannot generate it. The arrow therefore points log → client: the conversation state IS the log plus two cached folds inside `Session` (messages, header), and the "client" the loop talks to is the session itself. What the inversion buys over the original: the reconstruction is *checkable* against an independent record on every request — MiniCode's client has nothing to check itself against.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Client as source of truth** (literal MiniCode): a second operative truth beside the log — the two drift and nothing notices; see the section above.
|
||||
- **A stateful transmission client mirroring the log** (a `PromptPrefix` class holding committed/open message zones with an append/editTail/reset vocabulary, the log pushed into it per event): behaviorally equivalent on the happy path, but it duplicates conversation state outside the session, needs transactional rollback around listener seams, keeps an unlogged content-shaping surface (`editTail`) whose divergence the invariant must specially allow, and still cannot answer "what header did the model see" from the log. Dissolving it into the session's own caches plus logged header events made every one of those problems unrepresentable instead of guarded. (PR #162 is the archaeology of this alternative, three designs deep.)
|
||||
- **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records.
|
||||
- **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability.
|
||||
- **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline.
|
||||
- **Narrative fields on the header events** (a `reason`/`changed` list on deltas): derivable by diffing consecutive events — one home per fact; snapshots carry a reason because an anchor's cause is NOT derivable from the data.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event.
|
||||
- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replace node), a real prompt/tool change (`request/header-delta`), a config switch (ditto), a process boundary with drift (`'resume'` snapshot differing from its predecessor). The provider's own reasoning-content exclusion is managed server-side.
|
||||
- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam.
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-node surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per conversation (system + tool schemas: the dominant term), plus deltas on real changes — small next to `assistant/chunk` volume; `SESSION_FORMAT_VERSION` stays `0` (pre-release churn is absorbed, backends reject-not-migrate).
|
||||
- Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
Reference in New Issue
Block a user