diff --git a/AGENTS.md b/AGENTS.md index bfb4060e5e..8433ac6e79 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,7 +52,7 @@ pnpm run demo:acp # ACP server agent (needs DEEPSEEK_API_KEY) ### Run the CI gates locally before marking a PR ready -CI is the backstop, not the first run. From a fresh clone or worktree, `pnpm run build` first — publint and the NodeNext check validate built `lib/`. The CI-equivalent run: +From a fresh clone or worktree, `pnpm run build` first — publint and the NodeNext check validate built `lib/`. The CI-equivalent run: ```sh set -euo pipefail @@ -72,7 +72,7 @@ rm -rf .sessions pnpm exec vitest run --config vitest.e2e.config.ts packages/ui/stdio-agent/tests/built-bin.e2e.ts packages/ui/acp-agent/tests/built-bin.e2e.ts ``` -`test:coverage`, not `test`, is the gating run ([why](docs/testing.md)); a review sign-off counts only for the commands it actually ran. +`test:coverage`, not `test`, is the gating run ([why](docs/testing.md)); a sign-off counts only for commands actually run. ## Secrets / .env @@ -81,18 +81,19 @@ Real-API tests and demos read `DEEPSEEK_API_KEY` (and optional `DEEPSEEK_BASE_UR ## Conventions - Every npm package is `@deepseek-ai/dsh-`; vendored packages keep upstream names and are `private: true`. `cordis` is a peerDependency (+ dev) of every harness package. -- ESM everywhere (`"type": "module"`). Cross-package imports use package names, never relative paths; in-package relative imports use explicit `.ts` extensions. Dev/test/demo run unbuilt via tsx + the root tsconfig `paths` map; building is only for consumers outside the repo. +- ESM everywhere (`"type": "module"`). Cross-package imports use package names, never relative paths; in-package relative imports use explicit `.ts` extensions. Dev/test/demo run unbuilt via tsx + the root tsconfig `paths` map; builds are for outside consumers only. - **Registrations are effects**: every contribution goes through `ctx.effect()` / `ctx.on()`; a registry's `register()` returns the disposer. -- **Typed events via declaration merging**; extensible unions use the merge-extensible-map pattern (`ContentBlockMap`, `SessionEventMap`, …). Every new event's JSDoc carries an `@mode` tag and a `@param` per payload parameter (`this`/trailing `next` exempt); every public service-class method documents each parameter and non-void return (`@param`/`@returns`) — the catalog generator hard-errors otherwise ([completeness RFC](docs/rfc/implemented/process/2026-07-04-cordis-jsdoc-completeness-gate.md)); mode semantics are in the [generated events catalog](docs/cordis-catalog/events.md) header and [the catalog RFC](docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md). +- **Typed events via declaration merging**; extensible unions use the merge-extensible-map pattern (`ContentBlockMap`, `SessionEventMap`, …). Every new event's JSDoc carries an `@mode` tag and a `@param` per payload parameter (`this`/trailing `next` exempt); every public service-class method documents each parameter and non-void return (`@param`/`@returns`) — the catalog generator hard-errors otherwise ([completeness RFC](docs/rfc/implemented/process/2026-07-04-cordis-jsdoc-completeness-gate.md)); mode semantics are in the [generated events catalog](docs/cordis-catalog/events.md) header ([catalog RFC](docs/rfc/implemented/process/2026-06-20-generated-cordis-catalog.md)). - **Discriminated unions: `switch` on the tag**, not if-chains. Closed unions end with `default: assertNever(...)`; merge-extensible unions must NOT — handle known cases and fall through `default` with a comment. - **Waterfall listeners MUST call `next()`** to delegate; returning without it is the veto ([semantics](docs/cordis-primer.md#cordis-waterfall-semantics)). +- **Model-visible ⟺ logged**: anything that reaches a model request must be reconstructable from the session log; a new model-visible input requires a session event ([reconstructability RFC](docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). - **Plugins, not loop changes**: new behavior goes on the documented extension seams; changing `agent-loop` requires updating docs/architecture.md. - **Capability seams are three packages** — interface / implementation / consumer ([capability seams](docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)); don't split preemptively. -- **Explicit > implicit at package seams**: no optional field silently filled by a hidden `?? default` inside `run()`; defaulting is an explicit `resolve(request): Spec` step in the owning implementation (the `dsh-bash` request/spec split is the template). -- **No hardcoded tunables in plugins**: anything two deployments could want different — timeouts, caps, grace periods, model names, base URLs — is a defaulted, validated `Config` field, not a literal; a `DEFAULT_*` constant or test-only seam is not configurability. The test: changeable from `cordis.yml`, no code edit. Protocol/wire constants, external-spec values, security invariants stay hardcoded. +- **Explicit > implicit at package seams**: defaulting is an explicit `resolve(request): Spec` step in the owning implementation, never a hidden `?? default` inside `run()` (the `dsh-bash` request/spec split is the template). +- **No hardcoded tunables in plugins**: anything two deployments could want different — timeouts, caps, model names, base URLs — is a defaulted, validated `Config` field, not a literal; a `DEFAULT_*` constant or test-only seam is not configurability. The test: changeable from `cordis.yml`, no code edit. Protocol/wire constants, external-spec values, security invariants stay hardcoded. - **Opaque cross-boundary ids are branded** (`Branded` from `dsh-brand`), never bare `string` ([branded IDs](docs/rfc/implemented/architecture/2026-06-20-branded-ids.md)). - **An empty `catch` names what it swallows** and why nothing else can reach it; keep the `try` to one statement. -- **Symmetry is usually more correct**: parallel values get parallel form; asymmetry is a smell for a missed extraction. +- **Symmetry is usually more correct**: parallel values get parallel form; asymmetry smells of a missed extraction. - **Tests document behavior, not golden truth**: a green test pins what the code DOES, not what it SHOULD do. Before preserving a behavior solely for its test, ask whether it is load-bearing; an artifact changes together with its test, with the why in the PR ([worked example](docs/rfc/implemented/simplification/2026-06-19-drop-mutable-session-summary.md)). - **RFCs are proposals, not golden truth**: validate its premise against current code before implementing; friction is evidence of over-reach — amend on the way to `implemented/` ([worked example](docs/rfc/implemented/simplification/2026-06-20-public-agent-stop-surface.md)). - **Testing policy** — tiers, with-key generosity, real-over-mock, world-verification, real-load-path and published-bin guards: [docs/testing.md](docs/testing.md). A transcript/UX-affecting change needs a snapshot test, or a PR note why none applies. diff --git a/docs/architecture.md b/docs/architecture.md index d24b142869..9eec6199c7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -71,8 +71,8 @@ forever: assemble system prompt and tool schemas agent/pre-step 'step/start' - derive messages from the session log - agent/request -> llm/stream + snapshot the derived messages (the reconstruction boundary) + agent/request (config only) -> log request/header -> llm/stream (frozen) 'assistant/chunk' agent/step-result 'assistant/message' @@ -108,6 +108,8 @@ Every session event is turn-enclosed. Reloading a crashed session preserves the The session log is the source of truth. `deriveMessages()` projects session events into the `Message[]` sent to the model; raw `assistant/chunk` events stay in the log for replay and UI fidelity. Replay, fork, resume, transcript rendering, telemetry, and persistence all derive from the same event stream. +**Model-visible ⟺ logged**: the log reconstructs every conversation request byte-for-byte — messages by derivation at the `step/start` boundary, the header (system prompt, tools, model + sampling) by folding `request/header` events — asserted per request by the dev invariant ([reconstructability RFC](rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). + Durability is a plugin concern. Persistence backends buffer synchronous `session/event` notifications and the loop awaits a turn-end checkpoint before moving on. The `SessionPersistence` seam stores `SessionEvent` directly, with metadata in `SessionHeader`; JSONL and SQLite share one contract suite. ### Model Content diff --git a/docs/cookbook/extension-cookbook.md b/docs/cookbook/extension-cookbook.md index e579d13794..de21ece7b1 100644 --- a/docs/cookbook/extension-cookbook.md +++ b/docs/cookbook/extension-cookbook.md @@ -99,10 +99,10 @@ Every product feature maps to a listener on a documented extension seam — the | AGENTS.md (root) | a section provider reading the file | | AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener | | Built-in tools | `ctx.tools.register()`; schemas flow into the assembly automatically — the `dsh-tool-*` families (bash, fs, web, subagent, todo) are the shipped examples | -| ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` | +| ToolSearch / progressive disclosure | filter tools at `system-prompt/assemble` (the assembly carries the schemas; the loop logs the result as the request header, so disclosure stays reconstructable) | | Tool sandbox (landlock / sandbox-exec) | `tools/pre-execute` (deny), or a sandboxing `BashExecutor` on the `dsh-bash` seam | | Permission system / AskUserQuestion | `tools/pre-execute` (deny/ask); register an ask tool | -| Plan mode | `tools/pre-execute` (deny writes) + `agent/request` (inject mode prompt) | +| Plan mode | `tools/pre-execute` (deny writes) + a mode prompt section via `ctx.systemPrompt.section()` or `agent.inject()` (model-visible ⟺ logged: `agent/request` shapes call config only) | | Sub-agent delegation | the `ctx.subagents` provider registry (`dsh-subagent-spawn`/`-fork`/`-acp`) + `dsh-tool-subagent` exposing one configured provider to the model | | MCP | one plugin per server: discover tools → `ctx.tools.register()` | | Skills | section + tool registration; `inject()` skill content on invocation | diff --git a/docs/cordis-catalog/events.md b/docs/cordis-catalog/events.md index c34143bada..753cf8f4d0 100644 --- a/docs/cordis-catalog/events.md +++ b/docs/cordis-catalog/events.md @@ -189,7 +189,7 @@ Source: [`packages/fs/fs/src/index.ts:109`](../../packages/fs/fs/src/index.ts) ### `llm/stream` — waterfall -Waterfall around every streaming model call (retry, caching, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit. +Waterfall around every streaming model call (retry, replay, routing). Bound to the LlmService; call `next()` to reach the resolved adapter's stream, or yield your own chunks to short-circuit. ```ts cordis-catalog 'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable @@ -197,7 +197,7 @@ Waterfall around every streaming model call (retry, caching, routing). Bound to Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:35`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:39`](../../packages/llm/llm/src/index.ts) ## `session/*` diff --git a/docs/cordis-catalog/services.md b/docs/cordis-catalog/services.md index 0c3b8cb31a..60ee35c29c 100644 --- a/docs/cordis-catalog/services.md +++ b/docs/cordis-catalog/services.md @@ -124,7 +124,7 @@ stream(options: GenerateOptions): AsyncIterable Types: [GenerateOptions](../core-data-structures/core.md) · [StreamChunk](../core-data-structures/llm-streaming.md) -Source: [`packages/llm/llm/src/index.ts:80`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:84`](../../packages/llm/llm/src/index.ts) ## `ctx.sessionPersistence` — `SessionPersistence` (abstract seam) diff --git a/docs/core-data-structures/core.md b/docs/core-data-structures/core.md index dc3bab7437..8cd9f1c30b 100644 --- a/docs/core-data-structures/core.md +++ b/docs/core-data-structures/core.md @@ -27,6 +27,8 @@ Everything else is documented on a **sub-page**, not here. The rule that draws t > Type definitions on this page are pasted **verbatim** from source and drift-checked by `pnpm run verify-type-equiv` (see [development.md](../development.md#documenting-types-verbatim-ts-type-equiv)). Inline JSDoc is omitted for readability; follow the source link for the full contracts. +FIXME(catalog-verbs): the drift gate covers only the nouns (the pasted type shapes); every method surface on these pages is hand-written prose. core-data-structures should probably also generate the *verbs* — the public methods of the cataloged classes — so a signature change cannot silently outdate the catalog. + ## The `…Map → derived-union` pattern Almost every extensible sum type in the harness follows one shape: an interface keyed by a discriminant tag (the `…Map`), from which the union is derived with `keyof`. Plugins add variants by **declaration merging** — no edit to the owning package. @@ -183,6 +185,21 @@ interface ToolSchema { The model-facing `ToolSchema` is the wire shape; the registered `ToolDefinition` that produces it (schema + `execute`) is on [tools.md](tools.md). +### The request envelope: `LlmCallConfig` and the logged header + +Requests are built by the loop, not shaped per call: the non-content half of a request — the `EpochHeader`: this call configuration plus the rendered system prompt and assembled tool schemas — is logged session state (`request/header` snapshot and delta events, [session.md](session.md#the-request-header-events-requestheader-and-requestheader-delta)), so every conversation request is a pure function of the session log ([reconstructability RFC](../rfc/implemented/architecture/2026-07-05-reconstructable-requests.md)). The `agent/request` waterfall receives a frozen `LlmCallConfig` seed and a listener returns a replacement to switch model or sampling — the loop logs whatever the request actually uses. Loop-built requests arrive at `llm/stream` deep-frozen; mutation throws. + +FIXME(call-config-shape): revisit the exact definition of this type — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit here out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them. + +```ts type-equiv +interface LlmCallConfig { + model: string + temperature?: number + maxTokens?: number + stop?: string[] +} +``` + ## Sessions A `Session` is an **append-only log** of typed `SessionEvent`s — the single source of truth. The LLM message history is *derived* from the log (`deriveMessages()`), not stored separately. The event vocabulary derives from `SessionEventMap`: diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 13269b2a24..24a2793826 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -21,7 +21,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `fs/edit-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:123`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/observed` | `emit` | [`packages/fs/fs/src/index.ts:138`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`emit`) | [`fs-policy`](../packages/fs/fs-policy) | | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:109`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`) | [`fs-policy`](../packages/fs/fs-policy) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:35`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:39`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`invariants`](../packages/support/invariants), [`llm-replay`](../packages/support/llm-replay) | | `session/created` | `emit` | [`packages/core/session/src/index.ts:39`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence) | | `session/event` | `emit` | [`packages/core/session/src/index.ts:47`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`emit`) | [`acp`](../packages/ui/acp), [`invariants`](../packages/support/invariants), [`session-persistence`](../packages/session-persistence/session-persistence), [`stdio-agent`](../packages/ui/stdio-agent) | | `session/flush` | `parallel` | [`packages/core/session/src/index.ts:57`](../packages/core/session/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`parallel`) | [`session-persistence`](../packages/session-persistence/session-persistence) | diff --git a/docs/rfc/INDEX.md b/docs/rfc/INDEX.md index 0c5a11a4f9..921c6389e8 100644 --- a/docs/rfc/INDEX.md +++ b/docs/rfc/INDEX.md @@ -117,6 +117,7 @@ Generated by `pnpm run gen-rfc-index` from the RFC tree — never edit by hand; | [Tagged render-intent union for tool-call presentation](implemented/architecture/2026-07-02-tool-render-intent-union.md) | 2026-07-02 | | [Add direct directory listing to the filesystem seam](implemented/architecture/2026-07-03-filesystem-directory-listing-seam.md) | 2026-07-03 | | [Prompt variables and tool-guidance ownership](implemented/architecture/2026-07-05-prompt-variables-and-tool-guidance-ownership.md) | 2026-07-05 | +| [Every LLM request is reconstructable from the session log](implemented/architecture/2026-07-05-reconstructable-requests.md) | 2026-07-05 | | [Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`](implemented/architecture/2026-07-05-subagent-provider-lifecycle-events.md) | 2026-07-05 | ### Process diff --git a/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md new file mode 100644 index 0000000000..601e64f70a --- /dev/null +++ b/docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md @@ -0,0 +1,52 @@ +# RFC: Every LLM request is reconstructable from the session log + +Status: implemented + +## Problem + +Two gaps shared one root. First, provider KV caching (DeepSeek context caching) is prefix-based — a request pays full price only for the tokens after the longest stored prefix it matches — yet nothing in the request pipeline stated, checked, or measured prefix stability: every registered [`PromptSection`](../../../../packages/core/system-prompt/src/index.ts) happened to be static, the tool set happened not to change mid-session, no listener happened to rewrite requests. A single time-interpolating section would have silently multiplied context cost, and no test or metric would have moved. Second, and deeper: the session log — the system's single source of truth — could not actually answer *what the model saw*. It recorded every message but never the system prompt, the tool schemas, or even which model; the mutable `agent/request` waterfall handed listeners the whole `GenerateOptions` to rewrite per call; replay equivalence was therefore a property of the plugin population, not of the design. + +The reference shape for the happy path is MiniCode's `LLMClient`: a stateful conversation client, appended to — never rebuilt — as the conversation advances, resetting only when the system prompt, tool set, or compaction genuinely changes what the model must see. The design question this RFC answers is how to get that discipline without giving up event-sourcing. + +## Decision + +### The principle + +**Model-visible ⟺ logged.** Anything that reaches a model request must be recorded in the session log. The checkable consequence: **every conversation request the loop sends is a pure function of the session log** — anyone holding the log reconstructs it byte-for-byte. Scope, stated precisely: the guarantee covers the loop-built `GenerateOptions`; provider wire bytes follow from it because both adapters' serialization is a pure per-message function at a pinned code version; direct one-shots (compaction's summarize call) log their envelope scalars (`compact/summary.{model, maxTokens}`) and their input is deterministic code over the logged region — reconstructable from log + code, outside the invariant by the unfrozen-request marker. + +Prefix-cache stability is corollary #1, not the headline: an append-only log projected by a per-node pure function yields requests that are append-extensions of their predecessors whenever the header is unchanged — stability is emergent, not managed. Byte-exact audit/replay is corollary #2; resume and fork with *attributable* drift is corollary #3. + +### The mechanism + +**Messages.** `Session.deriveMessages()` is cached: each surface node is projected exactly once, when first seen, through the public per-node function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree. + +**The header.** The request's non-content half — `EpochHeader`: call config (`LlmCallConfig`: model + sampling scalars), rendered system prompt, assembled tool schemas — is logged session state, in canonical form (empty system/tools ≡ absent). Two log-only, turn-enclosed events in dsh-session carry it: `request/header`, a full snapshot with reason `'initial' | 'resume' | 'fallback'`, and `request/header-delta`, an amendment (`SystemDelta`: a common-prefix/suffix line trim; `ToolsDelta`: name-keyed added/removed/changed; `config`: replaced whole). The pure trio `foldRequestHeader` / `diffHeader` / `applyHeaderDelta` reconstructs; the live session tracks the fold with the same lazy cursor as the message cache. Snapshots anchor the fold where a fold needs anchors — conversation birth and process boundaries — and each loop instance appends one on its first request (`'initial'` when the log has none, `'resume'` otherwise, even when nothing changed: the boundary itself is a recorded fact, and cross-restart drift becomes attributable while an unchanged header resumes byte-identical). Deltas are an encoding optimization with a safety valve, never a correctness dependency: the writer verifies `applyHeaderDelta(prev, delta)` reproduces the new header exactly and records a `'fallback'` snapshot when the encoding cannot express a change (a pure tool reordering), so a well-formed log always folds. + +**The loop, transmission-stateless.** Per step: render assembly (every step — value comparison needs no change-signal discipline, and a section that varies per step surfaces as a *logged* header event per step instead of a silent bust) → `agent/pre-step` (compaction's surface mutations land before derivation) → **messages snapshot, then `step/start` appended as the next operation in the same synchronous frame** → seed the call config (first request of the instance: from `AgentOptions`, so explicit options always beat the logged baseline — fork model-overrides and resume reconfiguration stay correct; afterwards: from the folded header) → the `agent/request` waterfall, re-typed `(agent, turn, step, config: LlmCallConfig, next) → LlmCallConfig` — a frozen seed and a returned replacement are ALL a listener shapes; content flows through the log channels (`inject()`, steering, prompt-submit `additionalContext`, sections via `system-prompt/assemble`) — → the header event the request owes the log → build `GenerateOptions` from the snapshot + header, deep-freeze (`deepFreeze` exempts the `AbortSignal`, the one live control channel — freezing one breaks `AbortController.abort()`), dispatch. The loop's only in-process bookkeeping is one boolean: whether this instance has logged its anchoring snapshot. + +**The reconstruction boundary is `step/start`, unconditionally.** A step's messages are the derivation over `events[0..stepStartSeq)`. Because the snapshot precedes the `step/start` append in the same synchronous frame, nothing can enter this request past the boundary: an `agent.inject()` from an `agent/request` listener (or any concurrent task, or a `session/event` listener firing on `step/start` itself) lands in the log after the boundary and joins the NEXT request. For waterfall-window appends this matches the prior loop (it also derived before its waterfall); for a synchronous `step/start` listener it is a deliberate change — such a listener could previously reach the current request — and `agent/pre-step` is the sanctioned seam for content that must affect the CURRENT request. A step's header for reconstruction is the fold after its own `request/header*` event (which sits between its `step/start` and first response event) or the fold carried forward. + +**Enforcement.** Dev-mode ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)), on `llm/stream`: a frozen request with a live `sessionId` — the loop-built marker; hand-built one-shots are unfrozen and skipped — must carry messages deep-equal to the boundary derivation, rebuilt through a FRESH `Session` over `events[0..stepStartSeq)` so the live cache cannot vouch for itself, and header fields equal to `foldRequestHeader` over the log. There is no divergence allowance and nothing to allow: no seam can put unlogged content into a request. `prepend: true` only defends against the replay adapter's short-circuit (an append-registered listener); two prepended listeners have no defined mutual order in cordis, so correctness rests on the seq-bounded fold, never on listener timing. Measurement stays lean: the with-key e2e ([request-cache.e2e.ts](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts)) proves `usage.cacheReadTokens > 0` on every request after the first against the live API, and per-step usage in the log is the production observable — a header event or compaction shows up as a cache-read collapse on the next step. + +### The MiniCode shape: adopted, with the provenance arrow inverted + +What survives from `LLMClient`: the conversation is maintained, not rebuilt — one projection per message, ever; requests advance append-only; resets happen only for a system-prompt/tool change, a config change, or compaction, each now a *logged* fact. What is deliberately inverted: MiniCode's client is the source of truth and its event stream derives from client appends (`on_event(MessageAdded)`), which suits an advisory event stream. Here the log is contractual — persistence, crash recovery, fork seeding, transcript rendering, and the snapshot harness all replay it — and it carries strictly more than a message list (turn/step boundaries, raw chunk streams, tool-call pairing, provenance, log-only records), so a message-list client cannot generate it. The arrow therefore points log → client: the conversation state IS the log plus two cached folds inside `Session` (messages, header), and the "client" the loop talks to is the session itself. What the inversion buys over the original: the reconstruction is *checkable* against an independent record on every request — MiniCode's client has nothing to check itself against. + +## Alternatives considered + +- **Client as source of truth** (literal MiniCode): a second operative truth beside the log — the two drift and nothing notices; see the section above. +- **A stateful transmission client mirroring the log** (a `PromptPrefix` class holding committed/open message zones with an append/editTail/reset vocabulary, the log pushed into it per event): behaviorally equivalent on the happy path, but it duplicates conversation state outside the session, needs transactional rollback around listener seams, keeps an unlogged content-shaping surface (`editTail`) whose divergence the invariant must specially allow, and still cannot answer "what header did the model see" from the log. Dissolving it into the session's own caches plus logged header events made every one of those problems unrepresentable instead of guarded. (PR #162 is the archaeology of this alternative, three designs deep.) +- **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records. +- **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability. +- **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline. +- **Narrative fields on the header events** (a `reason`/`changed` list on deltas): derivable by diffing consecutive events — one home per fact; snapshots carry a reason because an anchor's cause is NOT derivable from the data. + +## Consequences + +- A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event. +- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replace node), a real prompt/tool change (`request/header-delta`), a config switch (ditto), a process boundary with drift (`'resume'` snapshot differing from its predecessor). The provider's own reasoning-content exclusion is managed server-side. +- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam. +- Tool-result trimming (planned) needs no new mechanism: a logged single-node surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic. +- Session logs grow one `request/header` snapshot per conversation (system + tool schemas: the dominant term), plus deltas on real changes — small next to `assistant/chunk` volume; `SESSION_FORMAT_VERSION` stays `0` (pre-release churn is absorbed, backends reject-not-migrate). +- Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths. +- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them. diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index a55722c53c..0bab45e1bb 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -26,10 +26,14 @@ declare module 'cordis' { interface Events { /** - * Waterfall around every streaming model call (retry, caching, routing). + * Waterfall around every streaming model call (retry, replay, routing). * Bound to the {@link LlmService}; call `next()` to reach the resolved * adapter's stream, or yield your own chunks to short-circuit. - * @param options - the full request; listeners may rewrite it before delegating. + * @param options - the full request. A LOOP-built request arrives + * deep-frozen (mutation throws): its content is a pure function of the + * session log (the reconstructability RFC), so listeners read it, never + * rewrite it. A hand-built one-shot (compaction summarize) is the + * caller's own object and stays mutable here. * @mode waterfall */ 'llm/stream'(this: LlmService, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 46390a6567..b5c648527e 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -8,6 +8,7 @@ { "doc": "docs/core-data-structures/core.md", "symbol": "FinishReasonMap", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "GenerateOptions", "source": "packages/llm/llm/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "ToolSchema", "source": "packages/llm/llm/src/types.ts" }, + { "doc": "docs/core-data-structures/core.md", "symbol": "LlmCallConfig", "source": "packages/llm/llm/src/call-config.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "SessionEvent", "source": "packages/core/session/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "Agent", "source": "packages/core/agent/src/types.ts" }, { "doc": "docs/core-data-structures/core.md", "symbol": "HookContext", "source": "packages/core/agent/src/types.ts" },