docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -76,18 +76,24 @@ Everything that goes beyond "call the model, run the tools, repeat" belongs to p
**Token effect**: System text, schemas, and prefix are paid again on every step. Per-agent scoping chooses the initial contributions, while the authoritative assembly waterfall can alter the final request and makes its listener responsible for protocol coherence.
**KV Cache effect**: Append-only only while system text, schemas, session prefix, and earlier history remain byte-identical under the same provider and model route. A token-bearing assembly rewrite or composition change may invalidate reuse from the first altered request token.
### Retained message history
**What the model sees**: Accepted user messages, assistant messages, tool calls and results, injected context, and steering are logged and sent on later steps. Raw stream chunks, lifecycle boundaries, and other log-only events are excluded.
**Token effect**: Input grows with every surface message until a compaction replacement shadows older nodes; a multi-step tool turn resends the accumulated prefix and history each step.
**KV Cache effect**: Ordinary history growth is append-only and preserves reusable entries. A surface replacement or compaction invalidates reuse from the first shadowed history token.
### Undispatched calls after cancellation
**What the model sees**: If a later request replays an aborted step, each tool call that cancellation prevented from dispatching has the error result text `Error: tool call skipped because the step was aborted before execution`.
**Token effect**: One fixed error result per skipped call remains in history until compaction shadows it.
**KV Cache effect**: Append-only; each synthetic result follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **Classification is unary** — calls whose safety depends on comparing siblings or resources must remain exclusive ([rationale](../../../docs/rfc/implemented/feature/2026-07-10-parallel-tool-call-execution.md)).

View File

@@ -75,12 +75,16 @@ The handle every plugin programs against:
**Token effect**: Accepted content becomes retained history or a repeated session prefix; blocked content contributes no request tokens. Size is caller- and plugin-dependent.
**KV Cache effect**: Accepted history and steering are append-only; a blocked submission sends no request. A session prefix remains stable within its loop instance, while a new or resumed instance may establish a different prefix.
### Agent-scoped request composition
**What the model sees**: Registrations through `agent.ctx` can shadow prompt sections or tools and can install agent-only interceptors during unpublished setup.
**Token effect**: The package adds zero tokens itself; scoped contributions affect only that agent and disappear on disposal.
**KV Cache effect**: Prefix-stable while an agent's scoped registrations are unchanged. Setup or reload that changes prompt sections, tool definitions, or request listeners may invalidate reuse from the first affected request token.
## Known Limitations and Deferred Work
- **Initiator scope is process-local** — workers, child processes, HTTP, durable queues, and restarts materialize any required identity explicitly.

View File

@@ -89,18 +89,24 @@ Every `SessionEvent` carries two optional top-level fields (structural metadata)
**Token effect**: Appended surface entries are resent on later steps. A `replace` surface operation removes the shadowed entries from future inputs without deleting their raw log records.
**KV Cache effect**: Appended surface entries preserve reusable prefixes. A `replace` operation invalidates reuse from the first shadowed message even though the underlying event log stays append-only.
### Crash-repair result
**What the model sees**: If a persisted turn ended with unanswered tool calls, each synthetic error result contains exactly `Tool call interrupted by a crash; no result was recorded.`
**Token effect**: Zero tokens in an intact session. Each repaired call adds this retained error text on resume.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Logged request header
**What the model sees**: The session reconstructs the system prompt, tool schemas, call config, and session prefix that the loop actually sent. Header events do not add a second copy to message history; the prefix is prepended outside `deriveMessages()`.
**Token effect**: Zero duplicate tokens from logging. The reconstructed prefix, system text, and schemas still incur their normal per-request cost.
**KV Cache effect**: Logging causes no invalidation, and exact reconstruction preserves request-prefix identity. A later header with changed prefix, prompt, or schemas may invalidate reuse from its first difference.
## Known Limitations and Deferred Work
- **Session branching/tree** (pi-style entry tree) — deferred unless needed beyond boundary-based `fork()`.

View File

@@ -48,6 +48,8 @@ Design rationale: [the prompt-variables RFC](../../../docs/rfc/implemented/archi
**Token effect**: Identity is a fixed per-request cost. Persona and plugin text are repeated per request and scale with their rendered content.
**KV Cache effect**: Prefix-stable while identity, persona, variables, section text, and order render identically. Any change may invalidate reuse from the first changed system-prompt token.
#### Harness identity
```markdown
@@ -60,6 +62,8 @@ You are an AI agent powered by the DeepSeek Harness SDK.
**Token effect**: Schema tokens repeat on every request. Restricting a tool removes its entire schema cost for that agent but not a separate prompt section; reordering changes cache shape but not semantic content.
**KV Cache effect**: Prefix-stable while the visible schema set, rendering, and order are unchanged. Registration, restriction, or reordering may invalidate reuse from the first changed schema token.
## Known Limitations and Deferred Work
- **Deployment-authored prompt text is config/composition only** — this plugin owns the global persona default, creator plugins may register agent-scoped shadows, and other sections come from the plugin that owns the fact; there is no end-user prompt-editing API.

View File

@@ -123,12 +123,16 @@ The agent loop groups consecutive `parallel` calls into a bounded rolling pool a
**Token effect**: Fixed per-request cost proportional to the visible definitions. Restrictions that hide tools remove their entire schema cost for that agent.
**KV Cache effect**: Prefix-stable while visible definitions and their order are unchanged. Registration, disposal, or scoped restriction may invalidate reuse from the first changed schema token.
### Code Mode schema and system prompt
**What the model sees**: Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact `declare const tools` block. `both` exposes normal schemas and this Code Mode surface.
**Token effect**: Fixed per-request cost proportional to the visible definitions. Code Mode trades end-tool schemas for generated SDK text plus one transport schema rather than promising a universal reduction.
**KV Cache effect**: Prefix-stable while the Code Mode selection, generated SDK, transport schema, and visible tool set are unchanged. Mode or filter changes may invalidate reuse from the first changed prompt or schema token.
#### Code Mode SDK instructions
```markdown
@@ -150,6 +154,8 @@ The available tools:
**Token effect**: Arguments, results, and additional context are data-dependent and resent until compaction. Restrictions that hide tools also remove their schemas before the model can call them.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **Concurrency policy is not an event seam** — `executionMode()` reads the resolved tool definition directly; plugins can only declare a classifier on definitions they own.