docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -65,12 +65,16 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and
**Token effect**: Zero direct tokens from this interface. A backend trades many retained history tokens for one summary and leaves the recent tail unchanged.
**KV Cache effect**: A successful backend replacement invalidates reuse from the first shadowed history token; the seam itself does not alter a request.
### Transcript supplied to a compaction consumer
**What the model sees**: `renderTranscript()` joins entries with one blank line and renders them exactly as `User: <content>`, `Assistant: <content>`, `Tool result (call <callId>): <content>`, `Tool error (call <callId>): <content>`, `[Context: <content>]`, or `[Steering: <content>]`. Non-text blocks render exactly as `[reasoning: <text>]`, `[tool-call: <name>(<arguments>)]`, `[tool-result: <content>]`, `[tool-result]`, or `[<block-type>]`.
**Token effect**: Data-dependent input tokens are paid only by the auxiliary model or consumer that requests this transcript; the conversation model does not receive a duplicate transcript.
**KV Cache effect**: No conversation-cache invalidation. A consumer's auxiliary request can reuse only the exact prefix produced by this rendering; changed or compacted entries invalidate reuse from their first difference.
## Known Limitations and Deferred Work
- **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener.