docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -56,12 +56,16 @@ Unit suites run against a local `node:http` mock SSE server (no network). Real-A
**Token effect**: Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available.
**KV Cache effect**: An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips.
### DeepSeek response
**What the model sees**: Reasoning, text, and raw-string tool arguments are translated into harness chunks for the loop to log and assemble.
**Token effect**: Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input.
**KV Cache effect**: Loop-retained response blocks append to the next request and preserve its earlier reusable prefix; dropped blocks have no later cache effect. Changing the provider or model selects a different cache domain.
## Known Limitations and Deferred Work
- **`tool_choice` is not mapped** — not part of the core vocabulary (MVP cut, shared with the pi-ai twin).