docs: document package KV cache effects
This commit is contained in:
@@ -56,12 +56,16 @@ Unit suites run against a local `node:http` mock SSE server (no network). Real-A
|
||||
|
||||
**Token effect**: Provider tokenization governs exact input. Conditional reasoning passback increases tool-round-trip context, while dropping other reasoning avoids paying those tokens again; cache-read usage is reported when available.
|
||||
|
||||
**KV Cache effect**: An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, or history change may prevent reuse from the first changed token; reasoning passback appends during tool round trips.
|
||||
|
||||
### DeepSeek response
|
||||
|
||||
**What the model sees**: Reasoning, text, and raw-string tool arguments are translated into harness chunks for the loop to log and assemble.
|
||||
|
||||
**Token effect**: Generated tokens follow provider thinking and effort settings plus the request's `maxTokens`; only loop-retained blocks affect later input.
|
||||
|
||||
**KV Cache effect**: Loop-retained response blocks append to the next request and preserve its earlier reusable prefix; dropped blocks have no later cache effect. Changing the provider or model selects a different cache domain.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **`tool_choice` is not mapped** — not part of the core vocabulary (MVP cut, shared with the pi-ai twin).
|
||||
|
||||
Reference in New Issue
Block a user