docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -27,12 +27,16 @@ This is the consumer package for the user-interaction seam. It does not render U
**Token effect**: Fixed schema cost on every request where the tool is visible.
**KV Cache effect**: Prefix-stable while the definition and visibility are unchanged. Plugin lifecycle or scoped restrictions may invalidate reuse from this schema.
### Tool-call history and result
**What the model sees**: The model's full questions remain in the assistant tool-call arguments. After the human answers, the next step sees compact JSON in the exact shape `{"answers":[{"id":"<id>","selected":["<label>"],"custom":"<text>"}]}`; `custom` is omitted when unused and `selected` can contain zero, one, or several labels. UI interaction while the call is pending is not model context.
**Token effect**: Arguments and answer JSON are data-dependent retained tokens; there is no token cost while waiting for the human.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **A pending question blocks the tool call until the human answers** — the tool declares no `timeout-policy` budget; cancellation rides the turn's `exec.signal` only.