docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -83,12 +83,16 @@ Swap `llm-deepseek` for a `mock-llm` leaf plugin and you have the echo demo —
**Token effect**: Child prompt and schema costs repeat per request; user input and tool history grow until compaction. Terminal banners, logger output, cards, and rendered transcripts add zero model tokens.
**KV Cache effect**: User and tool history is append-only while the composed prompt, schemas, child model route, and session prefix remain fixed. A composition change or compaction may invalidate reuse from its first changed token; terminal rendering has no cache effect.
### Human-answer result
**What the model sees**: Through `dsh-tool-ask-user`, successful terminal answers use that package's exact compact JSON shape. Interruption becomes exactly `Error: ask_user_question was interrupted before the user answered`; a closed stdin becomes `Error: ask_user_question cannot be answered because stdin is closed`.
**Token effect**: Only a completed or failed tool call adds retained result tokens; prompts printed while waiting are terminal-only.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **One pre-created `main` agent drives the selected terminal UI** — there is no multi-session or concurrent-agent surface in this app; a run is one conversation.