docs: document package KV cache effects

This commit is contained in:
Tianyi Cui
2026-07-19 17:39:50 +08:00
parent b3c4b732d8
commit bcc920369c
93 changed files with 366 additions and 34 deletions

View File

@@ -98,30 +98,40 @@ The JSON-RPC frames go on stdout, so this plugin MUST run in an example that loa
**Token effect**: Prompt tokens are data-dependent and remain in that session's history until compaction. Concurrent ACP sessions keep separate contexts.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Human answers and permission decisions
**What the model sees**: When optional consumers are loaded, ACP form answers become the exact JSON shape documented by `dsh-tool-ask-user`. Failures become `Error: ACP user questions must come from an agent-owned request`, `Error: ACP user question has no matching session`, `Error: ACP elicitation request failed`, `Error: ask_user_question was cancelled by the user`, `Error: ask_user_question returned no answer`, or `Error: ask_user_question was aborted before the user answered`. Permission decisions control whether another tool yields success or denial. ACP tool cards, terminal output, diffs, and streamed session updates are UI-only.
**Token effect**: Answer, error, and denial text enters context only through the owning tool result; presentation metadata adds zero model tokens.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Permission preset switches
**What the model sees**: `session/set_config_option` emits no model message itself. When `dsh-permission` is composed, the bridge writes the selected preset through that service; the resulting model-visible policy prompt and change notice belong to [`dsh-user-approval`](../user-approval/README.md), while sandbox-mode effects belong to [`dsh-tool-bash`](../../bash/tool-bash/README.md). The ACP `Permissions` select, its option descriptions, pending idle value, and refreshed config response remain client-only.
**Token effect**: Zero direct tokens from the ACP option or the log-only `permission/preset` event. Downstream cost is limited to the owning plugins' policy prompt, conditional retained change notice, and any changed tool outcome.
**KV Cache effect**: The ACP option and log event cause no direct invalidation. The downstream policy-prompt change may invalidate reuse from that system section, while its change notice appends to history.
### Model switches
**What the model sees**: The ACP selector itself emits no message. The selected provider/model pair supplies the next step's `{{provider}}` / `{{model}}` prompt variables and request routing together; all other call-config fields continue through the `agent/request` waterfall unchanged.
**Token effect**: The selector adds no direct tokens. A changed model may tokenize the same retained prompt/history differently, and any persona text that interpolates provider or model changes accordingly.
**KV Cache effect**: Switching provider or model selects a different cache domain. If the persona interpolates either value, the rendered system prompt also changes and prevents reuse from its first changed token.
### Loaded sessions
**What the model sees**: `session/load` resumes the persisted log, after which the loop sends its reconstructed history and request header. Replaying that log to the editor is not an extra model message.
**Token effect**: Restored context has the persistence and session packages' normal retained cost; ACP replay to the client adds none.
**KV Cache effect**: Loading does not rewrite the stored log, but the next request is reconstructed under the current envelope and route. Reuse requires that reconstruction to match; ACP replay to the client has no cache effect.
## Known Limitations and Deferred Work
- **`additionalDirectories`** — rejected. A session operates in its single `cwd` (see Per-session cwd); widening the tool/filesystem scope to extra roots is a separate sandbox concern, not yet implemented.

View File

@@ -20,6 +20,8 @@ This package carries no loader hooks and no dev-mode surface: the `dsh-scripts`
Indirectly, through the plugin tree it loads, which determines the prompts, schemas, messages, and model adapter in the resulting application.
**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **Bare package specifiers depend on Loader internals** — production bins need `node --expose-internals` or the Loader's optional native fallback; an in-process caller without either must use resolvable relative/file specifiers or tsx path mapping.

View File

@@ -30,6 +30,8 @@ The plugin answers `shutdown`, disposes SDK-owned agents and subscriptions to qu
**Token effect**: Data-dependent user-message tokens enter retained session history and are resent on later turns until another package compacts them. The JSON-RPC frames, session notifications, and server bookkeeping add zero model-context tokens.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **The wire has no per-session close or prompt-cancel method** — SDK-created agents remain live until process shutdown, and one accepted prompt runs to agent idle before that session accepts another.

View File

@@ -10,6 +10,8 @@ The service requires a confining `ctx.bash` executor and `ctx.approval`. A table
Indirectly, through `dsh-user-approval` and `dsh-tool-bash`, which render the approval-policy prompt, switch notice, and sandboxed tool outcomes selected by this service's knob events; `permission/preset` itself is log-only.
**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **Only two mechanism knobs are bundled** — presets select sandbox mode and approval policy; an agent/profile choice is not part of `PresetSpec` yet.

View File

@@ -29,12 +29,16 @@ The plugin seeds display labels from the live agent registry, then tracks `agent
**Token effect**: Submitted text is retained under the agent loop's normal session-history and compaction rules. The welcome banner, `> ` prompt, rendered transcript, and `[tool call]` / `[tool result]` terminal lines add no tokens.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Terminal user-interaction answers
**What the model sees**: When a consumer calls `ctx.userInteraction.ask()`, this provider renders the question in the terminal and returns selected option labels or `custom` text. Through `dsh-tool-ask-user`, closed stdin becomes `Error: ask_user_question cannot be answered because stdin is closed`; disposal or abort becomes `Error: ask_user_question was interrupted before the user answered`.
**Token effect**: Waiting and terminal prompts add no tokens; the resolved answer or error is model-visible only through the calling tool or plugin's result.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **One configured session receives stdin** — the session/event renderer can print output from any session, but input lines always drive the configured `sessionId` rather than routing by the visible label.

View File

@@ -27,12 +27,16 @@ This is the consumer package for the user-interaction seam. It does not render U
**Token effect**: Fixed schema cost on every request where the tool is visible.
**KV Cache effect**: Prefix-stable while the definition and visibility are unchanged. Plugin lifecycle or scoped restrictions may invalidate reuse from this schema.
### Tool-call history and result
**What the model sees**: The model's full questions remain in the assistant tool-call arguments. After the human answers, the next step sees compact JSON in the exact shape `{"answers":[{"id":"<id>","selected":["<label>"],"custom":"<text>"}]}`; `custom` is omitted when unused and `selected` can contain zero, one, or several labels. UI interaction while the call is pending is not model context.
**Token effect**: Arguments and answer JSON are data-dependent retained tokens; there is no token cost while waiting for the human.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **A pending question blocks the tool call until the human answers** — the tool declares no `timeout-policy` budget; cancellation rides the turn's `exec.signal` only.

View File

@@ -51,12 +51,16 @@ The palette uses the standard 16-color ANSI foregrounds and SGR attributes, whic
**Token effect**: Submitted text is retained under the agent loop's normal session-history and compaction rules. Headers, cards, Markdown rendering, status lines, plans, and help text add no tokens.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
### Interactive user-question answers
**What the model sees**: When a consumer calls `ctx.userInteraction.ask()`, this provider presents each question in order and returns selected option labels or `custom` text. Abort, cancellation, or UI disposal becomes `Error: ask_user_question was interrupted before the user answered` through `dsh-tool-ask-user`.
**Token effect**: Waiting and terminal overlays add no tokens; the resolved answer or error is model-visible only through the calling tool or plugin's result.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **One configured session owns the transcript and editor** — questions from other agents can still use the shared overlay provider, but session rendering and prompt input remain bound to `sessionId`.

View File

@@ -18,6 +18,8 @@ The tools pipeline routes `ask` decisions through this seam and fails closed whe
**Token effect**: Small fixed per-request cost, larger under `never`; a change notice is conditional and retained in history.
**KV Cache effect**: Prefix-stable while the approval policy is unchanged. An `ask`/`never` switch changes the system-prompt section and invalidates reuse from its first changed token; the accompanying notice is append-only.
#### Ask-policy prompt section
```markdown
@@ -37,6 +39,8 @@ Approval prompts are disabled in this session: actions that require approval are
**Token effect**: Zero duplicate audit tokens. A rejection may replace a normal tool result with a small retained error, while an allowance leaves the consumer's ordinary result.
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
## Known Limitations and Deferred Work
- **Requests are valid only inside an open turn** — an idle or between-turn caller throws before auditing; a durable out-of-turn approval workflow is deferred.

View File

@@ -27,6 +27,8 @@ This is the interface package. Model-facing consumers such as `@deepseek-ai/dsh-
Indirectly, through `dsh-tool-ask-user`, which retains a successful provider answer as compact JSON or one of these failures: `Error: ask_user_question was aborted before the user answered`, `Error: ask_user_question requires at least one question`, `Error: no user-interaction provider is registered`, or `Error: <message>`. Waiting for the human adds no tokens.
**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **One provider per context** — there is no routing or fan-out to multiple UIs; a second registration throws `DUPLICATE_PROVIDER`, and with none registered `ask()` throws `NO_PROVIDER` rather than degrading.