docs: document package KV cache effects
This commit is contained in:
@@ -29,6 +29,8 @@ Decided up front (per the [render-intent RFC](../../../docs/rfc/implemented/arch
|
||||
|
||||
**Token effect**: Small fixed guidance cost per request while the plugin is active.
|
||||
|
||||
**KV Cache effect**: Prefix-stable while the plugin scope and guidance text are unchanged. Activation or disposal may invalidate reuse from this prompt section.
|
||||
|
||||
#### Workflow guidance
|
||||
|
||||
```markdown
|
||||
@@ -41,12 +43,16 @@ Use the <toolName> tool ONLY when the user explicitly asks for a workflow or for
|
||||
|
||||
**Token effect**: Substantial fixed schema cost on each request where the tool is visible.
|
||||
|
||||
**KV Cache effect**: Prefix-stable while `toolName`, definition, and visibility are unchanged. Renaming, plugin lifecycle, or scoped restrictions may invalidate reuse from this schema.
|
||||
|
||||
### Tool-call history and result
|
||||
|
||||
**What the model sees**: The full model-written script, metadata, and args remain in the assistant tool call. Success is exactly `workflow "<name>" completed (<count> agent<optional-s>).`, newline, `Return value:`, newline, and pretty-printed data-dependent JSON; a cap adds `… [truncated: <omitted> more characters]` on a new line. Failures are exactly `Error: workflow run was cancelled`, optionally suffixed ` (<error>)`, `Error: workflow run failed: <error-or-unknown error>`, or defensively `Error: workflow run ended abnormally (<reason>)`; a call without an owning agent becomes `Error: workflow tool requires a calling agent (exec.agent was undefined)`. Intermediate child messages are omitted.
|
||||
|
||||
**Token effect**: Call tokens can be large and remain until compaction. Result rendering is capped by `maxResultChars`; child-model tokens are separate from the parent's retained context.
|
||||
|
||||
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The parent turn blocks until the whole workflow settles** — there is no background start/poll surface, and cancellation discards partial output as an error.
|
||||
|
||||
@@ -89,12 +89,16 @@ The host keeps a ledger of forwarded child starts. A graceful worker supplies th
|
||||
|
||||
**Token effect**: Potentially many independent child contexts are paid, bounded by `maxConcurrentAgents`, `maxTotalAgents`, and `maxItemsPerCall`; they never join the parent history directly.
|
||||
|
||||
**KV Cache effect**: Independent of the parent request cache and of sibling children. Each child can reuse only a byte-identical prefix under its own provider, model, prompt, and schema; its later history grows append-only.
|
||||
|
||||
### Parent tool result, indirectly
|
||||
|
||||
**What the model sees**: Through [`dsh-tool-workflow`](../tool-workflow/README.md), success exposes only the materialized final JSON value and child count in that consumer's wrapper. This engine supplies stable errors including `workflow script does not parse: <error>`, `invalid meta: <violations>`, `agent() requires a non-empty prompt string`, `agent() could not start a child: <error>`, `child agent run failed: <error>`, and its exact `parallel()`, `pipeline()`, `phase()`, option, schema, and JSON-boundary validation messages. Intermediate child outputs are available to the script but not the parent model.
|
||||
|
||||
**Token effect**: Zero direct parent tokens from this engine. Final result size is capped by the tool consumer and retained until compaction.
|
||||
|
||||
**KV Cache effect**: Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The worker/vm is not a security boundary** — model-written code can escape `node:vm` and reach the worker's process authority; a hostile-code deployment needs a separate-process or container engine.
|
||||
|
||||
@@ -42,6 +42,8 @@ A child that resolves normally with a non-completed stop reason is not an infras
|
||||
|
||||
Indirectly, through `dsh-tool-workflow` and a workflow engine, which create child-agent requests and return a retained parent tool result.
|
||||
|
||||
**KV Cache effect**: No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Foreground collection only** — the caller owns one live run and awaits it; background start/poll, spill handles, and detached collection are deferred.
|
||||
|
||||
Reference in New Issue
Block a user