Merge remote-tracking branch 'origin/master' into codex/project-instruction-files

# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/events.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/core.md
#	docs/event-producer-consumer.md
#	docs/persistence-catalog.md
#	docs/rfc/implemented/feature/2026-06-15-code-mode.md
#	docs/rfc/implemented/feature/2026-06-30-interception-seams.md
#	docs/rfc/proposed/simplification/2026-07-04-prune-dead-core-spine-surface.md
#	docs/tool-execution-pipeline.md
#	packages/cordis/tool-cordis/src/api-catalog.ts
#	packages/core/agent-core/tests/agent-core.spec.ts
#	packages/core/agent-loop/src/agent.ts
#	packages/core/agent/README.md
#	packages/core/session/src/index.ts
#	packages/core/tools/README.md
#	packages/core/tools/src/index.ts
#	pnpm-lock.yaml
#	scripts/gen-doc-graphs.ts
#	scripts/type-equiv.manifest.json
This commit is contained in:
Yichen Jiang
2026-07-13 14:37:32 +08:00
243 changed files with 14538 additions and 4798 deletions

View File

@@ -1,6 +1,6 @@
# dsh-tools
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the allow/deny gate) → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context). The registry also owns HOW its tools are presented to the model — its `mode` config selects native function calling, [Code Mode](#code-mode), or both.
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the extensible allow/deny gate) → monotonic registered guards → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context) → the observe-only `tools/result` notification. The registry also owns HOW its tools are presented to the model — its `mode` config selects native function calling, [Code Mode](#code-mode), or both.
## Service: `ToolRegistry` (ctx key: `tools`)
@@ -11,42 +11,42 @@ tools:
mode: native # native (default) | code | both
```
`native` contributes every registered tool as a wire function definition — the default, byte-for-byte the pre-config behavior. `code` contributes exactly ONE wire tool, `run_code`, plus the generated `tools:sdk` prompt section (see [Code Mode](#code-mode)). `both` contributes every native definition AND `run_code` + the SDK section. Non-native modes require a loaded `ctx.codeRuntime` with `language: 'typescript'`; a missing or mismatched runtime rejects every prompt assembly with an actionable error, and a `systemPrompt.toolOrder` naming tools the mode no longer contributes rejects the assembly the same way.
`native` contributes the calling agent's visible end capabilities as wire function definitions. Under `code`, this registry contributes the reserved `run_code` transport plus the generated `tools:sdk` prompt section (see [Code Mode](#code-mode)); `both` contributes the visible native definitions and both infrastructure pieces. Restrictions cannot remove `run_code`, and registering, shadowing, or explicitly filtering that reserved name fails loudly. An expert `system-prompt/assemble` listener may replace any prompt or schema contribution; its returned assembly is authoritative, so the listener owns preserving Code Mode when the protocol should remain active. Non-native modes require a loaded `ctx.codeRuntime` with `language: 'typescript'`; a missing or mismatched runtime rejects every prompt assembly with an actionable error, and a `systemPrompt.toolOrder` naming tools the mode no longer contributes rejects the assembly the same way.
### Public API
- `ctx.tools.register(definition: ToolDefinition): () => void` Register a tool. Disposed with the calling fiber.
- `ctx.tools.get(name: string): ToolDefinition | undefined`
- `ctx.tools.schemas(): ToolSchema[]` Schemas of all registered tools (without the `execute` functions). The shipped tools' schemas are catalogued in [docs/tool-catalog.md](../../../docs/tool-catalog.md), generated by booting each tool plugin and harvesting this method (see [the tool-schema-catalog RFC](../../../docs/rfc/implemented/process/2026-07-02-tool-schema-catalog.md)).
- `ctx.tools.execute(exec: ToolExecution): Promise<ToolExecutionResult>` Execute one tool call through the `tools/pre-execute``tools/execute``tools/post-execute` pipeline.
- `ctx.tools.register(definition: ToolDefinition): () => void` Register a trusted typed same-process definition. The layer is the calling context's scope: a plain plugin context registers globally; an agent's `agent.ctx` registers for that agent alone, shadowing a same-named global tool there. Duplicate names within one layer throw; non-native modes also reject the reserved `run_code` transport name. `timeoutMs`, when present, must be positive and finite. Disposed with the calling fiber.
- `ctx.tools.restrict(filter: ToolRestriction): () => void` Scoped-only (throws on a plain context): mask the global end-capability surface for the calling agent — `allow` keeps only the listed global tools, `deny` removes them; multiple restrictions intersect; scope-local registrations are merged afterward. The readonly arrays compile once into private sets. Every listed name must exist in the current pre-restriction global registry; scope-local, unknown, and reserved `run_code` names fail loudly. A deny-list admits a later global tool unless it names that tool; an allow-list excludes later names; neither filters a later scope-local registration. `restrict({})` rejects. This is live registration composition, not a parent-derived authority ceiling; see the [agent-scope security non-goal](../../../docs/rfc/implemented/architecture/2026-07-08-agent-scope-contexts.md#security-and-authority-are-explicit-non-goals).
- `ctx.tools.get(name: string, scope?: ScopeKey): ToolDefinition | undefined` Resolution as one scope sees it (shadowing applied; a restricted-away global reads as absent) — presenters pass the calling agent so the card matches what executed.
- `ctx.tools.schemas(scope?: ScopeKey): ToolSchema[]` Schemas of everything the scope can see (without the `execute` functions). The shipped tools' schemas are catalogued in [docs/tool-catalog.md](../../../docs/tool-catalog.md), generated by booting each tool plugin and harvesting this method (see [the tool-schema-catalog RFC](../../../docs/rfc/implemented/process/2026-07-02-tool-schema-catalog.md)).
- `ctx.tools.guard(guard: ToolGuard): () => void` Register a monotonic synchronous execution guard after `tools/pre-execute`: returning a reason denies the call, while `undefined` leaves it unchanged. A plain-context guard applies globally; an `agent.ctx` guard applies only to that agent. Later waterfall listeners cannot turn a guard denial back into permission. Disposed with the calling fiber.
- `ctx.tools.execute(exec: ToolExecutionInput): Promise<ToolExecutionResult>` Assign a fresh opaque correlation token, losslessly materialize and deep-freeze arguments once at the model/tool boundary, then run the call through `tools/pre-execute` → guards → `tools/execute``tools/post-execute`. Invalid arguments normalize through the same authoritative result path without reaching policy or the body. The final outcome is independently materialized and deep-frozen once before `tools/result`. Optional `signal` remains the operational field an around-dispatch wrapper may replace.
### Injected services
`SystemPrompt` — the registry automatically feeds its tool schemas into the system-prompt assembly via `ctx.systemPrompt.tools()`. The approval seam is consumed opportunistically instead (`ctx.get('approval')`, no static inject): a deployment without it keeps the ask→deny degrade, and the registry stays active either way.
### Events
### Live events
| Event | Mode | Purpose |
|---|---|---|
| `tools/pre-execute` | waterfall | Allow/deny gate BEFORE a tool runs (sandbox, permission, hooks); returns `PreToolDecision` |
| `tools/execute` | waterfall | Around-dispatch wrapper (timeout, retry, metrics): `(exec, next)` → the dispatched `ToolExecutionResult`; `next()` is dispatch-with-normalization |
| `tools/post-execute` | waterfall | Inspect/replace the result AFTER a tool runs, attach context; returns `PostToolDecision` |
| `tools/change` | emit | A tool was registered or unregistered |
The live registry pipeline has three transformable waterfalls followed by the observe-only `tools/result` boundary; registry changes are deliberately unfiltered shared-state notifications. Exact signatures, dispatch modes, scope filtering, and failure-containment contracts live in the generated [Cordis event catalog](../../../docs/cordis-catalog/events.md), while the complete ordering is visualized in the generated [tool execution pipeline](../../../docs/tool-execution-pipeline.md). `tools/result` is live; the similarly named `tool/result` is the durable session event the agent loop appends afterwards.
### Key types
- `ToolDefinition``ToolSchema` + `execute(args, exec: ToolRunContext): Promise<ContentBlock[] | { content: ContentBlock[]; meta? }>` (the bare array is the model-facing content; the object form additionally attaches an opaque, JSON-serializable `meta` presentation payload persisted on the `tool/result` event and handed back to `presentResult`), plus optional `presentCall(args)` / `presentResult(args, result)` for tool-owned UI presentation (see below). It also carries an optional cooperative timeout budget `timeoutMs?: number` (ms) enforced by `@deepseek-ai/dsh-timeout-policy`, never sent to the model.
- `ToolExecution` — one pending tool call: `{ callId, name, arguments, agent?, signal? }`.
- `ToolDefinition``ToolSchema` + `execute(args, exec)`, optional presentation callbacks, and cooperative `timeoutMs`.
- `ToolExecutionInput` — the caller-supplied call description: `{ callId, name, arguments, agent?, parent?, signal? }`; callers may pass an enclosing execution's opaque token as `parent` but never choose the new execution's own token.
- `ToolExecutionToken` — a fresh branded `Symbol` assigned by the registry. It supports equality correlation only and never crosses a model, log, or worker boundary.
- `ToolExecution` — the pipeline-owned call: immutable `{ token, callId, name, arguments, agent?, parent? }` identity plus optional operational `signal`, which an around wrapper may add, replace, remove, and restore. A nested call's `parent` is a `ToolExecutionToken`, not an execution object.
- `ToolRunContext` — the execution passed to a tool body, extending `ToolExecution` with `deferContext(context)`. Composite tools use it to ferry context produced by nested dispatches to the outer result even when the tool later throws; it never injects immediately.
- `ToolExecutionResult` — outcome: `{ callId, content, isError, error?, additionalContexts?, meta? }`. On failure with a `HarnessError`, `error: { name, code }` carries the structured failure class alongside the model-facing text (the loop forwards it onto the `tool/result` session event for retry/sandbox plugins and replay). `additionalContexts` preserves each deferred or post-execute `HookContext` with its own source, envelope, and durable JSON metadata; the loop buffers the array and appends each entry as a `context/message` after all `tool/result`s in the step. `meta` is the tool's opaque presentation payload from a successful `execute` (the object return form); the loop forwards it onto the `tool/result` session event for result-card rendering.
- `PreToolDecision``{kind:'allow'}` | `{kind:'deny', reason}` | `{kind:'ask', reason?}`. Input rewrite (changing `arguments`) is deliberately NOT offered (it would desync the pre-execution audit/history/UI from what ran — its own proposed RFC); `ask` is serviced by [`ctx.approval`](../../ui/user-approval/README.md) when a deployment mounts it (`allowed-once` proceeds to dispatch; `rejected`/`cancelled`/`unavailable` deny with distinct reasons) and degrades to `deny` when none is mounted or the execution carries no agent.
- `PostToolDecision``{kind:'accept', content?, additionalContexts?}` (keep the call successful, optionally replacing the model-facing content) | `{kind:'block', feedback, additionalContexts?}` (turn it into an `isError` whose content is the corrective feedback). Accept preserves tool-deferred contexts before decision contexts; block discards tool-deferred contexts and exposes only contexts explicitly supplied by the blocking decision. Output replacement is clean because `tool/result` is logged AFTER `execute()` returns.
- `ToolExecutionResult` losslessly JSON-serializable outcome: `{ callId, content, isError, error?, additionalContexts?, meta? }`. The registry materializes and freezes the complete post-policy value before final observation. On failure with a `HarnessError`, `error: { name, code }` carries the structured failure class alongside the model-facing text. `additionalContexts` preserves each deferred or post-execute `HookContext` with its own source, envelope, and durable JSON metadata; the loop buffers the array and appends each entry as a `context/message` after all `tool/result`s in the step.
- `PreToolDecision``{kind:'allow'}` | `{kind:'deny', reason}` | `{kind:'ask', reason?}`. Input rewrite is deliberately not offered; `ask` is serviced by [`ctx.approval`](../../ui/user-approval/README.md) when mounted and otherwise degrades to deny.
- `PostToolDecision``{kind:'accept', content?, additionalContexts?}` (keep the call successful, optionally replacing the model-facing content) | `{kind:'block', feedback, additionalContexts?}` (turn it into an `isError` whose content is the corrective feedback). Accept preserves tool-deferred contexts before decision contexts; block discards tool-deferred contexts and exposes only contexts explicitly supplied by the blocking decision.
- `ToolGuard``(execution) => string | undefined`; the returned string is a final monotonic denial reason evaluated after the reorderable pre-execute waterfall and before dispatch.
- `ToolCallView` / `ToolResultView` — provider-neutral `card`-tagged render intents a tool returns from `presentCall` / `presentResult` to own how a UI renders ITS calls (see "Tool-owned UI presentation").
### Extension points
- Tool plugins call `ctx.tools.register()` — schemas flow into the assembly automatically.
- `tools/pre-execute` is the allow/deny gate (sandbox, permission, hooks): listeners receive `(exec, next)` and call `next()` to delegate to the default (allow) or return a `PreToolDecision` to short-circuit; a `deny` skips dispatch and yields an `isError` result, and an `ask` resolves through the approval seam first — only a grant dispatches (see `PreToolDecision` above). `tools/execute` is the around-dispatch seam (timeout, retry, metrics): listeners receive `(exec, next)` and call `next()` to delegate to core dispatch (returning its `ToolExecutionResult`, optionally wrapped), or return a replacement result to short-circuit dispatch; the base `next()` IS dispatch-with-normalization, so `await next()` already yields an `isError` result for a thrown/unknown tool (never a raw throw). A wrapper mutates `exec` in place before `next()` — e.g. replacing `exec.signal` with a per-call deadline — because cordis `next()` ignores passed arguments. `tools/post-execute` is the inspect/transform seam: `(exec, result, next)` → a `PostToolDecision` that can replace content, block with feedback, or attach `additionalContexts`. Core dispatch is the base of the `tools/execute` waterfall; the tool body keeps its own try/catch so a thrown tool still reaches `post-execute` as an `isError`. All follow the typed-Decision idiom shared with the `agent/*` interception seams (see [`dsh-agent`](../agent/README.md)); `@deepseek-ai/dsh-timeout-policy` is the reference `tools/execute` wrapper.
- `tools/pre-execute` is the reorderable allow/deny/ask gate (sandbox, permission, hooks): listeners receive `(exec, next)` and call `next()` to delegate to the default (allow) or return a `PreToolDecision` to short-circuit; a `deny` skips dispatch, while an `ask` resolves through the approval seam and dispatches only after a grant. Either non-grant path yields an `isError` result. `ctx.tools.guard()` installs scope-aware monotonic policy after that waterfall when a denial must not be overridable by listener ordering. `tools/execute` is the around-dispatch seam (timeout, retry, metrics): listeners receive `(exec, next)` and call `next()` to delegate to core dispatch (returning its `ToolExecutionResult`, optionally wrapped), or return a replacement result to short-circuit dispatch; the base `next()` is dispatch-with-normalization, so `await next()` already yields an `isError` result for a thrown or unknown tool. A wrapper may change only `exec.signal` before `next()`—adding a per-call deadline, replacing a caller signal, or restoring absence afterwards—because call identity is protected before policy begins. `tools/post-execute` is the inspect/transform seam: `(exec, result, next)` → a `PostToolDecision` that can replace content, block with feedback, or attach `additionalContexts`. Core dispatch is the base of the `tools/execute` waterfall; the tool body keeps its own error boundary so a thrown tool still reaches `post-execute` as an `isError`. Finally, `tools/result` observes the immutable authoritative result after every transform and error boundary. All follow the typed-decision idiom shared with the `agent/*` seams (see [`dsh-agent`](../agent/README.md)); `@deepseek-ai/dsh-timeout-policy` is the reference `tools/execute` wrapper.
- MCP servers: one plugin per server, discover tools, call `ctx.tools.register()` with the server's schemas.
### Typed tool parameter schemas
@@ -76,7 +76,7 @@ ctx.tools.register(defineTool({
}))
```
The helper converts the author-facing `SchemaSpec` (with `required: true` as a per-property boolean) to standard JSON Schema for the wire format. Raw JSON-Schema tool definitions (from MCP servers) are still accepted by the registry directly.
The helper converts the author-facing `SchemaSpec` (with `required: true` as a per-property boolean) to standard JSON Schema for the wire format and uses the same typed spec for execute/presentation validation. Raw JSON-Schema tool definitions (from MCP servers) are still accepted by the registry directly.
A `defineTool` tool also **validates the model-generated arguments against its `SchemaSpec` before `execute` runs** (`validateArgs`). The model's JSON is untrusted — `InferArgs<S>` is a compile-time claim, not a runtime guarantee — so on a mismatch (missing required key, wrong primitive, bad enum member, nested violation) the tool throws a `ToolArgsError` (`code: 'INVALID_ARGS'`); the registry turns it into an `isError` result whose text lists the violations, which the model sees and self-corrects from. Validation mirrors the JSON Schema conversion exactly: extra keys are allowed, `default` is not applied, and an `object`/`array` prop without `properties`/`items` only type-checks. Raw-registered tools (MCP) are **not** validated by the harness — they validate their own input.
@@ -131,15 +131,15 @@ const bash = defineTool({
### Code Mode
Under `mode: code` (or `both`) the registry turns the tool surface into a programming API, per the [Code Mode RFC](../../../docs/rfc/implemented/feature/2026-06-15-code-mode.md): the model writes a TypeScript program (the body of an async function) and passes it to the ONE wire tool `run_code`; the program runs in `ctx.codeRuntime` (the [code-execution seam](../../code-runtime/README.md) — the shipped backend is a worker thread) with one async binding per registered tool (`await tools.bash({...})`), and ONLY what it prints or returns re-enters the model's context.
Under `mode: code` (or `both`) the registry turns the tool surface into a programming API, per the [Code Mode RFC](../../../docs/rfc/implemented/feature/2026-06-15-code-mode.md): the model writes a TypeScript program (the body of an async function) and passes it to the reserved wire transport `run_code`; the program runs in `ctx.codeRuntime` (the [code-execution seam](../../code-runtime/README.md) — the shipped backend is a worker thread) with one async binding per visible end-capability tool (`await tools.bash({...})`), and ONLY what it prints or returns re-enters the model's context. Scope restrictions change those SDK bindings but cannot remove or replace the transport itself.
- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, a `declare const tools: {...}` TypeScript declaration of every registered tool except `run_code` (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) is TOTAL: constructs outside the `defineTool` subset degrade to `unknown`, never throw.
- **The dispatch bridge** (`run_code`'s execute): every binding call is JSON-normalized BEFORE dispatch (a value that does not survive — `BigInt`, circulars — rejects that one call, so the dispatched form and the logged form are the same JSON value by construction), serialized through a per-run queue (even `Promise.all` executes the underlying `ctx.tools.execute()` calls one at a time in submission order — the tool contract carries no concurrency-safety metadata yet), gated by `tools/pre-execute`/`tools/post-execute` like any native call (a deny reaches the program as a binding rejection), and logged as one `tool/code-dispatch` session event (log-only: `deriveMessages()` never surfaces it) with the deterministic sub-id `<parent>:code:<n>`. A failed sub-call REJECTS the program-side promise with the tool's error text — real code error handling, no bespoke envelope. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/envelope/meta even when the program later fails.
- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, a `declare const tools: {...}` TypeScript declaration of the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) is total: constructs outside the `defineTool` subset degrade to `unknown`, never throw.
- **The dispatch bridge** (`run_code`'s execute): every binding call is JSON-normalized before dispatch (a value that does not survive — `BigInt`, circulars — rejects that one call, so the dispatched form and logged form are the same JSON value by construction), serialized through a per-run queue (even `Promise.all` executes underlying calls one at a time in submission order), given the outer execution's opaque token as `parent`, and run through the complete pre-execute → guards → execute → post-execute → result pipeline. A denial reaches the program as a binding rejection, and each sub-call is logged as a `tool/code-dispatch` session event with deterministic id `<parent>:code:<n>`; `deriveMessages()` does not surface that event. Token correlation lets commit-style observers defer an inner success until the final `run_code` result without exposing the live outer execution; ordinary tool side effects are not rolled back. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/envelope/meta even when the program later fails.
- **Settlement discipline**: the bridge owns a run-scoped abort that follows the outer signal in and fires when the run settles for any reason, so a budget expiry aborts an in-flight sub-tool instead of orphaning it; the bridge then drains its queue BEFORE returning, so every `tool/code-dispatch` lands inside the open turn. A failed run throws `CodeRunFailedError` (`code: 'CODE_RUN_FAILED'`, message = the failure kind + captured logs), which the pipeline converts to a structured `isError` the model self-corrects from.
The wire collapse is the registry's own contribution (`systemPrompt.tools()` is mode-aware), so the logged `request/header` records it for free — under `code`, the assembled tool list is exactly `[run_code]`, pinned by tests and the snapshot goldens. Try it: `pnpm run demo:code-mode` ([the coding-agent example's Code Mode overlay](../../../examples/coding-agent/README.md#code-mode)); `pnpm run demo:code-mode acp` serves the same mode over ACP instead of the REPL.
The wire collapse is the registry's own contribution (`systemPrompt.tools()` is mode-aware), so the logged `request/header` records it for free. When no assembly listener changes the registry's prompt or schema contributions, `code` assembles exactly `[run_code]`, pinned by tests and the snapshot goldens. Try it: `pnpm run demo:code-mode` ([the coding-agent example's Code Mode overlay](../../../examples/coding-agent/README.md#code-mode)); `pnpm run demo:code-mode acp` serves the same mode over ACP instead of the REPL.
### What is NOT here (TODO)
- **Tool shapes review** — when real tools land (e.g. a concurrency-safety hint for parallel execution); phase 1 executes tool calls sequentially.
- **Parallel execution** — the loop currently iterates tool calls sequentially.
- **Concurrency metadata** — tool definitions do not declare whether executions are safe to overlap.
- **Parallel execution** — the loop and Code Mode bridge execute tool calls sequentially until that metadata exists.