Merge commit 'refs/codex-unblock/20260723/pr570-master-initial' into worktree/pr570-merge-master-20260723

# Conflicts:
#	.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md
#	docs/core-data-structures/persistence.md
#	packages/session-persistence/session-persistence/README.md
This commit is contained in:
Tianyi Cui
2026-07-23 23:21:08 +08:00
366 changed files with 11617 additions and 1027 deletions

View File

@@ -67,7 +67,7 @@ Within a step, exclusive calls form barriers; parallel-safe calls use a bounded
### What belongs to plugins
Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy:
- Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute``tools/execute``tools/post-execute``tools/result` pipeline; exact signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md)
- Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute``tools/execute``tools/post-execute` definition-owned `finalizeContent` `tools/result` pipeline; exact event signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md)
- Compaction: pressure on `agent/post-step`; canonical context overflow on `agent/request-error`
- Transient model recovery: `dsh-llm-retry` on `agent/request-error`, with finite code-specific budgets and non-surface `llm/retry` status events
- Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation

View File

@@ -1,6 +1,6 @@
# dsh-tools
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the extensible allow/deny gate) → monotonic registered guards → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context) → the observe-only `tools/result` notification. The registry also owns HOW its tools are presented to the model — its `mode` config selects native function calling, [Code Mode](#code-mode), or both.
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the extensible allow/deny gate) → monotonic registered guards → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context) → the definition-owned `finalizeContent` boundary → the observe-only `tools/result` notification. The registry also owns HOW its tools are presented to the model — its `mode` config selects native function calling, [Code Mode](#code-mode), or both.
## Service: `ToolRegistry` (ctx key: `tools`)
@@ -15,7 +15,7 @@ tools:
### Public API
- `ctx.tools.register(definition: ToolDefinition): () => void` Register a trusted typed same-process definition with a mandatory canonical `output` declaration. The layer is the calling context's scope: a plain plugin context registers globally; an agent's `agent.ctx` registers for that agent alone, shadowing a same-named global tool there. Duplicate names within one layer throw; non-native modes also reject the reserved `run_code` transport name. Missing/unsupported output declarations and a non-positive/non-finite `timeoutMs` fail at registration. Disposed with the calling fiber.
- `ctx.tools.register(definition: ToolDefinition): () => void` Register a trusted typed same-process definition with a mandatory canonical `output` declaration. The layer is the calling context's scope: a plain plugin context registers globally; an agent's `agent.ctx` registers for that agent alone, shadowing a same-named global tool there. Duplicate names within one layer throw; non-native modes also reject the reserved `run_code` transport name. Missing or unsupported output declarations and a non-positive or non-finite `timeoutMs` fail at registration. The optional synchronous `finalizeContent` callback is snapshotted when a call starts and may replace only final model-facing content after every pipeline outcome is normalized, including an error discovered while materializing another result field. Disposed with the calling fiber.
- `ctx.tools.restrict(filter)` applies an agent-scoped allow/deny mask to global tools and throws from a plain context. The filter is snapshotted at registration; multiple masks intersect and scope-local tools merge afterwards. Deny masks admit later unnamed globals, while allow masks exclude later names. Unknown, local, or reserved names and empty filters reject. This is live visibility composition, not an authority boundary; see the [scope security non-goal](../../../.agents/notes/implemented/architecture/2026-07-08-agent-scope-contexts.md#security-and-authority-are-explicit-non-goals).
- `ctx.tools.get(name: string, scope?: ScopeKey): ToolDefinition | undefined` Resolution as one scope sees it (shadowing applied; a restricted-away global reads as absent) — presenters pass the calling agent so the card matches what executed.
- `ctx.tools.schemas(scope?: ScopeKey): ToolSchema[]` Schemas of everything the scope can see (without the `execute` functions). The shipped tools' schemas are catalogued in [docs/tool-catalog.md](../../../docs/tool-catalog.md), generated by booting each tool plugin and harvesting this method (see [the tool-schema-catalog Agent Note](../../../.agents/notes/implemented/process/2026-07-02-tool-schema-catalog.md)).
@@ -33,11 +33,11 @@ Cancellation is cooperative and quiescent. Every typed invocation supplies a cal
### Live events
The live registry pipeline has three transformable waterfalls followed by the observe-only `tools/result` boundary; registry changes are deliberately unfiltered shared-state notifications. Exact signatures, dispatch modes, scope filtering, and failure-containment contracts live in the generated [Cordis event catalog](../../../docs/cordis-catalog/events.md), while the complete ordering is visualized in the generated [tool execution pipeline](../../../docs/tool-execution-pipeline.md). `tools/result` is live; the similarly named `tool/result` is the durable session event the agent loop appends afterwards.
The live registry pipeline has three transformable waterfalls, then the definition-owned content finalizer, then the observe-only `tools/result` boundary; registry changes are deliberately unfiltered shared-state notifications. Exact signatures, dispatch modes, scope filtering, and failure-containment contracts live in the generated [Cordis event catalog](../../../docs/cordis-catalog/events.md), while the complete ordering is visualized in the generated [tool execution pipeline](../../../docs/tool-execution-pipeline.md). `tools/result` is live; the similarly named `tool/result` is the durable session event the agent loop appends afterwards.
### Key types
- `ToolDefinition``ToolSchema` + mandatory `output { schema, render, presentationMeta? }` + `execute(args, exec)`, optional presentation callbacks, cooperative `timeoutMs`, and optional per-call `isConcurrencySafe(args)` classification. A body returns only the canonical JSON value declared by the output schema and cooperatively stops through `exec.signal`.
- `ToolDefinition``ToolSchema` + mandatory `output { schema, render, presentationMeta? }` + `execute(args, exec)`, optional final-content and presentation callbacks, cooperative `timeoutMs`, and optional per-call `isConcurrencySafe(args)` classification. A body returns only the canonical JSON value declared by the output schema and cooperatively stops through `exec.signal`. `finalizeContent(exec, result)` runs exactly once for every normalized result, including failures that bypass post-policy, and can replace only `content`; it must be synchronous and total.
- `ToolExecutionInput` — the caller-supplied call description: `{ callId, name, arguments, signal, agent?, parent? }`; `signal` is required and readonly, callers may pass an enclosing execution's opaque token as `parent`, and callers never choose the new execution's own token.
- `ToolExecutionToken` — a fresh branded `Symbol` assigned by the registry. It supports equality correlation only and never crosses a model, log, or worker boundary.
- `ToolExecution` — the readonly pipeline view: immutable `{ token, callId, name, arguments, signal, agent?, parent? }`; the registry separately retains and re-fuses the original caller signal. `ToolDispatchExecution` is the `tools/execute`-only view whose required signal is mutable, so a wrapper may replace and restore it but cannot delete it. A nested call's `parent` is a `ToolExecutionToken`, not an execution object.
@@ -53,7 +53,7 @@ The live registry pipeline has three transformable waterfalls followed by the ob
- Tool plugins call `ctx.tools.register()` — schemas flow into the assembly automatically.
- `tools/pre-execute` is the reorderable allow/deny/ask gate; `ctx.tools.guard()` adds monotonic owner policy after it.
- `tools/execute` wraps normalized canonical dispatch for timeout, retry, or metrics. Wrappers may replace only the operational signal; a wrapper-authored success is normalized through the resolved tool's output declaration. Canonical-result provenance belongs to one immutable dispatch token, so a cached result from another call or tool is revalidated under the active declaration.
- `tools/post-execute` may replace presentation content, replace the canonical value, block with feedback, or attach ordered contexts; `tools/result` observes the immutable final outcome. Content replacement is not a confidentiality boundary: block or replace the value when programmatic consumers must not receive it.
- `tools/post-execute` may replace presentation content, replace the canonical value, block with feedback, or attach ordered contexts. A definition's optional `finalizeContent` then owns its last content-only invariant across normal results and outer pipeline failures; `tools/result` observes the immutable final outcome. Content replacement is not a confidentiality boundary: block or replace the value when programmatic consumers must not receive it.
- Exact signatures and ordering live in the generated [event catalog](../../../docs/cordis-catalog/events.md) and [pipeline](../../../docs/tool-execution-pipeline.md).
- MCP servers: one plugin per server, discover tools, call `ctx.tools.register()` with the server's schemas.

View File

@@ -169,6 +169,18 @@ export interface ToolDefinition extends ToolSchema {
* @returns the canonical value declared by `output.schema`.
*/
execute(args: unknown, exec: ToolRunContext): Promise<unknown>
/**
* Synchronous last-mile transform for model-facing content. The registry
* snapshots this callback when execution starts and invokes it exactly once
* for every normalized outcome, including pipeline failures that bypass
* `tools/post-execute`, immediately before lossless materialization.
* Returning `undefined` preserves the content; every other result field
* remains registry-owned. The callback must be total and must not throw.
* @param exec - immutable execution identity and arguments.
* @param result - complete normalized outcome before materialization.
* @returns replacement content, or `undefined` to preserve it.
*/
finalizeContent?(exec: Readonly<ToolExecution>, result: Readonly<ToolExecutionResult>): ContentBlock[] | undefined
/**
* Cooperative tool-call timeout budget in milliseconds. Omit for no deadline.
* Enforced by `@deepseek-ai/dsh-timeout-policy` (a `tools/execute` wrapper); it
@@ -330,9 +342,9 @@ export interface ToolRegistryScheduler {
prepare(exec: ToolExecutionInput): Promise<ScheduledToolPreparation>
/** Run only the around-dispatch/body stage. */
dispatch(exec: ToolRunContext): Promise<ScheduledToolDispatch>
/** Run ordered post-execute finalization, then materialize and notify the final outcome. */
/** Run post-execute and definition-owned content finalization, then materialize and notify. */
finalize(exec: ToolRunContext, result: ToolExecutionResult): Promise<ToolExecutionResult>
/** Materialize and notify a final outcome that must bypass post-execute. */
/** Run definition-owned content finalization, then materialize and notify without post-execute. */
finish(exec: ToolRunContext, result: ToolExecutionResult): ToolExecutionResult
}
@@ -638,6 +650,8 @@ export class ToolRegistry extends Service {
private deferredContexts = new WeakMap<ToolRunContext, HookContext[]>()
/** Original caller cancellation, kept outside the wrapper-mutable execution object. */
private cancellationStates = new WeakMap<ToolRunContext, ToolCancellationState>()
/** Definition-owned final content transform snapshotted before policy begins. */
private contentFinalizers = new WeakMap<ToolRunContext, ToolDefinition['finalizeContent']>()
private readonly layers = new ScopedLayers(
scope => new ToolLayer(scope),
() => { this.ctx.emit('tools/change') },
@@ -715,7 +729,7 @@ export class ToolRegistry extends Service {
/**
* Register globally or in the calling agent scope. Scoped tools shadow
* globals; duplicates within one layer and the reserved `run_code` name fail.
* @param definition - the tool schema, execution, and optional presentation functions.
* @param definition - tool schema, execution, and optional finalization/presentation callbacks.
* @returns the exact disposer that unregisters the tool.
*/
register(definition: ToolDefinition): () => void {
@@ -910,10 +924,11 @@ export class ToolRegistry extends Service {
}
/**
* Execute through pre-policy, guards, around-dispatch, post-policy, and final
* notification. Tool and listener failures resolve as materialized error
* results; an invisible tool reports `UNKNOWN_TOOL`. The returned outcome is
* the same lossless, frozen snapshot final observers receive. Cancellation
* Execute through pre-policy, guards, around-dispatch, post-policy,
* definition-owned content finalization, and final notification. Tool and
* listener failures resolve as materialized error results; an invisible tool
* reports `UNKNOWN_TOOL`. The returned outcome is the same lossless, frozen
* snapshot final observers receive. Cancellation
* arriving after entry and before final result materialization skips a
* not-yet-started body with `ABORTED_BEFORE_DISPATCH` or replaces a
* successful started outcome with `ABORTED`; already-started work is still
@@ -952,6 +967,8 @@ export class ToolRegistry extends Service {
const agent = exec.agent
const parent = exec.parent
const signal = exec.signal
const definition = this.get(name, agent)
const finalizeContent = definition?.finalizeContent?.bind(definition)
const base = {
token,
callId,
@@ -970,6 +987,7 @@ export class ToolRegistry extends Service {
}
const execution: MutableToolRunContext = { ...base, arguments: deepFreeze(detached) }
this.deferredContexts.set(execution, deferredContexts)
this.contentFinalizers.set(execution, finalizeContent)
this.cancellationStates.set(execution, {
callerSignal: signal,
bodyInvoked: false,
@@ -977,6 +995,7 @@ export class ToolRegistry extends Service {
return { kind: 'ready', exec: execution }
} catch (error: unknown) {
const execution: MutableToolRunContext = { ...base, arguments: undefined }
this.contentFinalizers.set(execution, finalizeContent)
return { kind: 'final-result', exec: execution, result: toolErrorResult(error) }
}
}
@@ -1130,7 +1149,8 @@ export class ToolRegistry extends Service {
}
/**
* Run ordered post-execute, then materialize and notify the final outcome.
* Run ordered post-execute, then apply definition-owned content finalization,
* materialize, and notify the final outcome.
* @param exec - the prepared execution.
* @param result - dispatch/pre result that still needs post-execute.
* @returns the materialized final result.
@@ -1151,16 +1171,23 @@ export class ToolRegistry extends Service {
}
/**
* Materialize and notify a final result that must bypass post-execute.
* Materialize the candidate, apply definition-owned content finalization,
* then materialize and notify the authoritative result.
* @param exec - the prepared execution.
* @param result - final result.
* @returns the materialized final result.
* @internal
*/
private finishScheduledExecution(exec: ToolRunContext, result: ToolExecutionResult): ToolExecutionResult {
let materializedResult: ToolExecutionResult
try {
materializedResult = this.materializeFinalResult(result)
} catch (error: unknown) {
materializedResult = this.materializeFinalResult(toolErrorResult(error))
}
let finalResult: ToolExecutionResult
try {
finalResult = this.materializeFinalResult(result)
finalResult = this.materializeFinalResult(this.applyFinalContent(exec, materializedResult))
} catch (error: unknown) {
finalResult = this.materializeFinalResult(toolErrorResult(error))
}
@@ -1168,6 +1195,14 @@ export class ToolRegistry extends Service {
return finalResult
}
/** Apply the snapshotted tool-owned content transform without exposing other result fields. */
private applyFinalContent(exec: ToolRunContext, result: ToolExecutionResult): ToolExecutionResult {
const finalizeContent = this.contentFinalizers.get(exec)
if (finalizeContent === undefined) return result
const content = finalizeContent(exec, result)
return content === undefined ? result : { ...result, content }
}
/** Notify observers without exposing a mutation or error channel into the outcome. */
private notifyResult(exec: ToolExecution, result: ToolExecutionResult): void {
// Freeze the registry's live object before observers receive its readonly

View File

@@ -3,7 +3,7 @@
import { HarnessError } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { JsonValue } from '@deepseek-ai/dsh-session'
import type { ToolDefinition, ToolRunContext, ToolResult } from './index.ts'
import type { ToolDefinition, ToolExecution, ToolExecutionResult, ToolRunContext, ToolResult } from './index.ts'
import { assertSupportedJsonSchema, isJsonSchemaRecord, isPlainJsonArray, JsonSchemaError, validateJsonSchemaValue } from './json-schema.ts'
import type { JsonSchemaNode, JsonSchemaScalar, ObjectJsonSchema } from './json-schema.ts'
import type { ToolCallView, ToolResultView } from './presentation.ts'
@@ -511,6 +511,15 @@ export interface DefineToolOptions<S extends ParameterSchemaSpec, O extends Valu
* @returns The canonical value declared by `output.schema`.
*/
execute(args: InferArgs<S>, exec: ToolRunContext): Promise<InferValue<NoInfer<O>>>
/**
* Optional last-mile content transform for every normalized outcome. Unlike
* `execute`, arguments remain `unknown` because invalid-input failures also
* reach this callback. See {@link ToolDefinition.finalizeContent}.
* @param exec - immutable execution identity and arguments.
* @param result - complete normalized outcome before materialization.
* @returns replacement content, or `undefined` to preserve it.
*/
finalizeContent?(exec: Readonly<ToolExecution>, result: Readonly<ToolExecutionResult>): ContentBlock[] | undefined
/**
* Pure pending-state presenter.
* @param args - typed validated arguments.
@@ -530,7 +539,7 @@ export interface DefineToolOptions<S extends ParameterSchemaSpec, O extends Valu
* Define a first-party tool with inferred arguments and strict execution
* validation. Replay-only presenters validate softly and fall back to generic
* rendering for obsolete logged arguments.
* @param options - typed definition and optional presenters.
* @param options - typed definition and optional finalizer and presenters.
* @returns A registry-ready definition.
*/
export function defineTool<const S extends ParameterSchemaSpec, const O extends ValueSchemaSpec>(
@@ -540,6 +549,8 @@ export function defineTool<const S extends ParameterSchemaSpec, const O extends
// eslint-disable-next-line @typescript-eslint/unbound-method
const userExecute = options.execute
// eslint-disable-next-line @typescript-eslint/unbound-method
const userFinalizeContent = options.finalizeContent
// eslint-disable-next-line @typescript-eslint/unbound-method
const userRender = options.output.render
// eslint-disable-next-line @typescript-eslint/unbound-method
const userPresentationMeta = options.output.presentationMeta
@@ -577,6 +588,13 @@ export function defineTool<const S extends ParameterSchemaSpec, const O extends
return userExecute(args as InferArgs<S>, exec) as Promise<JsonValue>
},
}
if (userFinalizeContent) {
tool.finalizeContent = (exec, result) => userFinalizeContent(exec, result)
}
// Presentation is display-only and may run on REPLAY of arbitrary logged args
// (possibly from an older schema), so it must never throw: validate softly and
// fall back to `undefined` (a generic UI presentation) on any mismatch, rather
// than the hard `ToolArgsError` the execute path raises.
if (userPresentCall) {
tool.presentCall = (args: unknown): ToolCallView | undefined => {
if (validate(args).length > 0) return undefined

View File

@@ -51,22 +51,22 @@ describe('ToolRegistry', () => {
expect(assembly.tools.map(t => t.name)).toEqual(['echo'])
})
it('schemas() drops the UI presentation callbacks — they must never reach the model', async () => {
it('schemas() drops host callbacks — they must never reach the model', async () => {
const ctx = await setup()
// A tool that declares presentCall/presentResult (functions). schemas() feeds
// the system-prompt assembly → the model request, so those callbacks (and
// `execute`) must be stripped: a function in the JSON tool schema would
// corrupt the request. schemas() is an explicit allowlist, so it can't leak.
// Tool definitions contain output, finalization, execution, and presentation
// callbacks. schemas() is an explicit allowlist so none can reach the model.
ctx.tools.register(defineContentToolFixture({
name: 'present',
description: 'has presenters',
parameters: { x: { type: 'string', required: true } },
async execute() { return [] },
finalizeContent: (_exec, result) => result.content,
presentCall: args => ({ card: 'generic', title: args.x }),
presentResult: (args, result) => ({ card: 'generic', title: args.x, content: result.content }),
}))
const schema = ctx.tools.schemas()[0] as unknown as Record<string, unknown>
expect(Object.keys(schema).sort()).toEqual(['description', 'name', 'parameters'])
expect(schema.finalizeContent).toBeUndefined()
expect(schema.presentCall).toBeUndefined()
expect(schema.presentResult).toBeUndefined()
expect(schema.execute).toBeUndefined()
@@ -155,6 +155,72 @@ describe('ToolRegistry', () => {
expect(observedError).toBe(true)
})
it('finalizes errors discovered while snapshotting non-content result fields', async () => {
const ctx = await setup()
let finalizeCalls = 0
ctx.tools.register({
...echoTool,
name: 'throwing-meta',
output: {
...echoTool.output,
presentationMeta() {
const meta = {}
Object.defineProperty(meta, 'value', {
enumerable: true,
get() { throw new Error('snapshot failed: '.repeat(100)) },
})
return meta
},
},
finalizeContent(_exec, result) {
finalizeCalls += 1
const block = result.content[0]
if (block?.type !== 'text') return undefined
return [{ type: 'text', text: block.text.slice(0, 32) }]
},
async execute() {
return 'body'
},
})
const result = await ctx.tools.execute({
signal: testToolSignal,
callId: CallId('throwing-meta'), name: 'throwing-meta', arguments: {},
})
expect(result.isError).toBe(true)
const block = result.content[0]
expect(block?.type).toBe('text')
expect(block?.type === 'text' ? block.text : '').toMatch(/^Error: tool "throwing-meta"/)
expect(block?.type === 'text' ? block.text : '').toHaveLength(32)
expect(finalizeCalls).toBe(1)
})
it('normalizes a throwing final content callback without invoking it again', async () => {
const ctx = await setup()
let finalizeCalls = 0
ctx.tools.register({
...echoTool,
name: 'throwing-finalizer',
finalizeContent() {
finalizeCalls += 1
throw new Error('finalizer violated its total contract')
},
})
const result = await ctx.tools.execute({
signal: testToolSignal,
callId: CallId('throwing-finalizer'), name: 'throwing-finalizer', arguments: {},
})
expect(result).toEqual({
content: [{ type: 'text', text: 'Error: finalizer violated its total contract' }],
isError: true,
error: { message: 'finalizer violated its total contract' },
})
expect(finalizeCalls).toBe(1)
})
it('requires every raw registration to declare its canonical output', async () => {
const ctx = await setup()
const missingOutput = {
@@ -747,6 +813,36 @@ describe('ToolRegistry', () => {
expect(result.content[0]).toMatchObject({ text: 'output rejected: try again' })
})
it('runs the snapshotted final content transform after outer pipeline normalization', async () => {
const ctx = await setup()
const dispose = ctx.tools.register(defineContentToolFixture({
name: 'bounded',
description: 'bounded result',
parameters: {},
async execute() { return [{ type: 'text', text: 'body' }] },
finalizeContent(exec, result) {
expect(exec.name).toBe('bounded')
expect(result.isError).toBe(true)
return [{ type: 'text', text: 'bounded failure' }]
},
}))
ctx.on('tools/pre-execute', async () => {
dispose()
throw new HarnessError('policy failed', 'POLICY_FAILED')
})
const result = await ctx.tools.execute({ signal: testToolSignal, callId: CallId('bounded'), name: 'bounded', arguments: {} })
expect(result).toEqual({
content: [{ type: 'text', text: 'bounded failure' }],
isError: true,
error: {
message: 'policy failed',
info: { name: 'HarnessError', code: 'POLICY_FAILED' },
},
})
})
it('a block decision can ALSO attach additionalContexts', async () => {
const ctx = await setup()
ctx.tools.register(echoTool)