Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: return typed values from Code Mode

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-21 04:34:14 +08:00
parent 66c36e7325
commit c1d7b0df81
50 changed files with 2155 additions and 580 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
packages/core/tools/README.md
View File

@@ -108,11 +108,12 @@ Returning `undefined` selects generic fallback. Presenters depend only on their
### Code Mode
Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic TypeScript SDK for the current scope; only program output re-enters model context. Each JSON-normalized binding re-enters the complete tool pipeline sequentially with logged correlation to the outer call. Denials reject that binding, ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; failures surface as `CodeRunFailedError`. See the [Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md) and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`.
Under `code` or `both`, the registry exposes the reserved `run_code` transport and a deterministic TypeScript SDK for the current scope; only the program's outer logs and return value re-enter model context. The SDK declares exact `ToolArgsMap` and `ToolOutputMap` entries for every visible tool, and each binding resolves to the tool's canonical JSON value. Each lossless-JSON binding call re-enters the complete tool pipeline sequentially with logged correlation to the outer call. Denials and other failed results reject with the real program-visible `ToolCallError` carrying only `toolName` and `message`; Native content and internal error codes stay outside the Code contract. Ordinary side effects are not rolled back, and sub-call `additionalContexts` are deferred through the parent result to preserve call/result adjacency. Run settlement aborts and drains outstanding bindings; runtime failures surface as `CodeRunFailedError`. See the [Code Mode foundation](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md), [typed-return contract](../../../.agents/notes/implemented/feature/2026-07-20-code-mode-typed-tool-returns.md), and [code-runtime seam](../../code-runtime/README.md). Try `pnpm run demo:code-mode`.
- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, a `declare const tools: {...}` TypeScript declaration of the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) is total: constructs outside the `defineTool` subset degrade to `unknown`, never throw.
- **The dispatch bridge** (`run_code`'s execute): every binding call is JSON-normalized before dispatch (a value that does not survive — `BigInt`, circulars — rejects that one call, so the dispatched form and logged form are the same JSON value by construction), serialized through a per-run queue (even `Promise.all` executes underlying calls one at a time in submission order), given the outer execution's opaque token as `parent`, and run through the complete pre-execute → guards → execute → post-execute → result pipeline. A denial reaches the program as a binding rejection, and each sub-call is logged as a `tool/code-dispatch` session event with deterministic id `<parent>:code:<n>`; `deriveMessages()` does not surface that event. Token correlation lets commit-style observers defer an inner success until the final `run_code` result without exposing the live outer execution; ordinary tool side effects are not rolled back. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/meta even when the program later fails.
- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, `JsonValue`, exact `ToolArgsMap` / `ToolOutputMap`, `ToolName`, the `ToolCallError` declaration, and a mapped `tools` namespace for the calling scope's visible end capabilities (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) handles every unified schema construct and degrades unsupported raw constructs to `unknown`, never throwing during prompt assembly.
- **The dispatch bridge** (`run_code`'s execute): every binding call is snapshotted as lossless JSON before dispatch (`undefined`, `BigInt`, cycles, sparse arrays, `-0`, and exotic objects reject that one call), serialized through a per-run queue (even `Promise.all` executes underlying calls one at a time in submission order), given the outer execution's opaque token as `parent`, and run through the complete pre-execute → guards → execute → post-execute → result pipeline. A success returns the final canonical value after policy; a failure reaches the worker as one message and becomes `ToolCallError(toolName, message)`. Each sub-call is logged as a `tool/code-dispatch` session event with deterministic id `<parent>:code:<n>` and a bounded Native-content summary; `deriveMessages()` does not surface that event or persist the value. Token correlation lets commit-style observers defer an inner success until the final `run_code` result without exposing the live outer execution; ordinary tool side effects are not rolled back. Every sub-call `additionalContexts` entry is deferred through the outer `ToolRunContext` in dispatch order; the loop appends those contexts only after the parent `run_code` result, preserving adjacency and retaining each source/meta even when the program later fails.
- **Settlement discipline**: the bridge owns a run-scoped abort that follows the outer signal in and fires when the run settles for any reason, so a budget expiry aborts an in-flight sub-tool instead of orphaning it; the bridge then drains its queue BEFORE returning, so every `tool/code-dispatch` lands inside the open turn. A failed run throws `CodeRunFailedError` (`code: 'CODE_RUN_FAILED'`, message = the failure kind + captured logs), which the pipeline converts to a structured `isError` the model self-corrects from.
- **Result boundary**: intermediate binding values cross the worker boundary whole and have no per-binding byte cap. `run_code` returns canonical `{ logs: string[], result?: JsonValue }`; strings render raw, every other present JSON root renders as pretty JSON, `null` remains explicit, and absent `result` means the program returned `undefined`. The worker's configurable `maxOutputBytes` (default 64 MiB) applies only to the combined serialized outer logs, completion, or failure diagnostic; invalid and over-limit completions fail explicitly, and only this outer result is eligible for ordinary spill.
### Parallel execution
@@ -147,8 +148,8 @@ Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.m
Pass `run_code` the body of an async TypeScript function (erasable syntax only — no `enum` or namespaces; type annotations are advisory, the code runs type-stripped). Inside the program:
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to the tool's text output as a string. Tool arguments must be JSON-serializable.
- A FAILED tool call rejects with an `Error` carrying the tool's error text — `try/catch` it to handle and continue.
- Call tools as `await tools.name(args)` — quoted access for exotic names: `tools["my-tool"](args)`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
- A FAILED tool call rejects with `ToolCallError`, whose `toolName` identifies the failed tool and whose `message` is human-readable — `try/catch` it to handle and continue.
- Calls execute sequentially, even under `Promise.all`.
- Emit results with `return` and/or `console.log(...)`. ONLY what you print or return comes back to you — intermediate tool results never enter the conversation, so extract just what you need.
@@ -181,8 +182,8 @@ Append-only; newly visible content follows the reusable request prefix and does
- **Concurrency policy is not an event seam** — `executionMode()` reads the resolved tool definition directly; plugins can only declare a classifier on definitions they own.
- **`tools/pre-execute` deliberately cannot rewrite `exec.arguments`** — logged and rendered args would desync from what ran; the rewrite design is [a proposed Agent Note](../../../.agents/notes/proposed/feature/2026-06-30-pre-tool-input-rewrite.md).
- **`defineTool`'s schema DSL is a deliberate subset** — string/number/boolean/object/array with string-only `enum`; `validateArgs` tolerates extra keys and preserves `default` as a model-visible JSON Schema annotation without applying it during validation; dynamic Cordis mounts may supply defaults even though first-party definitions do not, while raw-registered JSON-Schema tools validate their own input.
- **Caller-defined subagent and workflow structured outputs remain object-rooted** — this is a consumer-level guard; the shared schema vocabulary and tool outputs support every JSON root.
- **`timeoutMs` on a definition is declarative only** — the registry never enforces deadlines; enforcement requires the `@deepseek-ai/dsh-timeout-policy` wrapper.
- **Code Mode is TypeScript-only and the presentation mode is service-wide** — `mode: code`/`both` rejects prompt assembly unless `ctx.codeRuntime.language === 'typescript'`; scoped restrictions/shadows still choose each agent's visible bindings, but one tool cannot be native-only while another is code-only.
- **Code Mode bindings return text only** — non-text content blocks in a sub-call result collapse to `[<type> content]` placeholders.
- **Code Mode intermediate values are execution-local and unbounded by bytes** — they cannot be reconstructed from session replay and may exhaust process or worker memory; only the outer `run_code` output has the worker's configurable hard cap.
- **`run_code` state is fresh per run** — a persistent REPL-style kernel is rejected for the MVP (cross-call state would be invisible to the log); see [the Code Mode Agent Note](../../../.agents/notes/implemented/feature/2026-06-15-code-mode.md).

56
packages/core/tools/src/code-mode.ts
View File

@@ -6,10 +6,10 @@
*/
import { parse } from 'node:path'
import { inspect } from 'node:util'
import { CallId, HarnessError } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { CodeBindingFunction, CodeRunResult, CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
import { snapshotJsonValue } from '@deepseek-ai/dsh-session'
import type { JsonValue } from '@deepseek-ai/dsh-session'
import { defineTool } from './schema.ts'
import type { ToolDefinition, ToolRegistry } from './index.ts'
@@ -62,10 +62,7 @@ export class CodeRunFailedError extends HarnessError {
*/
const SUMMARY_MAX_CHARS = 200
/** Bounded inspect for rendering a program's completion value into the model-facing text. */
const INSPECT_OPTIONS = { depth: 4, maxArrayLength: 100, maxStringLength: 10_000 } as const
/** Join a result's text blocks; a non-text block becomes a placeholder (an MVP limitation, stated in the SDK instructions). */
/** Join Native content for the bounded durable sub-dispatch summary; non-text blocks become diagnostic placeholders. */
function textOf(content: ContentBlock[]): string {
return content
.map((block) => {
@@ -88,32 +85,26 @@ function summarize(text: string, cwd: string | undefined): string {
}
/**
* JSON-normalize one binding call's argument into TWO independent parses of the same canonical
* text: `dispatched` goes to the tool, `logged` to the `tool/code-dispatch` event — identical
* by construction (the runtime's structured-clone boundary is wider than JSON; the session log
* accepts only JSON), and separate objects, so a tool mutating its args can neither desync the
* log from what was dispatched nor re-poison the append.
* Snapshot one binding call's argument as lossless JSON, then clone it into
* independent dispatch/log values so a tool mutation cannot desynchronize the
* durable event from what was called.
*/
function jsonNormalizeArgs(value: unknown): { dispatched: unknown; logged: unknown } {
if (value === undefined) {
throw new Error('tool arguments must be JSON-serializable (call the tool with an arguments object, e.g. `{}`)')
}
let text: string | undefined
let snapshot: JsonValue | undefined
try {
text = JSON.stringify(value)
snapshot = snapshotJsonValue(value) as JsonValue | undefined
} catch (error: unknown) {
throw new Error(`tool arguments must be JSON-serializable: ${error instanceof Error ? error.message : String(error)}`)
throw new Error(`tool arguments must be lossless JSON: ${error instanceof Error ? error.message : String(error)}`)
}
// JSON.stringify's lib type claims `string`, but a bare function or symbol
// root really yields `undefined` at runtime — the guard is live.
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
if (text === undefined) throw new Error('tool arguments must be JSON-serializable (got a value JSON cannot represent)')
return { dispatched: JSON.parse(text) as unknown, logged: JSON.parse(text) as unknown }
if (snapshot === undefined) {
throw new Error('tool arguments must be lossless JSON (call the tool with an arguments object, e.g. `{}`)')
}
return { dispatched: structuredClone(snapshot), logged: structuredClone(snapshot) }
}
/** Render one present program completion value for the model-facing result text. */
function renderValue(value: JsonValue): string {
return typeof value === 'string' ? value : inspect(value, INSPECT_OPTIONS)
return typeof value === 'string' ? value : JSON.stringify(value, null, 2)
}
/** The run_code result's `meta` payload (JSON-serializable; `presentResult` narrows it back). */
@@ -203,7 +194,7 @@ export function createRunCodeTool(registry: ToolRegistry, requireRuntime: () =>
// would be narrowed away by control flow analysis.
const runOver = (): boolean => runController.signal.aborted
const binding = (name: string): CodeBindingFunction => async (rawArgs: unknown): Promise<unknown> => {
const binding = (name: string): CodeBindingFunction => async (rawArgs: unknown): Promise<JsonValue> => {
if (runOver()) {
throw new Error(`run_code run is over (${String(runController.signal.reason)}); ${name} not dispatched`)
}
@@ -234,7 +225,9 @@ export function createRunCodeTool(registry: ToolRegistry, requireRuntime: () =>
isError: result.isError,
resultSummary: summarize(text, exec.agent.session.header.cwd),
})
return { text, isError: result.isError }
return result.isError
? { isError: true as const, message: result.error.message }
: { isError: false as const, value: result.value }
})
// A budget expiry or outer cancel that lands while this call was in
// flight already aborted the dispatch; stop the program now rather
@@ -242,11 +235,11 @@ export function createRunCodeTool(registry: ToolRegistry, requireRuntime: () =>
if (runOver()) {
throw new Error(`run_code run is over (${String(runController.signal.reason)}); ${name} result discarded`)
}
// A failed tool call REJECTS — real code signals failure by throwing,
// so try/catch and Promise.all short-circuiting behave as models
// expect (the error text is the tool's model-facing result text).
if (outcome.isError) throw new Error(outcome.text)
return outcome.text
// The worker turns a binding rejection into ToolCallError and adds
// only the binding name. Native content and internal error metadata
// stay outside the program-facing failure contract.
if (outcome.isError) throw new Error(outcome.message)
return outcome.value
}
// Null-prototype + defineProperty, mirroring the worker-side namespace
@@ -283,12 +276,9 @@ export function createRunCodeTool(registry: ToolRegistry, requireRuntime: () =>
const logsText = result.logs.length > 0 ? `\nCaptured output:\n${result.logs.join('\n')}` : ''
throw new CodeRunFailedError(`code run failed (${result.error.kind}): ${result.error.message}${logsText}`)
}
// The runtime seam is wider than JSON until PR 3 makes this boundary
// lossless. The registry immediately snapshots and rejects any value
// that does not satisfy the declared JSON output.
return {
logs: result.logs,
...result.value !== undefined ? { result: result.value as JsonValue } : {},
...result.value !== undefined ? { result: result.value } : {},
}
} finally {
exec.signal?.removeEventListener('abort', onOuterAbort)

13
packages/core/tools/src/index.ts
View File

@@ -23,6 +23,7 @@ import { assertSupportedJsonSchema, validateJsonSchemaValue } from './json-schem
import type { JsonSchemaNode } from './json-schema.ts'
import { createRunCodeTool, RUN_CODE_NAME, SDK_SECTION_ORDER } from './code-mode.ts'
import { renderToolsSdk } from './ts-types.ts'
import type { ToolSdkSchema } from './ts-types.ts'
export {
defineTool,
@@ -550,7 +551,7 @@ export class ToolRegistry extends Service {
// Regenerate from the calling scope's visible tools in stable order.
text: (context) => {
this.requireCodeRuntime()
return renderToolsSdk(this.schemas(context.scope).filter(schema => schema.name !== RUN_CODE_NAME))
return renderToolsSdk(this.sdkSchemas(context.scope))
},
})
}
@@ -815,6 +816,16 @@ export class ToolRegistry extends Service {
return [...this.view(scope).visible.values()].map(definition => this.schemaOf(definition, true))
}
/** Project visible callable tools onto the generated Code Mode SDK contract. */
private sdkSchemas(scope?: ScopeKey): ToolSdkSchema[] {
return [...this.view(scope).visible.values()]
.filter(definition => definition.name !== RUN_CODE_NAME)
.map((definition): ToolSdkSchema => ({
...this.schemaOf(definition, true),
output: structuredClone(definition.output.schema),
}))
}
/** Project one definition onto the model-facing schema fields. */
private schemaOf(definition: ToolDefinition, detachParameters: boolean): ToolSchema {
const { name, description, parameters } = definition

34
packages/core/tools/src/ts-types.ts
View File

@@ -8,7 +8,13 @@
import type { ToolSchema } from '@deepseek-ai/dsh-llm'
import { assertSupportedJsonSchema } from './json-schema.ts'
import type { JsonSchemaScalar } from './json-schema.ts'
import type { JsonSchemaNode, JsonSchemaScalar } from './json-schema.ts'
/** Internal Code Mode projection: the model-facing schema plus the canonical output schema. */
export interface ToolSdkSchema extends ToolSchema {
/** Validated canonical value returned by the tool binding. */
output: JsonSchemaNode
}
/** Property names that are valid bare TS identifiers; anything else is quoted. */
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/
@@ -107,8 +113,8 @@ const SDK_INSTRUCTIONS = `## Writing code for run_code
Pass \`run_code\` the body of an async TypeScript function (erasable syntax only — no \`enum\` or namespaces; type annotations are advisory, the code runs type-stripped). Inside the program:
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to the tool's text output as a string. Tool arguments must be JSON-serializable.
- A FAILED tool call rejects with an \`Error\` carrying the tool's error text — \`try/catch\` it to handle and continue.
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
- A FAILED tool call rejects with \`ToolCallError\`, whose \`toolName\` identifies the failed tool and whose \`message\` is human-readable — \`try/catch\` it to handle and continue.
- Calls execute sequentially, even under \`Promise.all\`.
- Emit results with \`return\` and/or \`console.log(...)\`. ONLY what you print or return comes back to you — intermediate tool results never enter the conversation, so extract just what you need.
@@ -123,16 +129,24 @@ The available tools:`
* `run_code` itself).
* @returns the complete section text.
*/
export function renderToolsSdk(schemas: ToolSchema[]): string {
export function renderToolsSdk(schemas: ToolSdkSchema[]): string {
const sorted = [...schemas].sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0)
const members: string[] = []
const argsMembers: string[] = []
const outputMembers: string[] = []
for (const schema of sorted) {
members.push(...docLines(schema.description, 1))
members.push(`${pad(1)}${renderKey(schema.name)}(args: ${jsonSchemaToTs(schema.parameters, 1)}): Promise<string>;`)
argsMembers.push(...docLines(schema.description, 1))
argsMembers.push(`${pad(1)}${renderKey(schema.name)}: ${jsonSchemaToTs(schema.parameters, 1)};`)
outputMembers.push(`${pad(1)}${renderKey(schema.name)}: ${jsonSchemaToTs(schema.output, 1)};`)
}
const declaration = members.length > 0
? `declare const tools: {\n${members.join('\n')}\n}`
: 'declare const tools: {}'
const argsMap = `interface ToolArgsMap {${argsMembers.length > 0 ? `\n${argsMembers.join('\n')}\n` : ''}}`
const outputMap = `interface ToolOutputMap {${outputMembers.length > 0 ? `\n${outputMembers.join('\n')}\n` : ''}}`
const declaration = [
argsMap,
outputMap,
'type ToolName = keyof ToolOutputMap',
['declare class ToolCallError extends Error {', ' readonly name: "ToolCallError";', ' readonly toolName: ToolName;', '}'].join('\n'),
['declare const tools: {', ' [K in ToolName]: (args: ToolArgsMap[K]) => Promise<ToolOutputMap[K]>;', '}'].join('\n'),
].join('\n\n')
const jsonValue = 'type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue }'
return `${SDK_INSTRUCTIONS}\n\n\`\`\`ts\n${jsonValue}\n\n${declaration}\n\`\`\``
}

114
packages/core/tools/tests/code-mode.spec.ts
View File

@@ -6,7 +6,7 @@ import type { Scope } from '@deepseek-ai/dsh-scope'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
import ToolRegistry, { CodeRunFailedError, RUN_CODE_NAME, defineContentToolFixture } from '@deepseek-ai/dsh-tools'
import ToolRegistry, { CodeRunFailedError, RUN_CODE_NAME, defineContentToolFixture, defineTool } from '@deepseek-ai/dsh-tools'
import type { Config, PostToolDecision, ToolExecutionResult } from '@deepseek-ai/dsh-tools'
import type { Agent } from '@deepseek-ai/dsh-agent'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
@@ -68,13 +68,17 @@ async function mintAgentScope(ctx: Context, name = 'scoped'): Promise<{ scope: S
/** Register a trivial echo tool; returns the calls it received. */
function registerEcho(ctx: Context, name = 'echo'): unknown[] {
const calls: unknown[] = []
ctx.tools.register(defineContentToolFixture({
ctx.tools.register(defineTool({
name,
description: `Echo tool ${name}.`,
parameters: { value: { type: 'string', required: true } },
output: {
schema: { type: 'string' },
render: (_args, value) => [{ type: 'text', text: value }],
},
execute(args) {
calls.push(args)
return Promise.resolve([{ type: 'text' as const, text: `${name}:${args.value}` }])
return Promise.resolve(`${name}:${args.value}`)
},
}))
return calls
@@ -119,8 +123,8 @@ describe('mode-aware wire contribution', () => {
expect(assembly.tools.map(tool => tool.name)).toEqual([RUN_CODE_NAME])
const sdk = assembly.sections.find(section => section.name === 'tools:sdk')
expect(sdk?.text).toContain('declare const tools: {')
expect(sdk?.text).toContain('echo(args:')
expect(sdk?.text).not.toContain('run_code(args:')
expect(sdk?.text).toContain('echo: {')
expect(sdk?.text).not.toContain('run_code:')
})
it.each(['code', 'both'] as const)('treats expert assembly output as authoritative in mode %s', async (mode) => {
@@ -172,8 +176,8 @@ describe('mode-aware wire contribution', () => {
? [RUN_CODE_NAME]
: ['echo', RUN_CODE_NAME])
const sdk = assembly.sections.find(section => section.name === 'tools:sdk')?.text
expect(sdk).toContain('echo(args:')
expect(sdk).not.toContain('hidden(args:')
expect(sdk).toContain('echo: {')
expect(sdk).not.toContain('hidden:')
runtime.behavior = request => Promise.resolve({
logs: [],
@@ -202,8 +206,8 @@ describe('mode-aware wire contribution', () => {
? [RUN_CODE_NAME]
: ['kept', RUN_CODE_NAME])
const sdk = assembly.sections.find(section => section.name === 'tools:sdk')?.text
expect(sdk).not.toContain('denied(args:')
expect(sdk).toContain('kept(args:')
expect(sdk).not.toContain('denied:')
expect(sdk).toContain('kept: {')
runtime.behavior = request => Promise.resolve({
logs: [],
@@ -241,7 +245,7 @@ describe('mode-aware wire contribution', () => {
expect(transports).toHaveLength(1)
expect(transports[0]?.description).toContain('Execute a TypeScript program')
expect(assembly.sections.find(section => section.name === 'scoped-note')?.text).toBe('safe note')
expect(assembly.sections.find(section => section.name === 'tools:sdk')?.text).toContain('scoped_safe(args:')
expect(assembly.sections.find(section => section.name === 'tools:sdk')?.text).toContain('scoped_safe:')
expect(ctx.tools.get(RUN_CODE_NAME, agent)).toBe(ctx.tools.get(RUN_CODE_NAME))
const result = await runCode(ctx, 'return 1', { agent })
expect(result.content).toEqual([{ type: 'text', text: '(run_code completed with no output)' }])
@@ -328,7 +332,8 @@ describe('the run_code dispatch bridge', () => {
const tools = request.bindings[0]!.functions
const first = await tools.echo!({ value: 'one' })
const second = await tools.echo!({ value: 'two' })
return { logs: [`saw ${String(first)}`], value: second }
if (typeof first !== 'string' || typeof second !== 'string') throw new Error('echo returned a non-string')
return { logs: [`saw ${first}`], value: second }
}
const result = await runCode(ctx, 'const …: string = …', { agent })
expect(result.isError).toBe(false)
@@ -376,10 +381,14 @@ describe('the run_code dispatch bridge', () => {
const { ctx, runtime } = await setup({ mode: 'code' })
const intervals: [string, string][] = []
let active = 0
ctx.tools.register(defineContentToolFixture({
ctx.tools.register(defineTool({
name: 'probe',
description: 'Records execution overlap.',
parameters: { id: { type: 'string', required: true } },
output: {
schema: { type: 'string' },
render: (_args, value) => [{ type: 'text', text: value }],
},
async execute(args) {
active++
expect(active, 'probe executions overlapped').toBe(1)
@@ -387,12 +396,13 @@ describe('the run_code dispatch bridge', () => {
await new Promise(resolve => setTimeout(resolve, 20))
intervals.push(['exit', args.id])
active--
return [{ type: 'text' as const, text: args.id }]
return args.id
},
}))
runtime.behavior = async (request) => {
const tools = request.bindings[0]!.functions
const values = await Promise.all([tools.probe!({ id: 'a' }), tools.probe!({ id: 'b' }), tools.probe!({ id: 'c' })])
if (!values.every(value => typeof value === 'string')) throw new Error('probe returned a non-string')
return { logs: [], value: values.join(',') }
}
const result = await runCode(ctx, 'program')
@@ -422,7 +432,7 @@ describe('the run_code dispatch bridge', () => {
}
}
const result = await runCode(ctx, 'program')
expect(result.content[0]).toEqual({ type: 'text', text: 'caught: Error: deliberate failure' })
expect(result.content[0]).toEqual({ type: 'text', text: 'caught: deliberate failure' })
})
it('a tools/pre-execute deny reaches the program as a binding rejection', async () => {
@@ -445,7 +455,7 @@ describe('the run_code dispatch bridge', () => {
expect((result.content[0] as { text: string }).text).toContain('not on my watch')
})
it('rejects a binding argument that does not survive JSON normalization, dispatching nothing', async () => {
it('rejects a binding argument that is not lossless JSON, dispatching nothing', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
const calls = registerEcho(ctx)
const { agent, events } = fakeAgent()
@@ -458,25 +468,24 @@ describe('the run_code dispatch bridge', () => {
}
}
const result = await runCode(ctx, 'program', { agent })
expect((result.content[0] as { text: string }).text).toContain('JSON-serializable')
expect((result.content[0] as { text: string }).text).toContain('lossless JSON')
expect(calls).toEqual([])
expect(events.filter(event => event.type === 'tool/code-dispatch')).toEqual([])
})
it('dispatches the JSON-normalized value: what the tool sees is what the event logs', async () => {
it('dispatches and logs independent snapshots of the same lossless JSON value', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
const calls = registerEcho(ctx)
const { agent, events } = fakeAgent()
runtime.behavior = async (request) => {
// A Date survives structured clone but is not JSON; the bridge
// normalizes it to its JSON form (an ISO string) BEFORE dispatch.
await request.bindings[0]!.functions.echo!({ value: 'x', when: new Date(0) }).catch(() => undefined)
const args = Object.assign(Object.create(null) as Record<string, unknown>, { value: 'x', nested: ['same'] })
await request.bindings[0]!.functions.echo!(args)
return { logs: [] }
}
await runCode(ctx, 'program', { agent })
expect(calls).toEqual([{ value: 'x', when: '1970-01-01T00:00:00.000Z' }])
expect(calls).toEqual([{ value: 'x', nested: ['same'] }])
const dispatch = events.find(event => event.type === 'tool/code-dispatch')?.data as SessionEventMap['tool/code-dispatch']
expect(dispatch.arguments).toEqual({ value: 'x', when: '1970-01-01T00:00:00.000Z' })
expect(dispatch.arguments).toEqual({ value: 'x', nested: ['same'] })
})
it('defers sub-call additionalContexts onto the outer run_code result', async () => {
@@ -691,15 +700,19 @@ describe('the run_code dispatch bridge', () => {
const { ctx, runtime } = await setup({ mode: 'code' })
const { agent, events } = fakeAgent()
const long = 'x'.repeat(300)
ctx.tools.register(defineContentToolFixture({
ctx.tools.register(defineTool({
name: 'mixed',
description: 'Returns mixed content.',
parameters: {},
output: {
schema: { type: 'string' },
render: () => [
{ type: 'text', text: long },
{ type: 'reasoning', text: 'hidden' },
],
},
execute() {
return Promise.resolve([
{ type: 'text' as const, text: long },
{ type: 'reasoning' as const, text: 'hidden' },
])
return Promise.resolve('mixed-value')
},
}))
runtime.behavior = async (request) => {
@@ -708,7 +721,7 @@ describe('the run_code dispatch bridge', () => {
}
const result = await runCode(ctx, 'program', { agent })
expect(result.isError).toBe(false)
expect((result.content[0] as { text: string }).text).toBe(`${long}\n[reasoning content]`)
expect((result.content[0] as { text: string }).text).toBe('mixed-value')
const dispatch = events.find(event => event.type === 'tool/code-dispatch')?.data as SessionEventMap['tool/code-dispatch']
expect(dispatch.resultSummary.length).toBe(201)
expect(dispatch.resultSummary.endsWith('…')).toBe(true)
@@ -716,13 +729,17 @@ describe('the run_code dispatch bridge', () => {
it('normalizes the session workspace root before bounding durable result summaries', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
ctx.tools.register(defineContentToolFixture({
ctx.tools.register(defineTool({
name: 'workspace_path',
description: 'Return a path beneath the session workspace.',
parameters: {},
output: {
schema: { type: 'string' },
render: (_args, value) => [{ type: 'text', text: value }],
},
execute(_args, exec) {
const cwd = exec.agent?.session.header.cwd ?? ''
return Promise.resolve([{ type: 'text' as const, text: `<path>${cwd}/nested/task.txt</path>\n${'x'.repeat(240)}` }])
return Promise.resolve(`<path>${cwd}/nested/task.txt</path>\n${'x'.repeat(240)}`)
},
}))
runtime.behavior = async request => ({
@@ -760,7 +777,7 @@ describe('the run_code dispatch bridge', () => {
expect((root.events[0]!.data as SessionEventMap['tool/code-dispatch']).resultSummary).toBe('echo:/workspace/value')
})
it('rejects undefined, JSON-throwing, and JSON-unrepresentable binding arguments BEFORE dispatch', async () => {
it('rejects undefined, getter-throwing, exotic, and unrepresentable binding arguments before dispatch', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
const calls = registerEcho(ctx)
const { agent, events } = fakeAgent()
@@ -773,8 +790,9 @@ describe('the run_code dispatch bridge', () => {
// Root undefined must reject up front: the event log rejects it as
// data, and nothing may execute unlogged.
await catchMessage(echo(undefined)),
// A toJSON that throws a NON-Error propagates out of JSON.stringify.
await catchMessage(echo({ toJSON() { throw 'raw-throw' } })),
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw 'raw-throw' } }))),
await catchMessage(echo(Object.defineProperty({}, 'bad', { enumerable: true, get() { throw new Error('error-throw') } }))),
await catchMessage(echo(new Date(0))),
// A bare function is a value JSON cannot represent at all.
await catchMessage(echo(() => 1)),
].join(' | '),
@@ -783,9 +801,10 @@ describe('the run_code dispatch bridge', () => {
const result = await runCode(ctx, 'program', { agent })
const text = (result.content[0] as { text: string }).text
expect(text).toContain('call the tool with an arguments object')
expect(text).toContain('JSON-serializable: raw-throw')
expect(text).toContain('a value JSON cannot represent')
// None of the three dispatched, none logged.
expect(text).toContain('lossless JSON: raw-throw')
expect(text).toContain('lossless JSON: error-throw')
expect(text.match(/tool arguments must be lossless JSON/g)).toHaveLength(5)
// None dispatched or logged.
expect(calls).toEqual([])
expect(events.filter(event => event.type === 'tool/code-dispatch')).toEqual([])
})
@@ -816,11 +835,15 @@ describe('the run_code dispatch bridge', () => {
it('exposes a tool named __proto__ as an ordinary own binding', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
ctx.tools.register(defineContentToolFixture({
ctx.tools.register(defineTool({
name: '__proto__',
description: 'A prototype-colliding tool name.',
parameters: {},
execute() { return Promise.resolve([{ type: 'text' as const, text: 'proto-tool-ok' }]) },
output: {
schema: { type: 'string' },
render: (_args, value) => [{ type: 'text', text: value }],
},
execute() { return Promise.resolve('proto-tool-ok') },
}))
runtime.behavior = async (request) => {
const functions = request.bindings[0]!.functions
@@ -833,11 +856,20 @@ describe('the run_code dispatch bridge', () => {
expect(result.content[0]).toEqual({ type: 'text', text: 'proto-tool-ok' })
})
it('renders a non-string completion value inspect-style', async () => {
it('renders every non-string JSON root as pretty JSON while preserving strings raw', async () => {
const { ctx, runtime } = await setup({ mode: 'code' })
runtime.behavior = () => Promise.resolve({ logs: [], value: { n: 42 } })
const result = await runCode(ctx, 'program')
expect((result.content[0] as { text: string }).text).toBe('{ n: 42 }')
expect((await runCode(ctx, 'object')).content[0]).toEqual({ type: 'text', text: '{\n "n": 42\n}' })
runtime.behavior = () => Promise.resolve({ logs: [], value: ['x', 2] })
expect((await runCode(ctx, 'array')).content[0]).toEqual({ type: 'text', text: '[\n "x",\n 2\n]' })
runtime.behavior = () => Promise.resolve({ logs: [], value: null })
expect((await runCode(ctx, 'null')).content[0]).toEqual({ type: 'text', text: 'null' })
runtime.behavior = () => Promise.resolve({ logs: [], value: 'raw' })
expect((await runCode(ctx, 'string')).content[0]).toEqual({ type: 'text', text: 'raw' })
runtime.behavior = () => Promise.resolve({ logs: [] })
const absent = await runCode(ctx, 'undefined')
expect(absent.content[0]).toEqual({ type: 'text', text: '(run_code completed with no output)' })
expect(absent.isError ? undefined : absent.value).toEqual({ logs: [] })
})
it('reports a pre-aborted outer signal as the run failure without dispatching anything', async () => {

36
packages/core/tools/tests/ts-types.spec.ts
View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from 'vitest'
import { jsonSchemaToTs, renderToolsSdk } from '@deepseek-ai/dsh-tools/src/ts-types.ts'
import type { ToolSdkSchema } from '@deepseek-ai/dsh-tools/src/ts-types.ts'
import { parameterSchemaSpecToJsonSchema } from '@deepseek-ai/dsh-tools'
import type { ToolSchema } from '@deepseek-ai/dsh-llm'
describe('jsonSchemaToTs', () => {
it('maps every unified schema construct', () => {
@@ -96,31 +96,45 @@ describe('jsonSchemaToTs', () => {
})
describe('renderToolsSdk', () => {
const bash: ToolSchema = {
const bash: ToolSdkSchema = {
name: 'bash',
description: 'Run a shell command.',
parameters: parameterSchemaSpecToJsonSchema({ command: { type: 'string', required: true } }) as unknown as Record<string, unknown>,
output: {
type: 'object',
additionalProperties: false,
properties: { exitCode: { type: 'integer' } },
required: ['exitCode'],
},
}
const exotic: ToolSchema = {
const exotic: ToolSdkSchema = {
name: 'my-mcp.tool',
description: 'Exotic name.',
parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record<string, unknown>,
output: { type: 'array', items: { type: 'string' } },
}
it('declares every tool in lexicographic order with quoted keys for exotic names', () => {
const text = renderToolsSdk([exotic, bash])
expect(text).toContain('interface ToolArgsMap {')
expect(text).toContain('interface ToolOutputMap {')
expect(text).toContain('type ToolName = keyof ToolOutputMap')
expect(text).toContain('declare class ToolCallError extends Error')
expect(text).toContain('readonly toolName: ToolName;')
expect(text).toContain('declare const tools: {')
expect(text).toContain('type JsonValue = null | boolean | number | string')
expect(text.indexOf('bash(args:')).toBeGreaterThan(0)
expect(text).toContain('"my-mcp.tool"(args:')
expect(text.indexOf('bash(args:')).toBeLessThan(text.indexOf('"my-mcp.tool"(args:'))
expect(text).toContain('): Promise<string>;')
expect(text.indexOf('bash: {')).toBeGreaterThan(0)
expect(text).toContain('"my-mcp.tool":')
expect(text.indexOf('bash:')).toBeLessThan(text.indexOf('"my-mcp.tool":'))
expect(text).toContain('exitCode: number;')
expect(text).toContain('"my-mcp.tool": string[];')
expect(text).toContain('[K in ToolName]: (args: ToolArgsMap[K]) => Promise<ToolOutputMap[K]>;')
expect(text).toContain('/** Run a shell command. */')
// The fixed instruction lines the model relies on.
expect(text).toContain('erasable syntax only')
expect(text).toContain('rejects with an `Error`')
expect(text).toContain('rejects with `ToolCallError`')
expect(text).toContain('sequentially, even under `Promise.all`')
expect(text).toContain('JSON-serializable')
expect(text).toContain('lossless JSON')
})
it('is deterministic: same tool set, byte-identical text regardless of input order', () => {
@@ -130,6 +144,8 @@ describe('renderToolsSdk', () => {
})
it('renders an empty declaration for an empty tool set', () => {
expect(renderToolsSdk([])).toContain('declare const tools: {}')
const text = renderToolsSdk([])
expect(text).toContain('interface ToolArgsMap {}')
expect(text).toContain('interface ToolOutputMap {}')
})
})