Merge remote-tracking branch 'origin/master' into codex/skill-system
# Conflicts: # docs/architecture.md # docs/config-catalog.md # docs/module-graph.md # docs/rfc/INDEX.md # examples/acp-agent/tests/snapshots/text-turn/session.jsonl # packages/core/agent-core/src/index.ts # packages/core/tools/tests/gen-tool-catalog.spec.ts # packages/support/acp-snapshot/src/suite.ts # packages/ui/acp-agent/src/index.ts
This commit is contained in:
Yichen Jiang
@@ -1,10 +1,10 @@
|
||||
# Packages
|
||||
|
||||
Harness packages, all under the `@deepseek-ai/dsh-*` scope. Each package is a Cordis plugin (microkernel-style): it exports either a default `Service` subclass or a functional plugin, declares its ctx key/events through declaration merging, and exposes extension points through `ctx.effect()`, `ctx.on()`, and `ctx.waterfall()`. Authoring conventions: [AGENTS.md](AGENTS.md) (subtree) and the root [AGENTS.md](../AGENTS.md) § Conventions.
|
||||
Harness packages use the `@deepseek-ai/dsh-*` scope. Each is a Cordis plugin: a default `Service` subclass or functional plugin declaring ctx keys/events through declaration merging and contributing through `ctx.effect()`, `ctx.on()`, or `ctx.waterfall()`. Authoring conventions: [AGENTS.md](AGENTS.md) and root [AGENTS.md](../AGENTS.md) § Conventions.
|
||||
|
||||
## Hierarchy
|
||||
|
||||
Packages are grouped by modular role at `packages/<group>/<pkg>/`. The group directory is a pure container (no `package.json` of its own); the package name stays `@deepseek-ai/dsh-<pkg>` regardless of group. **Each group README is the canonical per-package map** — package roles, ctx keys, and the product-vs-support split live there, next to the code.
|
||||
Packages are grouped by role at `packages/<group>/<pkg>/`. Group directories are containers without a `package.json`; package names remain `@deepseek-ai/dsh-<pkg>`. **Each group README is the canonical per-package map** for roles, ctx keys, and the product/support split.
|
||||
|
||||
| Group | Role | Release expectation |
|
||||
|---|---|---|
|
||||
@@ -16,6 +16,7 @@ Packages are grouped by modular role at `packages/<group>/<pkg>/`. The group dir
|
||||
| [`skill/`](skill/README.md) | Skill capability family: the provider registry, local provider, and model-facing catalog/loader | Product — stable surface |
|
||||
| [`compact/`](compact/README.md) | Compaction capability family: the abstract seam + a basic backend (tool deferred) | Product — stable surface |
|
||||
| [`subagent/`](subagent/README.md) | Subagent capability family: the provider-registry seam and the model-facing delegation tool | Product — stable surface |
|
||||
| [`workflow/`](workflow/README.md) | Workflow capability family: the script-engine seam, the worker-thread engine, and the model-facing `workflow` tool | Product — stable surface |
|
||||
| [`web/`](web/README.md) | Web capability family: the abstract seam, search/fetch provider impls, and the model-facing web tools | Product — stable surface |
|
||||
| [`timeout/`](timeout/README.md) | Tool-call timeout policy: the `tools/execute` deadline enforcer | Product — stable surface |
|
||||
| [`todo/`](todo/README.md) | Todo/planning family: the model-facing `todo_write` tool | Product — stable surface |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# code-runtime/ — code-execution capability family
|
||||
|
||||
The code-execution capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's Code Mode, specified alongside the seam in the [Code Mode RFC](../../docs/rfc/proposed/feature/2026-06-15-code-mode.md). **Product** packages.
|
||||
The code-execution capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): an abstract runtime interface for executing one model-written program against host-provided async bindings, capturing what it printed and returned. The consumer is the tool registry's [Code Mode](../core/tools/README.md) (`tools: { mode: code }` — the `run_code` tool and the generated TypeScript SDK); design in the [Code Mode RFC](../../docs/rfc/implemented/feature/2026-06-15-code-mode.md). **Product** packages.
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# @deepseek-ai/dsh-code-runtime-worker
|
||||
|
||||
Worker-thread implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam: `WorkerCodeRuntime` runs each program in ONE fresh Node `worker_threads.Worker` — TypeScript in, type-stripped host-side, bindings bridged over the message port, `{ value, logs, error? }` out. **Containment, not a security boundary**: trust posture is bash-equivalent by design (the [Code Mode RFC](../../../docs/rfc/proposed/feature/2026-06-15-code-mode.md) § Trust posture), with containment bash does not have — separate isolate, empty environment, heap cap, hard termination.
|
||||
Worker-thread implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam: `WorkerCodeRuntime` runs each program in ONE fresh Node `worker_threads.Worker` — TypeScript in, type-stripped host-side, bindings bridged over the message port, `{ value, logs, error? }` out. **Containment, not a security boundary**: trust posture is bash-equivalent by design (the [Code Mode RFC](../../../docs/rfc/implemented/feature/2026-06-15-code-mode.md) § Trust posture), with containment bash does not have — separate isolate, empty environment, heap cap, hard termination.
|
||||
|
||||
## Config
|
||||
|
||||
|
||||
@@ -11,6 +11,10 @@
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./worker": {
|
||||
"types": "./lib/types/worker.d.ts",
|
||||
"default": "./lib/worker.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
The **code-execution seam**: an abstract `CodeRuntime` service (`ctx.codeRuntime`) defining WHAT a code runtime does — run one model-written program against a set of host-provided async bindings and report `{ value, logs, error? }` — without saying HOW.
|
||||
|
||||
This package is the interface third of the capability (the bash trio is the template — see [capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): implementations subclass `CodeRuntime` and register the service; the consumer is the tool registry's Code Mode, which generates the model-facing SDK and bridges tool dispatch — both specified in the [Code Mode RFC](../../../docs/rfc/proposed/feature/2026-06-15-code-mode.md), whose first implementation is a Node worker-thread backend. The runtime knows nothing about tools or sessions: it is handed named async functions and a program string, and everything tool-shaped stays with the consumer.
|
||||
This package is the interface third of the capability (the bash trio is the template — see [capability seams](../../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)): implementations subclass `CodeRuntime` and register the service; the consumer is the tool registry's Code Mode, which generates the model-facing SDK and bridges tool dispatch — both specified in the [Code Mode RFC](../../../docs/rfc/implemented/feature/2026-06-15-code-mode.md), whose first implementation is a Node worker-thread backend. The runtime knows nothing about tools or sessions: it is handed named async functions and a program string, and everything tool-shaped stays with the consumer.
|
||||
|
||||
## Service API (`ctx.codeRuntime`)
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
* substrate (worker thread, separate process, container) and by source
|
||||
* language, both declared as readonly descriptors. The design and its
|
||||
* consumer (the tool registry's Code Mode) are specified in the Code Mode RFC
|
||||
* (docs/rfc/proposed/feature/2026-06-15-code-mode.md).
|
||||
* (docs/rfc/implemented/feature/2026-06-15-code-mode.md).
|
||||
*
|
||||
* The split mirrors the bash seam (`BashExecutor`): the runtime knows nothing
|
||||
* about tools or sessions — it is handed named async functions and a program,
|
||||
|
||||
@@ -206,6 +206,13 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
'async fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult>',
|
||||
],
|
||||
},
|
||||
{
|
||||
key: 'workflows',
|
||||
summary: 'Abstract workflow execution service.',
|
||||
methods: [
|
||||
'abstract start(request: WorkflowStartRequest): WorkflowRun',
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
/** Every harness event, sorted by name. */
|
||||
@@ -396,6 +403,42 @@ export const EVENT_API: readonly EventApiEntry[] = [
|
||||
signature: '\'tools/pre-execute\'(this: ToolRegistry, exec: ToolExecution, next: () => Promise<PreToolDecision>): Promise<PreToolDecision>',
|
||||
summary: 'Waterfall BEFORE a tool runs — the gate where sandbox, permission, and hook plugins allow or deny a call (Claude Code\'s `PreToolUse`).',
|
||||
},
|
||||
{
|
||||
name: 'workflow/agent-end',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/agent-end\'(info: WorkflowRunInfo, agent: WorkflowAgentEndInfo): void',
|
||||
summary: 'One `agent()` call settled (clean result, child failure, or run cancellation).',
|
||||
},
|
||||
{
|
||||
name: 'workflow/agent-start',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/agent-start\'(info: WorkflowRunInfo, agent: WorkflowAgentInfo): void',
|
||||
summary: 'One `agent()` call started a child run.',
|
||||
},
|
||||
{
|
||||
name: 'workflow/end',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/end\'(info: WorkflowRunInfo, result: WorkflowResultInfo): void',
|
||||
summary: 'A workflow run settled (any stop reason).',
|
||||
},
|
||||
{
|
||||
name: 'workflow/log',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/log\'(info: WorkflowRunInfo, message: string): void',
|
||||
summary: 'The script emitted a narration line (a `log(message)` call).',
|
||||
},
|
||||
{
|
||||
name: 'workflow/phase',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/phase\'(info: WorkflowRunInfo, title: string): void',
|
||||
summary: 'The script entered a phase (a `phase(title)` call) — progress grouping for observers; no execution semantics.',
|
||||
},
|
||||
{
|
||||
name: 'workflow/start',
|
||||
mode: 'emit',
|
||||
signature: '\'workflow/start\'(info: WorkflowRunInfo): void',
|
||||
summary: 'A workflow run started — the script\'s meta block validated, the body about to execute.',
|
||||
},
|
||||
]
|
||||
|
||||
/** Shapes of every exported type the SERVICE_API signatures reference (transitively), sorted by name. */
|
||||
@@ -892,6 +935,34 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'WebSearchSource',
|
||||
declaration: 'export interface WebSearchSource {\n readonly url: string;\n readonly title?: string;\n readonly snippet?: string;\n readonly publishedAt?: string;\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowMeta',
|
||||
declaration: 'export interface WorkflowMeta {\n name: string;\n description: string;\n whenToUse?: string;\n phases?: WorkflowPhase[];\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowPhase',
|
||||
declaration: 'export interface WorkflowPhase {\n title: string;\n detail?: string;\n model?: string;\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowResult',
|
||||
declaration: 'export interface WorkflowResult {\n value: unknown;\n stopReason: WorkflowStopReason;\n error?: string;\n agentsStarted: number;\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowRun',
|
||||
declaration: 'export interface WorkflowRun {\n readonly id: WorkflowRunId;\n readonly meta: WorkflowMeta;\n readonly result: Promise<WorkflowResult>;\n cancel(reason?: string): void;\n dispose(): Promise<void>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowRunId',
|
||||
declaration: 'export type WorkflowRunId = Branded<\'WorkflowRunId\'>;',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowStartRequest',
|
||||
declaration: 'export interface WorkflowStartRequest {\n script: string;\n meta: WorkflowMeta;\n args?: unknown;\n parent: Agent;\n signal?: AbortSignal;\n}',
|
||||
},
|
||||
{
|
||||
name: 'WorkflowStopReason',
|
||||
declaration: 'export type WorkflowStopReason = \'completed\' | \'cancelled\' | \'error\';',
|
||||
},
|
||||
]
|
||||
|
||||
/** The inherited `ctx` surface (cordis core + loader/hmr/timer), in curated order. */
|
||||
|
||||
@@ -39,11 +39,11 @@ This is the [interface/implementation/consumer seam](../../../docs/rfc/implement
|
||||
|
||||
```ts
|
||||
import type { Config } from '@deepseek-ai/dsh-agent-core'
|
||||
// { agents?, persona?, toolOrder?, skills? } — the schema intersects the owner schemas,
|
||||
// { agents?, persona?, toolOrder?, tools?, skills? } — the schema intersects the owner schemas,
|
||||
// so validation and defaulting can never drift from the owners.
|
||||
```
|
||||
|
||||
The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — `persona` to `dsh-system-prompt` (default `''`), the deployment's persona section; `toolOrder` to `dsh-system-prompt` (absent — lexicographic), the explicit model-facing tool order; and `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer. Forwarding is exactly why the owners can live in the shared spine even though the apps disagree on what to configure.
|
||||
The bundle FORWARDS each field to the child that owns it: `agents` to `agent-loop` (default `[]`), so each app supplies its own pre-created agents — a stdio app pre-creates a `main`; the ACP app pre-creates none (it creates agents on demand at `session/new`) — `persona` and `toolOrder` to `dsh-system-prompt`; `tools` to the tool registry for its presentation mode; and `skills.registry`, `skills.local`, and `skills.tool` to the skill registry, local provider, and model-facing consumer. Forwarding is exactly why the owners can live in the shared spine even though the apps disagree on what to configure.
|
||||
|
||||
## Why a code bundle, not a shared YAML include
|
||||
|
||||
|
||||
@@ -51,7 +51,7 @@ import z from 'schemastery'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt, { type Config as SystemPromptConfig } from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools'
|
||||
import SkillService, { type Config as SkillRegistryConfig } from '@deepseek-ai/dsh-skill'
|
||||
import * as SkillLocal from '@deepseek-ai/dsh-skill-local'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
@@ -77,10 +77,11 @@ export interface SkillConfig {
|
||||
* `agents` to the agent loop (an app that pre-creates no agents, like the ACP
|
||||
* bridge, simply omits it), `persona` and `toolOrder` to the system-prompt
|
||||
* plugin (the deployment's persona section and the explicit model-facing tool
|
||||
* order), and `skills` to the skill registry/local provider/tool consumer. Every field is
|
||||
* optional INPUT here because each owner's schema supplies the default (`[]` /
|
||||
* `''` / absent — lexicographic / the DSH skill roots); the schema is the
|
||||
* INTERSECTION of the owners' own schemas, so validation and defaulting can
|
||||
* order), the `tools` object to the tool registry (its presentation `mode`),
|
||||
* and `skills` to the skill registry/local provider/tool consumer. Every field
|
||||
* is optional INPUT here because each owner's schema supplies the default;
|
||||
* the schema is the INTERSECTION of the owners' own schemas (with registry
|
||||
* schemas nested under their bundle keys), so validation and defaulting can
|
||||
* never drift from them.
|
||||
*/
|
||||
export interface Config {
|
||||
@@ -90,6 +91,8 @@ export interface Config {
|
||||
persona?: SystemPromptConfig['persona']
|
||||
/** The explicit model-facing tool order (see dsh-system-prompt's `Config`). */
|
||||
toolOrder?: SystemPromptConfig['toolOrder']
|
||||
/** The tool registry's config — its presentation `mode` (see dsh-tools' `Config`). */
|
||||
tools?: ToolsConfig
|
||||
/** Skill registry, local provider, and model-facing consumer config. */
|
||||
skills?: SkillConfig
|
||||
}
|
||||
@@ -105,7 +108,7 @@ export const SkillConfigSchema: z<SkillConfig> = z.object({
|
||||
export const Config = z.intersect([
|
||||
AgentLoop.Config,
|
||||
SystemPrompt.Config,
|
||||
z.object({ skills: SkillConfigSchema }),
|
||||
z.object({ tools: ToolRegistry.Config, skills: SkillConfigSchema }),
|
||||
]) as unknown as z<Config>
|
||||
|
||||
/**
|
||||
@@ -131,7 +134,7 @@ export function apply(ctx: Context, config: Config): void {
|
||||
persona: config.persona ?? '',
|
||||
...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {},
|
||||
})
|
||||
ctx.plugin(ToolRegistry)
|
||||
ctx.plugin(ToolRegistry, config.tools ?? {})
|
||||
ctx.plugin(SkillService, config.skills?.registry ?? {})
|
||||
ctx.plugin(SkillLocal, config.skills?.local ?? {})
|
||||
ctx.plugin(AgentRegistry)
|
||||
|
||||
@@ -1,9 +1,18 @@
|
||||
# dsh-tools
|
||||
|
||||
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the allow/deny gate) → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context).
|
||||
Tool registry and execution pipeline. Tool plugins register their schemas and executors; the agent loop executes each call through `tools/pre-execute` (the allow/deny gate) → `tools/execute` (an around-dispatch wrapper for timeout/retry/metrics plugins) → `tools/post-execute` (inspect/replace the result, attach context). The registry also owns HOW its tools are presented to the model — its `mode` config selects native function calling, [Code Mode](#code-mode), or both.
|
||||
|
||||
## Service: `ToolRegistry` (ctx key: `tools`)
|
||||
|
||||
### Config
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
mode: native # native (default) | code | both
|
||||
```
|
||||
|
||||
`native` contributes every registered tool as a wire function definition — the default, byte-for-byte the pre-config behavior. `code` contributes exactly ONE wire tool, `run_code`, plus the generated `tools:sdk` prompt section (see [Code Mode](#code-mode)). `both` contributes every native definition AND `run_code` + the SDK section. Non-native modes require a loaded `ctx.codeRuntime` with `language: 'typescript'`; a missing or mismatched runtime rejects every prompt assembly with an actionable error, and a `systemPrompt.toolOrder` naming tools the mode no longer contributes rejects the assembly the same way.
|
||||
|
||||
### Public API
|
||||
|
||||
- `ctx.tools.register(definition: ToolDefinition): () => void` Register a tool. Disposed with the calling fiber.
|
||||
@@ -119,6 +128,16 @@ const bash = defineTool({
|
||||
})
|
||||
```
|
||||
|
||||
### Code Mode
|
||||
|
||||
Under `mode: code` (or `both`) the registry turns the tool surface into a programming API, per the [Code Mode RFC](../../../docs/rfc/implemented/feature/2026-06-15-code-mode.md): the model writes a TypeScript program (the body of an async function) and passes it to the ONE wire tool `run_code`; the program runs in `ctx.codeRuntime` (the [code-execution seam](../../code-runtime/README.md) — the shipped backend is a worker thread) with one async binding per registered tool (`await tools.bash({...})`), and ONLY what it prints or returns re-enters the model's context.
|
||||
|
||||
- **The SDK section** (`tools:sdk`, order 150): a lazy prompt section regenerating, at each assembly, a `declare const tools: {...}` TypeScript declaration of every registered tool except `run_code` (exotic names via quoted keys), plus fixed usage instructions. Deterministic — lexicographic tool order, byte-identical text for an unchanged tool set (prefix-cache-friendly). The codegen (`jsonSchemaToTs`, exported) is TOTAL: constructs outside the `defineTool` subset degrade to `unknown`, never throw.
|
||||
- **The dispatch bridge** (`run_code`'s execute): every binding call is JSON-normalized BEFORE dispatch (a value that does not survive — `BigInt`, circulars — rejects that one call, so the dispatched form and the logged form are the same JSON value by construction), serialized through a per-run queue (even `Promise.all` executes the underlying `ctx.tools.execute()` calls one at a time in submission order — the tool contract carries no concurrency-safety metadata yet), gated by `tools/pre-execute`/`tools/post-execute` like any native call (a deny reaches the program as a binding rejection), and logged as one `tool/code-dispatch` session event (log-only: `deriveMessages()` never surfaces it) with the deterministic sub-id `<parent>:code:<n>`. A failed sub-call REJECTS the program-side promise with the tool's error text — real code error handling, no bespoke envelope. A sub-call's `additionalContext` is deliberately DROPPED (no safe outlet mid-run without breaking tool-call/result adjacency; deferred until a real hook needs it through Code Mode).
|
||||
- **Settlement discipline**: the bridge owns a run-scoped abort that follows the outer signal in and fires when the run settles for any reason, so a budget expiry aborts an in-flight sub-tool instead of orphaning it; the bridge then drains its queue BEFORE returning, so every `tool/code-dispatch` lands inside the open turn. A failed run throws `CodeRunFailedError` (`code: 'CODE_RUN_FAILED'`, message = the failure kind + captured logs), which the pipeline converts to a structured `isError` the model self-corrects from.
|
||||
|
||||
The wire collapse is the registry's own contribution (`systemPrompt.tools()` is mode-aware), so the logged `request/header` records it for free — under `code`, the assembled tool list is exactly `[run_code]`, pinned by tests and the snapshot goldens. Try it: `pnpm run demo:code-mode` ([the coding-agent example's Code Mode overlay](../../../examples/coding-agent/README.md#code-mode)); `pnpm run demo:code-mode acp` serves the same mode over ACP instead of the REPL.
|
||||
|
||||
### What is NOT here (TODO)
|
||||
|
||||
- **Tool shapes review** — when real tools land (e.g. a concurrency-safety hint for parallel execution); phase 1 executes tool calls sequentially.
|
||||
|
||||
@@ -23,13 +23,20 @@
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-code-runtime": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-code-runtime": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
|
||||
318
packages/core/tools/src/code-mode.ts
Normal file
318
packages/core/tools/src/code-mode.ts
Normal file
@@ -0,0 +1,318 @@
|
||||
/**
|
||||
* Code Mode: the `run_code` tool and its dispatch bridge. The model writes a
|
||||
* TypeScript program; the bridge hands it to `ctx.codeRuntime` with one
|
||||
* async binding per registered tool, serializes every binding call through a
|
||||
* per-run queue onto `ToolRegistry.execute()` (so `tools/pre-execute` /
|
||||
* `tools/post-execute` gate sub-calls exactly like native ones), logs each
|
||||
* sub-dispatch as a `tool/code-dispatch` session event, and returns only the
|
||||
* program's curated output. The registry itself decides WHEN this tool
|
||||
* exists (its `mode` config); this module owns only the tool and the bridge.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tools/src/code-mode
|
||||
*/
|
||||
|
||||
import { inspect } from 'node:util'
|
||||
import { CallId, HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { CodeBindingFunction, CodeRunResult, CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
|
||||
import type {} from '@deepseek-ai/dsh-session'
|
||||
import { defineTool } from './schema.ts'
|
||||
import type { ToolDefinition, ToolRegistry } from './index.ts'
|
||||
|
||||
declare module '@deepseek-ai/dsh-session' {
|
||||
interface SessionEventMap {
|
||||
/**
|
||||
* One bridged sub-dispatch from a `run_code` program: the parent
|
||||
* `run_code` call id, the deterministic sub-call id
|
||||
* (`<parent>:code:<n>`), the tool `name` with its JSON-normalized
|
||||
* `arguments` — the exact value dispatched, normalized BEFORE dispatch,
|
||||
* so this append can never fail on payload shape — whether the sub-call
|
||||
* errored, and a bounded `resultSummary` of its model-facing text.
|
||||
* Log-only: `deriveMessages()` ignores it, so sub-calls never re-enter
|
||||
* model context; persistence and UIs get every call. Appended inside the
|
||||
* parent `run_code`'s execution (the bridge drains its queue before
|
||||
* returning), so the turn-enclosure invariant holds by construction.
|
||||
*/
|
||||
'tool/code-dispatch': { parentCallId: CallId; subCallId: CallId; name: string; arguments: unknown; isError: boolean; resultSummary: string }
|
||||
}
|
||||
}
|
||||
|
||||
/** The model-facing name of the Code Mode tool. */
|
||||
export const RUN_CODE_NAME = 'run_code'
|
||||
|
||||
/** The `tools:sdk` section order: inside the 100–199 tool-guidance band, after per-tool guidance sections. */
|
||||
export const SDK_SECTION_ORDER = 150
|
||||
|
||||
/**
|
||||
* Thrown by `run_code` when the program run itself failed — a program
|
||||
* exception, a budget expiry, an abort, or substrate death. Extends
|
||||
* {@link HarnessError} (`code: 'CODE_RUN_FAILED'`); the registry's execution
|
||||
* pipeline converts it into a structured `isError` result whose text carries
|
||||
* the failure kind plus the captured logs, so the model can self-correct.
|
||||
*/
|
||||
export class CodeRunFailedError extends HarnessError {
|
||||
constructor(message: string) {
|
||||
super(message, 'CODE_RUN_FAILED')
|
||||
this.name = 'CodeRunFailedError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cap for a `tool/code-dispatch` event's `resultSummary`. A log-ergonomics
|
||||
* constant, not config: the full result already flows to the program; the
|
||||
* summary exists so log readers see what a sub-call returned at a glance.
|
||||
*/
|
||||
const SUMMARY_MAX_CHARS = 200
|
||||
|
||||
/** Bounded inspect for rendering a program's completion value into the model-facing text. */
|
||||
const INSPECT_OPTIONS = { depth: 4, maxArrayLength: 100, maxStringLength: 10_000 } as const
|
||||
|
||||
/** Join a result's text blocks; a non-text block becomes a placeholder (an MVP limitation, stated in the SDK instructions). */
|
||||
function textOf(content: ContentBlock[]): string {
|
||||
return content
|
||||
.map((block) => {
|
||||
switch (block.type) {
|
||||
case 'text': return block.text
|
||||
// ContentBlockMap is merge-extensible — future block kinds land here
|
||||
// deliberately (no assertNever on merge-extensible unions).
|
||||
default: return `[${block.type} content]`
|
||||
}
|
||||
})
|
||||
.join('\n')
|
||||
}
|
||||
|
||||
/** Bound a sub-call's model-facing text for the log event's `resultSummary`. */
|
||||
function summarize(text: string): string {
|
||||
return text.length > SUMMARY_MAX_CHARS ? `${text.slice(0, SUMMARY_MAX_CHARS)}…` : text
|
||||
}
|
||||
|
||||
/**
|
||||
* JSON-normalize one binding call's argument into TWO independent parses of
|
||||
* the same canonical text: `dispatched` goes to the tool, `logged` to the
|
||||
* `tool/code-dispatch` event — identical by construction (the runtime's
|
||||
* structured-clone boundary is wider than JSON; the session log accepts only
|
||||
* JSON), and separate objects, so a tool mutating its args can neither
|
||||
* desync the log from what was dispatched nor re-poison the append. A value
|
||||
* that does not survive the round-trip (`undefined` — the log rejects it as
|
||||
* event data — `BigInt`, a circular structure, a bare function) rejects that
|
||||
* one call BEFORE dispatch with a model-correctable error: nothing ever
|
||||
* executes unlogged.
|
||||
*/
|
||||
function jsonNormalizeArgs(value: unknown): { dispatched: unknown; logged: unknown } {
|
||||
if (value === undefined) {
|
||||
throw new Error('tool arguments must be JSON-serializable (call the tool with an arguments object, e.g. `{}`)')
|
||||
}
|
||||
let text: string | undefined
|
||||
try {
|
||||
text = JSON.stringify(value)
|
||||
} catch (error: unknown) {
|
||||
throw new Error(`tool arguments must be JSON-serializable: ${error instanceof Error ? error.message : String(error)}`)
|
||||
}
|
||||
// JSON.stringify's lib type claims `string`, but a bare function or symbol
|
||||
// root really yields `undefined` at runtime — the guard is live.
|
||||
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition
|
||||
if (text === undefined) throw new Error('tool arguments must be JSON-serializable (got a value JSON cannot represent)')
|
||||
return { dispatched: JSON.parse(text) as unknown, logged: JSON.parse(text) as unknown }
|
||||
}
|
||||
|
||||
/** Render the program's completion value for the model-facing result text (`''` when the program returned nothing). */
|
||||
function renderValue(value: unknown): string {
|
||||
if (value === undefined) return ''
|
||||
return typeof value === 'string' ? value : inspect(value, INSPECT_OPTIONS)
|
||||
}
|
||||
|
||||
/** The run_code result's `meta` payload (JSON-serializable; `presentResult` narrows it back). */
|
||||
interface RunCodeMeta {
|
||||
logs: CodeRunResult['logs']
|
||||
dispatches: number
|
||||
}
|
||||
|
||||
/** Soft-narrow a result `meta` back to {@link RunCodeMeta} (replay may carry older shapes; presentation must not throw). */
|
||||
function asRunCodeMeta(meta: unknown): RunCodeMeta | undefined {
|
||||
if (typeof meta !== 'object' || meta === null) return undefined
|
||||
const m = meta as Record<string, unknown>
|
||||
if (!Array.isArray(m.logs) || typeof m.dispatches !== 'number') return undefined
|
||||
return m as unknown as RunCodeMeta
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the `run_code` {@link ToolDefinition}: one required `code` parameter,
|
||||
* executed through the dispatch bridge described in the module doc. The
|
||||
* registry registers it under non-native modes.
|
||||
* @param registry - the owning registry (sub-calls go through its `execute`,
|
||||
* bindings cover its registered tools).
|
||||
* @param requireRuntime - resolves `ctx.codeRuntime` or throws the loud
|
||||
* misconfiguration error (shared with the registry's assembly-time checks).
|
||||
* @returns the registry-ready definition.
|
||||
*/
|
||||
export function createRunCodeTool(registry: ToolRegistry, requireRuntime: () => CodeRuntime): ToolDefinition {
|
||||
return defineTool({
|
||||
name: RUN_CODE_NAME,
|
||||
description:
|
||||
'Execute a TypeScript program against the available tools. Write the BODY of an '
|
||||
+ 'async function (erasable syntax only; top-level `await` and `return` work) and '
|
||||
+ 'call tools as `await tools.name(args)` per the declarations in the system prompt. '
|
||||
+ 'Only what you print or return comes back — curate it.',
|
||||
parameters: {
|
||||
code: { type: 'string', required: true, description: 'The program: the body of an async TypeScript function.' },
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const runtime = requireRuntime()
|
||||
|
||||
// The run-scoped abort: follows the outer signal in, and fires when the
|
||||
// run settles for ANY reason, so an in-flight sub-dispatch is aborted
|
||||
// (its executor kills on this signal) instead of orphaned, and
|
||||
// queued-unstarted dispatches are abandoned.
|
||||
const runController = new AbortController()
|
||||
const onOuterAbort = (): void => { runController.abort(exec.signal?.reason) }
|
||||
if (exec.signal?.aborted) onOuterAbort()
|
||||
exec.signal?.addEventListener('abort', onOuterAbort, { once: true })
|
||||
|
||||
let dispatches = 0
|
||||
// The per-run serialization queue: every binding call chains onto the
|
||||
// tail, so even `Promise.all` executes the underlying tool calls one at
|
||||
// a time in submission order (the tool contract carries no
|
||||
// concurrency-safety metadata yet). The fold keeps the tail non-rejecting
|
||||
// so one failed dispatch never poisons the chain.
|
||||
let queue: Promise<void> = Promise.resolve()
|
||||
const enqueue = <T>(task: () => Promise<T>): Promise<T> => {
|
||||
const turn = queue.then(() => {
|
||||
if (runController.signal.aborted) {
|
||||
throw new Error(`run_code run is over (${String(runController.signal.reason)}); tool call abandoned`)
|
||||
}
|
||||
return task()
|
||||
})
|
||||
queue = turn.then(() => undefined, () => undefined)
|
||||
return turn
|
||||
}
|
||||
|
||||
// Read through a call, not a bare property: the abort state genuinely
|
||||
// changes across awaits, and a direct `.aborted` re-check after one
|
||||
// would be narrowed away by control flow analysis.
|
||||
const runOver = (): boolean => runController.signal.aborted
|
||||
|
||||
const binding = (name: string): CodeBindingFunction => async (rawArgs: unknown): Promise<unknown> => {
|
||||
if (runOver()) {
|
||||
throw new Error(`run_code run is over (${String(runController.signal.reason)}); ${name} not dispatched`)
|
||||
}
|
||||
const normalized = jsonNormalizeArgs(rawArgs)
|
||||
const outcome = await enqueue(async () => {
|
||||
const n = ++dispatches
|
||||
const subCallId = CallId(`${String(exec.callId)}:code:${n}`)
|
||||
const result = await registry.execute({
|
||||
callId: subCallId,
|
||||
name,
|
||||
arguments: normalized.dispatched,
|
||||
...exec.agent ? { agent: exec.agent } : {},
|
||||
signal: runController.signal,
|
||||
})
|
||||
const text = textOf(result.content)
|
||||
// Sub-call `additionalContext` is deliberately DROPPED here: the
|
||||
// loop's buffering (append after the step's tool/results) has no
|
||||
// safe analogue from inside a running run_code — injecting now
|
||||
// would break tool-call/result adjacency. Deferred until a real
|
||||
// hook needs it through Code Mode.
|
||||
exec.agent?.session.append('tool/code-dispatch', {
|
||||
parentCallId: exec.callId,
|
||||
subCallId,
|
||||
name,
|
||||
// The SIBLING parse of the dispatched value: byte-identical JSON,
|
||||
// but a separate object — a tool mutating its args cannot desync
|
||||
// this record from what it actually received.
|
||||
arguments: normalized.logged,
|
||||
isError: result.isError,
|
||||
resultSummary: summarize(text),
|
||||
})
|
||||
return { text, isError: result.isError }
|
||||
})
|
||||
// A budget expiry or outer cancel that lands while this call was in
|
||||
// flight already aborted the dispatch; stop the program now rather
|
||||
// than hand it a result from a run that is over.
|
||||
if (runOver()) {
|
||||
throw new Error(`run_code run is over (${String(runController.signal.reason)}); ${name} result discarded`)
|
||||
}
|
||||
// A failed tool call REJECTS — real code signals failure by throwing,
|
||||
// so try/catch and Promise.all short-circuiting behave as models
|
||||
// expect (the error text is the tool's model-facing result text).
|
||||
if (outcome.isError) throw new Error(outcome.text)
|
||||
return outcome.text
|
||||
}
|
||||
|
||||
// Null-prototype + defineProperty, mirroring the worker-side namespace
|
||||
// build: a registered tool named `__proto__` must become an ordinary
|
||||
// own key (a plain-object assignment would hit the prototype setter,
|
||||
// silently dropping the binding), and the runtime host resolves
|
||||
// binding names as own properties only.
|
||||
const functions: Record<string, CodeBindingFunction> = Object.create(null) as Record<string, CodeBindingFunction>
|
||||
for (const schema of registry.schemas()) {
|
||||
if (schema.name === RUN_CODE_NAME) continue
|
||||
Object.defineProperty(functions, schema.name, { enumerable: true, value: binding(schema.name) })
|
||||
}
|
||||
|
||||
try {
|
||||
let result: CodeRunResult
|
||||
try {
|
||||
result = await runtime.run({
|
||||
program: args.code,
|
||||
bindings: [{ global: 'tools', functions }],
|
||||
signal: runController.signal,
|
||||
})
|
||||
} finally {
|
||||
// Quiescence before returning, whether the runtime fulfilled or
|
||||
// REJECTED (a backend that starts a binding call and then throws
|
||||
// must not leak a live sub-dispatch past this settlement): fire
|
||||
// the run-scoped abort (cancelling an in-flight sub-dispatch,
|
||||
// abandoning queued ones), then await the queue's drain — an
|
||||
// aborted sub-call still settles and logs its event INSIDE the
|
||||
// open turn; nothing can append after we return. `queue` is the
|
||||
// FOLDED tail (every link swallows its rejection into undefined),
|
||||
// so this await cannot itself reject — an abandoned queued call
|
||||
// can never mask the runtime's own failure, returned or thrown;
|
||||
// rejections surface only on the per-call promises the program
|
||||
// holds.
|
||||
runController.abort('run_code settled')
|
||||
await queue
|
||||
}
|
||||
|
||||
if (result.error) {
|
||||
const logsText = result.logs.length > 0 ? `\nCaptured output:\n${result.logs.map(entry => entry.text).join('\n')}` : ''
|
||||
throw new CodeRunFailedError(`code run failed (${result.error.kind}): ${result.error.message}${logsText}`)
|
||||
}
|
||||
const rendered = renderValue(result.value)
|
||||
const parts = [result.logs.map(entry => entry.text).join('\n'), rendered].filter(part => part.length > 0)
|
||||
const meta: RunCodeMeta = { logs: result.logs, dispatches }
|
||||
return {
|
||||
content: [{ type: 'text', text: parts.length > 0 ? parts.join('\n') : '(run_code completed with no output)' }],
|
||||
meta,
|
||||
}
|
||||
} finally {
|
||||
exec.signal?.removeEventListener('abort', onOuterAbort)
|
||||
}
|
||||
},
|
||||
// The program IS the title, the way command tools title their cards with
|
||||
// the command: an execute-card's title is the one slot an ACP client
|
||||
// always shows (Zed's execute cards render no body content and no raw
|
||||
// input without a real terminal attached), so anywhere else the code
|
||||
// would be invisible. Multi-line titles are the execute-card idiom —
|
||||
// capable clients render them whole; others truncate to the first line
|
||||
// and still hold the full program in rawInput.
|
||||
presentCall: args => ({
|
||||
card: 'generic',
|
||||
title: args.code,
|
||||
kind: 'execute',
|
||||
rawInput: args.code,
|
||||
}),
|
||||
// Title omitted on the result: an update replaces only the fields it
|
||||
// carries, so the pending card's program title persists through
|
||||
// completion; the captured output rides as body content.
|
||||
presentResult: (_args, result) => {
|
||||
const meta = asRunCodeMeta(result.meta)
|
||||
if (!meta) return undefined
|
||||
const output = meta.logs.map(entry => entry.text).join('\n')
|
||||
return {
|
||||
card: 'generic',
|
||||
...output.length > 0 ? { content: [{ type: 'text' as const, text: output }] } : {},
|
||||
}
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -6,15 +6,26 @@
|
||||
* (inspect/replace the result, attach context) for sandbox, permission, and hook
|
||||
* plugins to gate or transform a call.
|
||||
*
|
||||
* The registry also owns HOW its tools are presented to the model — its
|
||||
* `mode` config: `'native'` (every tool as a wire function definition,
|
||||
* today's behavior and the default), `'code'` (the wire carries exactly one
|
||||
* tool, `run_code`, plus a generated TypeScript SDK prompt section), or
|
||||
* `'both'`. See `code-mode.ts` (the tool + dispatch bridge) and
|
||||
* `ts-types.ts` (the SDK codegen); design in the Code Mode RFC.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tools
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { CallId, ContentBlock, ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
import { HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import type { Agent, HookContext } from '@deepseek-ai/dsh-agent'
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
import type { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
|
||||
import type { ToolCallView, ToolResultView } from './presentation.ts'
|
||||
import { createRunCodeTool, RUN_CODE_NAME, SDK_SECTION_ORDER } from './code-mode.ts'
|
||||
import { renderToolsSdk } from './ts-types.ts'
|
||||
|
||||
export {
|
||||
defineTool,
|
||||
@@ -39,6 +50,9 @@ export {
|
||||
type StructuredScalar,
|
||||
} from './json-schema.ts'
|
||||
|
||||
export { CodeRunFailedError, RUN_CODE_NAME } from './code-mode.ts'
|
||||
export { jsonSchemaToTs, renderToolsSdk } from './ts-types.ts'
|
||||
|
||||
// The render-intent vocabulary a tool declares via `presentCall`/`presentResult`
|
||||
// lives in its own UI-facing module; re-export it so `@deepseek-ai/dsh-tools`
|
||||
// stays the single public surface for consumers (producers + the ACP bridge).
|
||||
@@ -298,20 +312,100 @@ function errorInfo(error: unknown): ToolErrorInfo | undefined {
|
||||
return error instanceof HarnessError ? { name: error.name, code: error.code } : undefined
|
||||
}
|
||||
|
||||
/** How the registry presents its tools to the model (see {@link Config.mode}). */
|
||||
export type ToolPresentationMode = 'native' | 'code' | 'both'
|
||||
|
||||
/** Plugin config: how the registered tools are presented to the model. */
|
||||
export interface Config {
|
||||
/**
|
||||
* The presentation mode. `'native'` (the default) contributes every
|
||||
* registered tool as a wire function definition — byte-for-byte today's
|
||||
* behavior. `'code'` contributes exactly ONE wire tool, `run_code`, plus
|
||||
* the generated `tools:sdk` prompt section declaring every other tool as a
|
||||
* TypeScript API the program calls. `'both'` contributes every native
|
||||
* definition AND `run_code` + the SDK section. Non-native modes require a
|
||||
* loaded `ctx.codeRuntime` whose `language` is `'typescript'` — a missing
|
||||
* or mismatched runtime rejects every prompt assembly with an actionable
|
||||
* error (misconfiguration fails loud, before any model request). A
|
||||
* configured `systemPrompt.toolOrder` naming native tools likewise rejects
|
||||
* every assembly under `'code'` (those names are no longer contributed) —
|
||||
* a deployment switching modes updates its order config or drops it.
|
||||
*/
|
||||
mode?: ToolPresentationMode
|
||||
}
|
||||
|
||||
/**
|
||||
* Tool registry (`ctx.tools`): tool plugins register definitions; the agent
|
||||
* loop executes calls through the `tools/pre-execute` → `tools/execute` →
|
||||
* `tools/post-execute` pipeline. The registry contributes its schemas into the
|
||||
* system-prompt assembly.
|
||||
* system-prompt assembly — WHICH schemas is governed by its `mode` config
|
||||
* (see {@link Config.mode}); under a non-native mode it also registers the
|
||||
* `run_code` tool and the `tools:sdk` prompt section itself.
|
||||
*/
|
||||
export class ToolRegistry extends Service {
|
||||
static inject = ['systemPrompt']
|
||||
|
||||
private store = new Map<string, ToolDefinition>()
|
||||
static Config: z<Config> = z.object({
|
||||
mode: z.union(['native', 'code', 'both'] as const).default('native'),
|
||||
})
|
||||
|
||||
constructor(ctx: Context) {
|
||||
private store = new Map<string, ToolDefinition>()
|
||||
private readonly mode: ToolPresentationMode
|
||||
|
||||
constructor(ctx: Context, config: Config = {}) {
|
||||
super(ctx, 'tools')
|
||||
ctx.systemPrompt.tools(() => this.schemas())
|
||||
// The schema already defaulted an omitted mode; the ?? narrows the
|
||||
// optional-input type for direct (non-Loader) construction in tests.
|
||||
this.mode = config.mode ?? 'native'
|
||||
ctx.systemPrompt.tools(() => this.wireSchemas())
|
||||
if (this.mode !== 'native') {
|
||||
this.register(createRunCodeTool(this, () => this.requireCodeRuntime()))
|
||||
ctx.systemPrompt.section({
|
||||
name: 'tools:sdk',
|
||||
order: SDK_SECTION_ORDER,
|
||||
// A lazy thunk over the live store: regenerated at each assembly, in
|
||||
// lexicographic tool order, so an unchanged tool set renders
|
||||
// byte-identical text (prefix-cache-friendly) and a mid-session
|
||||
// registration surfaces exactly like a native-mode tool change.
|
||||
text: () => {
|
||||
this.requireCodeRuntime()
|
||||
return renderToolsSdk(this.schemas().filter(schema => schema.name !== RUN_CODE_NAME))
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The registry's contribution to the wire tool list, per {@link Config.mode}.
|
||||
* Because `PromptAssembly.tools` is what the loop's request header
|
||||
* snapshots, the mode's collapse is logged and reconstructable for free.
|
||||
* Under a non-native mode this is also the loud misconfiguration gate: no
|
||||
* usable code runtime → every assembly rejects before any model request.
|
||||
*/
|
||||
private wireSchemas(): ToolSchema[] {
|
||||
if (this.mode === 'native') return this.schemas()
|
||||
this.requireCodeRuntime()
|
||||
const all = this.schemas()
|
||||
return this.mode === 'code' ? all.filter(schema => schema.name === RUN_CODE_NAME) : all
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the code runtime or throw the actionable misconfiguration error.
|
||||
* Read at use time (assembly / run_code execution), NOT via static
|
||||
* `inject`: an inject entry would hold `ctx.tools` — and every tool plugin
|
||||
* behind it — hostage to a code runtime existing even under `mode:
|
||||
* 'native'` (the loop's optional-backend idiom, same as
|
||||
* `sessionPersistence`).
|
||||
*/
|
||||
private requireCodeRuntime(): CodeRuntime {
|
||||
const runtime = this.ctx.get('codeRuntime')
|
||||
if (!runtime) {
|
||||
throw new Error(`dsh-tools: mode "${this.mode}" requires a code runtime — load a ctx.codeRuntime implementation (e.g. @deepseek-ai/dsh-code-runtime-worker) or set tools mode to "native"`)
|
||||
}
|
||||
if (runtime.language !== 'typescript') {
|
||||
throw new Error(`dsh-tools: mode "${this.mode}" generates a TypeScript SDK, but the loaded code runtime's language is "${runtime.language}"`)
|
||||
}
|
||||
return runtime
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
121
packages/core/tools/src/ts-types.ts
Normal file
121
packages/core/tools/src/ts-types.ts
Normal file
@@ -0,0 +1,121 @@
|
||||
/**
|
||||
* Code Mode codegen: the pure projection from registered tool schemas to the
|
||||
* TypeScript SDK text the model programs against (the `tools:sdk` prompt
|
||||
* section). Sibling of `json-schema.ts` — `schemas()` (native function
|
||||
* calling) and this module (the generated `declare const tools` surface) are
|
||||
* two projections of the same store.
|
||||
*
|
||||
* TOTAL by design: {@link jsonSchemaToTs} maps the JSON-Schema subset the
|
||||
* `defineTool` DSL emits and degrades every construct outside it (`$ref`,
|
||||
* `oneOf`, `integer`, future MCP shapes, …) to `unknown` without ever
|
||||
* throwing — codegen must never be the thing that fails an assembly.
|
||||
* Deterministic: a fixed tool set renders byte-identical text (tools in
|
||||
* lexicographic name order), so the section is prefix-cache-friendly.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tools/src/ts-types
|
||||
*/
|
||||
|
||||
import type { ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
/** Property names that are valid bare TS identifiers; anything else is quoted. */
|
||||
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/
|
||||
|
||||
/** Render an object key: bare when it is a valid identifier, quoted otherwise (every name stays reachable, no aliasing). */
|
||||
function renderKey(name: string): string {
|
||||
return IDENTIFIER.test(name) ? name : JSON.stringify(name)
|
||||
}
|
||||
|
||||
/** One `indent`-deep line prefix (two spaces per level). */
|
||||
function pad(indent: number): string {
|
||||
return ' '.repeat(indent)
|
||||
}
|
||||
|
||||
/** A one-line JSDoc block for a schema `description`, or no lines when there is none. */
|
||||
function docLines(description: unknown, indent: number): string[] {
|
||||
if (typeof description !== 'string' || description.length === 0) return []
|
||||
// Keep the doc a single-line comment per property: descriptions are prose
|
||||
// (possibly with newlines); collapse whitespace so the rendered SDK stays
|
||||
// stable and compact. A comment-closer inside the description is escaped so
|
||||
// it cannot terminate the generated JSDoc early.
|
||||
const collapsed = description.replace(/\s+/g, ' ').trim()
|
||||
return [`${pad(indent)}/** ${collapsed.replaceAll('*/', String.raw`*\/`)} */`]
|
||||
}
|
||||
|
||||
/**
|
||||
* Map one JSON-Schema node to a TypeScript type literal. Handles exactly the
|
||||
* subset the `defineTool` DSL emits — `object` (`properties` + `required`),
|
||||
* `string` (with `enum` → a literal union), `number`, `boolean`, `array`
|
||||
* (`items`) — and returns `unknown` for anything else, without throwing.
|
||||
* @param schema - the JSON-Schema node (any shape; hostile inputs degrade).
|
||||
* @param indent - the indentation level for nested object members.
|
||||
* @returns the TS type text (multi-line for objects with properties).
|
||||
*/
|
||||
export function jsonSchemaToTs(schema: unknown, indent = 0): string {
|
||||
if (typeof schema !== 'object' || schema === null) return 'unknown'
|
||||
const node = schema as Record<string, unknown>
|
||||
switch (node.type) {
|
||||
case 'string': {
|
||||
if (Array.isArray(node.enum) && node.enum.length > 0 && node.enum.every(value => typeof value === 'string')) {
|
||||
return node.enum.map(value => JSON.stringify(value)).join(' | ')
|
||||
}
|
||||
return 'string'
|
||||
}
|
||||
case 'number': return 'number'
|
||||
case 'boolean': return 'boolean'
|
||||
case 'array': {
|
||||
const item = jsonSchemaToTs(node.items, indent)
|
||||
// Parenthesize a union item type so `('a' | 'b')[]` parses as intended.
|
||||
return item.includes('|') ? `(${item})[]` : `${item}[]`
|
||||
}
|
||||
case 'object': {
|
||||
const properties = node.properties
|
||||
if (typeof properties !== 'object' || properties === null) return 'Record<string, unknown>'
|
||||
const entries = Object.entries(properties as Record<string, unknown>)
|
||||
if (entries.length === 0) return 'Record<string, unknown>'
|
||||
const required = new Set(Array.isArray(node.required) ? node.required.filter(name => typeof name === 'string') : [])
|
||||
const lines: string[] = ['{']
|
||||
for (const [name, prop] of entries) {
|
||||
const description = typeof prop === 'object' && prop !== null ? (prop as Record<string, unknown>).description : undefined
|
||||
lines.push(...docLines(description, indent + 1))
|
||||
lines.push(`${pad(indent + 1)}${renderKey(name)}${required.has(name) ? '' : '?'}: ${jsonSchemaToTs(prop, indent + 1)};`)
|
||||
}
|
||||
lines.push(`${pad(indent)}}`)
|
||||
return lines.join('\n')
|
||||
}
|
||||
default: return 'unknown'
|
||||
}
|
||||
}
|
||||
|
||||
/** The fixed model-facing usage contract rendered above the declarations (see the Code Mode RFC's "What the model sees"). */
|
||||
const SDK_INSTRUCTIONS = `## Writing code for run_code
|
||||
|
||||
Pass \`run_code\` the body of an async TypeScript function (erasable syntax only — no \`enum\` or namespaces; type annotations are advisory, the code runs type-stripped). Inside the program:
|
||||
|
||||
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to the tool's text output as a string. Tool arguments must be JSON-serializable.
|
||||
- A FAILED tool call rejects with an \`Error\` carrying the tool's error text — \`try/catch\` it to handle and continue.
|
||||
- Calls execute sequentially, even under \`Promise.all\`.
|
||||
- Emit results with \`return\` and/or \`console.log(...)\`. ONLY what you print or return comes back to you — intermediate tool results never enter the conversation, so extract just what you need.
|
||||
|
||||
The available tools:`
|
||||
|
||||
/**
|
||||
* Render the full `tools:sdk` prompt section: the fixed usage instructions
|
||||
* plus one `declare const tools` interface covering every given tool.
|
||||
* Deterministic — tools are emitted in lexicographic name order, so an
|
||||
* unchanged tool set produces byte-identical text across assemblies.
|
||||
* @param schemas - the tool schemas to declare (the caller excludes
|
||||
* `run_code` itself).
|
||||
* @returns the complete section text.
|
||||
*/
|
||||
export function renderToolsSdk(schemas: ToolSchema[]): string {
|
||||
const sorted = [...schemas].sort((a, b) => a.name < b.name ? -1 : a.name > b.name ? 1 : 0)
|
||||
const members: string[] = []
|
||||
for (const schema of sorted) {
|
||||
members.push(...docLines(schema.description, 1))
|
||||
members.push(`${pad(1)}${renderKey(schema.name)}(args: ${jsonSchemaToTs(schema.parameters, 1)}): Promise<string>;`)
|
||||
}
|
||||
const declaration = members.length > 0
|
||||
? `declare const tools: {\n${members.join('\n')}\n}`
|
||||
: 'declare const tools: {}'
|
||||
return `${SDK_INSTRUCTIONS}\n\n\`\`\`ts\n${declaration}\n\`\`\``
|
||||
}
|
||||
640
packages/core/tools/tests/code-mode.spec.ts
Normal file
640
packages/core/tools/tests/code-mode.spec.ts
Normal file
@@ -0,0 +1,640 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
|
||||
import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
|
||||
import ToolRegistry, { CodeRunFailedError, RUN_CODE_NAME, defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { Config, PostToolDecision, ToolExecutionResult } from '@deepseek-ai/dsh-tools'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { Session, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEventMap } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* Code Mode unit tier (per the RFC's plan): provider contribution per mode,
|
||||
* misconfiguration rejections, the run_code dispatch bridge (serialization,
|
||||
* abort, JSON normalization, error mapping, events, quiescence), and HMR
|
||||
* safety — all against an in-repo fake runtime, exactly the
|
||||
* interface/implementation/consumer shape the seam promises.
|
||||
*/
|
||||
|
||||
/** A scriptable in-repo CodeRuntime: each test sets `behavior` to drive the bindings however it needs. */
|
||||
class FakeRuntime extends CodeRuntime {
|
||||
readonly language: string
|
||||
readonly isolation = 'fake'
|
||||
behavior: (request: CodeRunRequest) => Promise<CodeRunResult> = () => Promise.resolve({ logs: [] })
|
||||
lastRequest?: CodeRunRequest
|
||||
|
||||
constructor(ctx: Context, config: { language?: string } = {}) {
|
||||
super(ctx)
|
||||
this.language = config.language ?? 'typescript'
|
||||
}
|
||||
|
||||
run(request: CodeRunRequest): Promise<CodeRunResult> {
|
||||
this.lastRequest = request
|
||||
return this.behavior(request)
|
||||
}
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
mode?: Config['mode']
|
||||
runtime?: false | { language?: string }
|
||||
toolOrder?: string[]
|
||||
}
|
||||
|
||||
async function setup(options: SetupOptions = {}) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt, { ...options.toolOrder ? { toolOrder: options.toolOrder } : {} })
|
||||
await ctx.plugin(ToolRegistry, { mode: options.mode ?? 'code' })
|
||||
let runtime: FakeRuntime | undefined
|
||||
if (options.runtime !== false) {
|
||||
await ctx.plugin(FakeRuntime, options.runtime ?? {})
|
||||
runtime = ctx.codeRuntime as FakeRuntime
|
||||
}
|
||||
return { ctx, tools: ctx.tools, systemPrompt: ctx.systemPrompt, runtime: runtime! }
|
||||
}
|
||||
|
||||
/** Register a trivial echo tool; returns the calls it received. */
|
||||
function registerEcho(ctx: Context, name = 'echo'): unknown[] {
|
||||
const calls: unknown[] = []
|
||||
ctx.tools.register(defineTool({
|
||||
name,
|
||||
description: `Echo tool ${name}.`,
|
||||
parameters: { value: { type: 'string', required: true } },
|
||||
execute(args) {
|
||||
calls.push(args)
|
||||
return Promise.resolve([{ type: 'text' as const, text: `${name}:${args.value}` }])
|
||||
},
|
||||
}))
|
||||
return calls
|
||||
}
|
||||
|
||||
/** A structural fake of the owning agent: captures session appends. */
|
||||
function fakeAgent(): { agent: Agent; events: { type: string; data: unknown }[] } {
|
||||
const events: { type: string; data: unknown }[] = []
|
||||
const agent = {
|
||||
session: {
|
||||
append: (type: string, data: unknown) => { events.push({ type, data }) },
|
||||
},
|
||||
} as unknown as Agent
|
||||
return { agent, events }
|
||||
}
|
||||
|
||||
/** Dispatch run_code through the registry pipeline, as the loop would. */
|
||||
async function runCode(ctx: Context, code: string, extras: { agent?: Agent; signal?: AbortSignal } = {}): Promise<ToolExecutionResult> {
|
||||
return ctx.tools.execute({
|
||||
callId: CallId('call-1'),
|
||||
name: RUN_CODE_NAME,
|
||||
arguments: { code },
|
||||
...extras.agent ? { agent: extras.agent } : {},
|
||||
...extras.signal ? { signal: extras.signal } : {},
|
||||
})
|
||||
}
|
||||
|
||||
describe('mode-aware wire contribution', () => {
|
||||
it("mode 'native' contributes every schema, no run_code, no SDK section — and needs no runtime", async () => {
|
||||
const { ctx, systemPrompt } = await setup({ mode: 'native', runtime: false })
|
||||
registerEcho(ctx)
|
||||
const assembly = await systemPrompt.assemble()
|
||||
expect(assembly.tools.map(tool => tool.name)).toEqual(['echo'])
|
||||
expect(assembly.sections.some(section => section.name === 'tools:sdk')).toBe(false)
|
||||
})
|
||||
|
||||
it("mode 'code' contributes exactly [run_code] plus the SDK section declaring the other tools", async () => {
|
||||
const { ctx, systemPrompt } = await setup({ mode: 'code' })
|
||||
registerEcho(ctx)
|
||||
const assembly = await systemPrompt.assemble()
|
||||
expect(assembly.tools.map(tool => tool.name)).toEqual([RUN_CODE_NAME])
|
||||
const sdk = assembly.sections.find(section => section.name === 'tools:sdk')
|
||||
expect(sdk?.text).toContain('declare const tools: {')
|
||||
expect(sdk?.text).toContain('echo(args:')
|
||||
expect(sdk?.text).not.toContain('run_code(args:')
|
||||
})
|
||||
|
||||
it("mode 'both' contributes every native schema plus run_code, and the SDK section", async () => {
|
||||
const { ctx, systemPrompt } = await setup({ mode: 'both' })
|
||||
registerEcho(ctx)
|
||||
const assembly = await systemPrompt.assemble()
|
||||
expect(assembly.tools.map(tool => tool.name)).toEqual(['echo', RUN_CODE_NAME])
|
||||
expect(assembly.sections.some(section => section.name === 'tools:sdk')).toBe(true)
|
||||
})
|
||||
|
||||
it("never exposes run_code to programs, even under mode 'both' (no recursive dispatch path)", async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'both' })
|
||||
registerEcho(ctx)
|
||||
runtime.behavior = (request) => {
|
||||
const functions = request.bindings[0]!.functions
|
||||
return Promise.resolve({
|
||||
logs: [],
|
||||
value: JSON.stringify({
|
||||
names: Object.keys(functions).sort(),
|
||||
// Own-property AND prototype-chain reads both come back empty —
|
||||
// there is no handle a program could re-enter run_code through.
|
||||
runCode: String(functions[RUN_CODE_NAME]),
|
||||
}),
|
||||
})
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(JSON.parse((result.content[0] as { text: string }).text)).toEqual({ names: ['echo'], runCode: 'undefined' })
|
||||
})
|
||||
|
||||
it('renders byte-identical SDK text across consecutive assemblies of an unchanged tool set', async () => {
|
||||
const { ctx, systemPrompt } = await setup({ mode: 'code' })
|
||||
registerEcho(ctx)
|
||||
const first = await systemPrompt.assemble()
|
||||
const second = await systemPrompt.assemble()
|
||||
const text = (assembly: typeof first) => assembly.sections.find(section => section.name === 'tools:sdk')?.text
|
||||
expect(text(first)).toBe(text(second))
|
||||
})
|
||||
|
||||
it('rejects every assembly when a non-native mode has no code runtime', async () => {
|
||||
const { systemPrompt } = await setup({ mode: 'code', runtime: false })
|
||||
await expect(systemPrompt.assemble()).rejects.toThrow(/requires a code runtime/)
|
||||
})
|
||||
|
||||
it("rejects every assembly when the runtime's language is not typescript", async () => {
|
||||
const { systemPrompt } = await setup({ mode: 'code', runtime: { language: 'python' } })
|
||||
await expect(systemPrompt.assemble()).rejects.toThrow(/language is "python"/)
|
||||
})
|
||||
|
||||
it("rejects the assembly when toolOrder names a native tool that mode 'code' no longer contributes", async () => {
|
||||
const { ctx, systemPrompt } = await setup({ mode: 'code', toolOrder: ['echo', '<unlisted-tools>'] })
|
||||
registerEcho(ctx)
|
||||
await expect(systemPrompt.assemble()).rejects.toThrow(/toolOrder lists unregistered tool "echo"/)
|
||||
})
|
||||
|
||||
it('removes run_code and the SDK section when the registry fiber disposes (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt, {})
|
||||
await ctx.plugin(FakeRuntime, {})
|
||||
const fiber = await ctx.plugin(ToolRegistry, { mode: 'code' })
|
||||
expect(ctx.tools.get(RUN_CODE_NAME)).toBeDefined()
|
||||
await fiber.dispose()
|
||||
const assembly = await ctx.systemPrompt.assemble()
|
||||
expect(assembly.tools).toEqual([])
|
||||
expect(assembly.sections.some(section => section.name === 'tools:sdk')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('the run_code dispatch bridge', () => {
|
||||
it('bridges tool calls, returns only the curated output, and logs one event per dispatch', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
const tools = request.bindings[0]!.functions
|
||||
const first = await tools.echo!({ value: 'one' })
|
||||
const second = await tools.echo!({ value: 'two' })
|
||||
return { logs: [{ source: 'console', level: 'log', text: `saw ${String(first)}` }], value: second }
|
||||
}
|
||||
const result = await runCode(ctx, 'const …: string = …', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content).toEqual([{ type: 'text', text: 'saw echo:one\necho:two' }])
|
||||
expect(calls).toEqual([{ value: 'one' }, { value: 'two' }])
|
||||
const dispatches = events.filter(event => event.type === 'tool/code-dispatch')
|
||||
expect(dispatches.map(event => event.data)).toEqual([
|
||||
{ parentCallId: 'call-1', subCallId: 'call-1:code:1', name: 'echo', arguments: { value: 'one' }, isError: false, resultSummary: 'echo:one' },
|
||||
{ parentCallId: 'call-1', subCallId: 'call-1:code:2', name: 'echo', arguments: { value: 'two' }, isError: false, resultSummary: 'echo:two' },
|
||||
])
|
||||
expect(result.meta).toEqual({ logs: [{ source: 'console', level: 'log', text: 'saw echo:one' }], dispatches: 2 })
|
||||
})
|
||||
|
||||
it('serializes Promise.all dispatches: tool executions never overlap, in submission order', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const intervals: [string, string][] = []
|
||||
let active = 0
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'probe',
|
||||
description: 'Records execution overlap.',
|
||||
parameters: { id: { type: 'string', required: true } },
|
||||
async execute(args) {
|
||||
active++
|
||||
expect(active, 'probe executions overlapped').toBe(1)
|
||||
intervals.push(['enter', args.id])
|
||||
await new Promise(resolve => setTimeout(resolve, 20))
|
||||
intervals.push(['exit', args.id])
|
||||
active--
|
||||
return [{ type: 'text' as const, text: args.id }]
|
||||
},
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
const tools = request.bindings[0]!.functions
|
||||
const values = await Promise.all([tools.probe!({ id: 'a' }), tools.probe!({ id: 'b' }), tools.probe!({ id: 'c' })])
|
||||
return { logs: [], value: values.join(',') }
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(intervals).toEqual([
|
||||
['enter', 'a'], ['exit', 'a'],
|
||||
['enter', 'b'], ['exit', 'b'],
|
||||
['enter', 'c'], ['exit', 'c'],
|
||||
])
|
||||
expect(result.content[0]).toEqual({ type: 'text', text: 'a,b,c' })
|
||||
})
|
||||
|
||||
it('rejects the program-side call when the tool errors, with the tool error text', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'fail',
|
||||
description: 'Always fails.',
|
||||
parameters: {},
|
||||
execute(): Promise<never> { return Promise.reject(new Error('deliberate failure')) },
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.fail!({})
|
||||
return { logs: [], value: 'unreachable' }
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: `caught: ${error instanceof Error ? error.message : String(error)}` }
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.content[0]).toEqual({ type: 'text', text: 'caught: Error: deliberate failure' })
|
||||
})
|
||||
|
||||
it('a tools/pre-execute deny reaches the program as a binding rejection', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
registerEcho(ctx)
|
||||
ctx.on('tools/pre-execute', (exec, next) => {
|
||||
if (exec.name === 'echo') return Promise.resolve({ kind: 'deny' as const, reason: 'not on my watch' })
|
||||
return next()
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: `denied: ${error instanceof Error ? error.message : String(error)}` }
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.content[0]?.type).toBe('text')
|
||||
expect((result.content[0] as { text: string }).text).toContain('not on my watch')
|
||||
})
|
||||
|
||||
it('rejects a binding argument that does not survive JSON normalization, dispatching nothing', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
try {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x', big: 1n })
|
||||
return { logs: [], value: 'unreachable' }
|
||||
} catch (error: unknown) {
|
||||
return { logs: [], value: error instanceof Error ? error.message : String(error) }
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect((result.content[0] as { text: string }).text).toContain('JSON-serializable')
|
||||
expect(calls).toEqual([])
|
||||
expect(events.filter(event => event.type === 'tool/code-dispatch')).toEqual([])
|
||||
})
|
||||
|
||||
it('dispatches the JSON-normalized value: what the tool sees is what the event logs', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
// A Date survives structured clone but is not JSON; the bridge
|
||||
// normalizes it to its JSON form (an ISO string) BEFORE dispatch.
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x', when: new Date(0) }).catch(() => undefined)
|
||||
return { logs: [] }
|
||||
}
|
||||
await runCode(ctx, 'program', { agent })
|
||||
expect(calls).toEqual([{ value: 'x', when: '1970-01-01T00:00:00.000Z' }])
|
||||
const dispatch = events.find(event => event.type === 'tool/code-dispatch')?.data as SessionEventMap['tool/code-dispatch']
|
||||
expect(dispatch.arguments).toEqual({ value: 'x', when: '1970-01-01T00:00:00.000Z' })
|
||||
})
|
||||
|
||||
it('suppresses sub-call additionalContext (deliberately; pinned)', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
registerEcho(ctx)
|
||||
ctx.on('tools/post-execute', (exec, _result, next): Promise<PostToolDecision> => {
|
||||
if (exec.name === 'echo') {
|
||||
return Promise.resolve({
|
||||
kind: 'accept' as const,
|
||||
additionalContext: { content: [{ type: 'text' as const, text: 'context for the next request' }], source: { kind: 'plugin' as const, plugin: 'test' } },
|
||||
})
|
||||
}
|
||||
return next()
|
||||
})
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'done' }
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
// The sub-call's context has no safe outlet mid-run; the parent result
|
||||
// must not carry it either.
|
||||
expect(result.additionalContext).toBeUndefined()
|
||||
})
|
||||
|
||||
it('converts a failed run into a structured isError result carrying kind, message, and captured logs', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
runtime.behavior = () => Promise.resolve({
|
||||
logs: [{ source: 'console', level: 'log', text: 'got this far' }],
|
||||
error: { kind: 'timeout', message: 'compute budget exhausted (300ms busy)' },
|
||||
})
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error).toEqual({ name: 'CodeRunFailedError', code: 'CODE_RUN_FAILED' })
|
||||
const text = (result.content[0] as { text: string }).text
|
||||
expect(text).toContain('code run failed (timeout)')
|
||||
expect(text).toContain('compute budget exhausted')
|
||||
expect(text).toContain('got this far')
|
||||
})
|
||||
|
||||
it('CodeRunFailedError is a HarnessError with the CODE_RUN_FAILED code', () => {
|
||||
const error = new CodeRunFailedError('boom')
|
||||
expect(error.code).toBe('CODE_RUN_FAILED')
|
||||
expect(error.name).toBe('CodeRunFailedError')
|
||||
})
|
||||
|
||||
it('aborting the outer signal aborts the in-flight sub-dispatch and abandons queued ones', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const seen: string[] = []
|
||||
let sawAbort = false
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'slow',
|
||||
description: 'Slow tool observing its signal.',
|
||||
parameters: { id: { type: 'string', required: true } },
|
||||
async execute(args, exec) {
|
||||
seen.push(args.id)
|
||||
await new Promise<void>((resolve) => {
|
||||
const timer = setTimeout(resolve, 500)
|
||||
exec.signal?.addEventListener('abort', () => { sawAbort = true; clearTimeout(timer); resolve() }, { once: true })
|
||||
})
|
||||
return [{ type: 'text' as const, text: args.id }]
|
||||
},
|
||||
}))
|
||||
const controller = new AbortController()
|
||||
runtime.behavior = async (request) => {
|
||||
const tools = request.bindings[0]!.functions
|
||||
const calls = [tools.slow!({ id: 'first' }).catch(() => 'rejected'), tools.slow!({ id: 'second' }).catch(() => 'rejected')]
|
||||
setTimeout(() => { controller.abort('user-cancel') }, 50)
|
||||
await Promise.all(calls)
|
||||
// A real runtime would be terminated by the abort; the fake honors the
|
||||
// contract by reporting the abort as the run failure.
|
||||
return { logs: [], error: { kind: 'abort', message: 'user-cancel' } }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('code run failed (abort)')
|
||||
expect(seen).toEqual(['first'])
|
||||
expect(sawAbort).toBe(true)
|
||||
})
|
||||
|
||||
it('a runtime that starts a binding call and then REJECTS still reaches quiescence before returning', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const { agent, events } = fakeAgent()
|
||||
let sawAbort = false
|
||||
let started!: () => void
|
||||
const inFlight = new Promise<void>((resolve) => { started = resolve })
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'slow',
|
||||
description: 'Slow tool observing its signal.',
|
||||
parameters: { id: { type: 'string', required: true } },
|
||||
async execute(args, exec) {
|
||||
started()
|
||||
await new Promise<void>((resolve) => {
|
||||
const timer = setTimeout(resolve, 500)
|
||||
exec.signal?.addEventListener('abort', () => { sawAbort = true; clearTimeout(timer); resolve() }, { once: true })
|
||||
})
|
||||
return [{ type: 'text' as const, text: args.id }]
|
||||
},
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
// Start a sub-dispatch, keep its rejection held, and fail the run once
|
||||
// the tool is genuinely in flight — a seam error AFTER work has begun.
|
||||
// The bridge's settlement still owes quiescence: without the finally,
|
||||
// run_code would return now and the slow tool would finish (and log)
|
||||
// afterwards.
|
||||
request.bindings[0]!.functions.slow!({ id: 'orphan' }).catch(() => 'held')
|
||||
await inFlight
|
||||
throw new Error('backend exploded')
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('backend exploded')
|
||||
// Quiescence held: the in-flight sub-dispatch was aborted and its event
|
||||
// logged INSIDE the run_code execution, not after it returned.
|
||||
expect(sawAbort).toBe(true)
|
||||
expect(events.filter(event => event.type === 'tool/code-dispatch').map(event => (event.data as { name: string }).name)).toEqual(['slow'])
|
||||
})
|
||||
|
||||
it('runs without an owning agent: dispatches work, event logging is skipped', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
return { logs: [], value: 'ok' }
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(calls).toEqual([{ value: 'x' }])
|
||||
})
|
||||
|
||||
it('executing run_code under a missing runtime is a structured isError, not a crash', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt, {})
|
||||
await ctx.plugin(ToolRegistry, { mode: 'code' })
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('requires a code runtime')
|
||||
})
|
||||
|
||||
it('presents the PROGRAM as the execute-card title on both call and result (the one slot execute cards always show)', async () => {
|
||||
const { ctx } = await setup({ mode: 'code' })
|
||||
const tool = ctx.tools.get(RUN_CODE_NAME)!
|
||||
// The program IS the title, mirroring how command tools title their cards
|
||||
// with the command: an ACP client's execute-card header is the only
|
||||
// always-visible slot (Zed renders no body content and no raw input for
|
||||
// execute-kind cards without a real terminal).
|
||||
expect(tool.presentCall?.({ code: 'return 1' })).toEqual({
|
||||
card: 'generic',
|
||||
title: 'return 1',
|
||||
kind: 'execute',
|
||||
rawInput: 'return 1',
|
||||
})
|
||||
const view = tool.presentResult?.({ code: 'return 1' }, {
|
||||
content: [{ type: 'text', text: 'model-facing' }],
|
||||
isError: false,
|
||||
meta: { logs: [{ source: 'console', level: 'log', text: 'printed' }], dispatches: 1 },
|
||||
})
|
||||
// The result omits the title — an update replaces only provided fields,
|
||||
// so the pending card's program title persists through completion.
|
||||
expect(view).toEqual({
|
||||
card: 'generic',
|
||||
content: [{ type: 'text', text: 'printed' }],
|
||||
})
|
||||
// No captured output → no content either; everything pending persists.
|
||||
expect(tool.presentResult?.({ code: 'x' }, { content: [], isError: false, meta: { logs: [], dispatches: 2 } }))
|
||||
.toEqual({ card: 'generic' })
|
||||
// Replay with an unrecognizable meta falls back to the generic rendering.
|
||||
expect(tool.presentResult?.({ code: 'x' }, { content: [], isError: false, meta: { other: true } })).toBeUndefined()
|
||||
expect(tool.presentResult?.({ code: 'x' }, { content: [], isError: false })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('renders non-text sub-result blocks as placeholders and truncates long event summaries', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const { agent, events } = fakeAgent()
|
||||
const long = 'x'.repeat(300)
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'mixed',
|
||||
description: 'Returns mixed content.',
|
||||
parameters: {},
|
||||
execute() {
|
||||
return Promise.resolve([
|
||||
{ type: 'text' as const, text: long },
|
||||
{ type: 'reasoning' as const, text: 'hidden' },
|
||||
])
|
||||
},
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
const value = await request.bindings[0]!.functions.mixed!({})
|
||||
return { logs: [], value }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
expect((result.content[0] as { text: string }).text).toBe(`${long}\n[reasoning content]`)
|
||||
const dispatch = events.find(event => event.type === 'tool/code-dispatch')?.data as SessionEventMap['tool/code-dispatch']
|
||||
expect(dispatch.resultSummary.length).toBe(201)
|
||||
expect(dispatch.resultSummary.endsWith('…')).toBe(true)
|
||||
})
|
||||
|
||||
it('rejects undefined, JSON-throwing, and JSON-unrepresentable binding arguments BEFORE dispatch', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
const { agent, events } = fakeAgent()
|
||||
runtime.behavior = async (request) => {
|
||||
const echo = request.bindings[0]!.functions.echo!
|
||||
const catchMessage = (promise: Promise<unknown>) => promise.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return {
|
||||
logs: [],
|
||||
value: [
|
||||
// Root undefined must reject up front: the event log rejects it as
|
||||
// data, and nothing may execute unlogged.
|
||||
await catchMessage(echo(undefined)),
|
||||
// A toJSON that throws a NON-Error propagates out of JSON.stringify.
|
||||
await catchMessage(echo({ toJSON() { throw 'raw-throw' } })),
|
||||
// A bare function is a value JSON cannot represent at all.
|
||||
await catchMessage(echo(() => 1)),
|
||||
].join(' | '),
|
||||
}
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
const text = (result.content[0] as { text: string }).text
|
||||
expect(text).toContain('call the tool with an arguments object')
|
||||
expect(text).toContain('JSON-serializable: raw-throw')
|
||||
expect(text).toContain('a value JSON cannot represent')
|
||||
// None of the three dispatched, none logged.
|
||||
expect(calls).toEqual([])
|
||||
expect(events.filter(event => event.type === 'tool/code-dispatch')).toEqual([])
|
||||
})
|
||||
|
||||
it('logs the value the tool RECEIVED even when the tool mutates its arguments', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const { agent, events } = fakeAgent()
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'mutator',
|
||||
description: 'Mutates its own args object.',
|
||||
parameters: { list: { type: 'array', required: true } },
|
||||
execute(args) {
|
||||
args.list.push('injected-by-tool')
|
||||
return Promise.resolve([{ type: 'text' as const, text: 'mutated' }])
|
||||
},
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
await request.bindings[0]!.functions.mutator!({ list: ['original'] })
|
||||
return { logs: [] }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { agent })
|
||||
expect(result.isError).toBe(false)
|
||||
const dispatch = events.find(event => event.type === 'tool/code-dispatch')?.data as SessionEventMap['tool/code-dispatch']
|
||||
expect(dispatch.arguments).toEqual({ list: ['original'] })
|
||||
})
|
||||
|
||||
it('exposes a tool named __proto__ as an ordinary own binding', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
ctx.tools.register(defineTool({
|
||||
name: '__proto__',
|
||||
description: 'A prototype-colliding tool name.',
|
||||
parameters: {},
|
||||
execute() { return Promise.resolve([{ type: 'text' as const, text: 'proto-tool-ok' }]) },
|
||||
}))
|
||||
runtime.behavior = async (request) => {
|
||||
const functions = request.bindings[0]!.functions
|
||||
expect(Object.getPrototypeOf(functions)).toBeNull()
|
||||
const value = await functions['__proto__']!({})
|
||||
return { logs: [], value }
|
||||
}
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect(result.isError).toBe(false)
|
||||
expect(result.content[0]).toEqual({ type: 'text', text: 'proto-tool-ok' })
|
||||
})
|
||||
|
||||
it('renders a non-string completion value inspect-style', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
runtime.behavior = () => Promise.resolve({ logs: [], value: { n: 42 } })
|
||||
const result = await runCode(ctx, 'program')
|
||||
expect((result.content[0] as { text: string }).text).toBe('{ n: 42 }')
|
||||
})
|
||||
|
||||
it('reports a pre-aborted outer signal as the run failure without dispatching anything', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
runtime.behavior = (request) => {
|
||||
// The fake honors the seam contract for an already-aborted signal.
|
||||
if (request.signal?.aborted) return Promise.resolve({ logs: [], error: { kind: 'abort' as const, message: String(request.signal.reason) } })
|
||||
return Promise.resolve({ logs: [], value: 'unreachable' })
|
||||
}
|
||||
const controller = new AbortController()
|
||||
controller.abort('too-late')
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('code run failed (abort)')
|
||||
expect(calls).toEqual([])
|
||||
})
|
||||
|
||||
it('rejects a binding invoked after the run is over without dispatching it', async () => {
|
||||
const { ctx, runtime } = await setup({ mode: 'code' })
|
||||
const calls = registerEcho(ctx)
|
||||
const controller = new AbortController()
|
||||
runtime.behavior = async (request) => {
|
||||
controller.abort('cancelled-mid-run')
|
||||
const message = await request.bindings[0]!.functions.echo!({ value: 'x' })
|
||||
.then(() => 'resolved', (error: unknown) => error instanceof Error ? error.message : String(error))
|
||||
return { logs: [], value: message }
|
||||
}
|
||||
const result = await runCode(ctx, 'program', { signal: controller.signal })
|
||||
expect(result.isError).toBe(false)
|
||||
expect((result.content[0] as { text: string }).text).toContain('not dispatched')
|
||||
expect(calls).toEqual([])
|
||||
})
|
||||
|
||||
it('a tool/code-dispatch event never derives a model message', () => {
|
||||
const session = new Session(SessionId('code-mode-derive'))
|
||||
session.append('user/message', { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, { surfaceOp: 'append' })
|
||||
session.append('tool/code-dispatch', {
|
||||
parentCallId: CallId('p1'),
|
||||
subCallId: CallId('p1:code:1'),
|
||||
name: 'echo',
|
||||
arguments: { value: 'x' },
|
||||
isError: false,
|
||||
resultSummary: 'echo:x',
|
||||
})
|
||||
const derived = session.deriveMessages()
|
||||
expect(derived).toHaveLength(1)
|
||||
expect(derived[0]?.role).toBe('user')
|
||||
})
|
||||
|
||||
it('defaults to native mode under direct construction with no config', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt, {})
|
||||
const registry = new ToolRegistry(ctx)
|
||||
expect(registry.get(RUN_CODE_NAME)).toBeUndefined()
|
||||
const assembly = await ctx.systemPrompt.assemble()
|
||||
expect(assembly.sections.some(section => section.name === 'tools:sdk')).toBe(false)
|
||||
})
|
||||
})
|
||||
@@ -35,7 +35,7 @@ describe('gen-tool-catalog collectToolCatalog', () => {
|
||||
it('boots every shipped tool package and harvests its model-facing schemas', async () => {
|
||||
const catalog = await collectToolCatalog()
|
||||
const names = catalog.flatMap(entry => entry.schemas.map(s => s.name)).sort()
|
||||
expect(names).toEqual(['ask_user_question', 'bash', 'bash_kill', 'bash_output', 'cordis_inspect', 'cordis_mount', 'cordis_unmount', 'edit', 'read', 'skill', 'subagent', 'todo_write', 'web_fetch', 'web_search', 'write'])
|
||||
expect(names).toEqual(['ask_user_question', 'bash', 'bash_kill', 'bash_output', 'cordis_inspect', 'cordis_mount', 'cordis_unmount', 'edit', 'read', 'run_code', 'skill', 'subagent', 'todo_write', 'web_fetch', 'web_search', 'workflow', 'write'])
|
||||
// Every tool carries a JSON-Schema `parameters` object (what the model sees).
|
||||
for (const entry of catalog) {
|
||||
for (const schema of entry.schemas) {
|
||||
|
||||
124
packages/core/tools/tests/ts-types.spec.ts
Normal file
124
packages/core/tools/tests/ts-types.spec.ts
Normal file
@@ -0,0 +1,124 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { jsonSchemaToTs, renderToolsSdk } from '@deepseek-ai/dsh-tools/src/ts-types.ts'
|
||||
import { schemaSpecToJsonSchema } from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolSchema } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
describe('jsonSchemaToTs', () => {
|
||||
it('maps the defineTool DSL subset', () => {
|
||||
const cases: [unknown, string][] = [
|
||||
[{ type: 'string' }, 'string'],
|
||||
[{ type: 'number' }, 'number'],
|
||||
[{ type: 'boolean' }, 'boolean'],
|
||||
[{ type: 'string', enum: ['a', 'b'] }, '"a" | "b"'],
|
||||
[{ type: 'array', items: { type: 'number' } }, 'number[]'],
|
||||
[{ type: 'array', items: { type: 'string', enum: ['x', 'y'] } }, '("x" | "y")[]'],
|
||||
[{ type: 'array' }, 'unknown[]'],
|
||||
[{ type: 'object' }, 'Record<string, unknown>'],
|
||||
[{ type: 'object', properties: {} }, 'Record<string, unknown>'],
|
||||
]
|
||||
for (const [schema, expected] of cases) {
|
||||
expect(jsonSchemaToTs(schema), JSON.stringify(schema)).toBe(expected)
|
||||
}
|
||||
})
|
||||
|
||||
it('renders objects with required/optional keys, nested shapes, and per-property docs', () => {
|
||||
const schema = schemaSpecToJsonSchema({
|
||||
path: { type: 'string', required: true, description: 'Absolute file path' },
|
||||
limit: { type: 'number' },
|
||||
opts: {
|
||||
type: 'object',
|
||||
properties: { deep: { type: 'boolean', required: true } },
|
||||
},
|
||||
})
|
||||
expect(jsonSchemaToTs(schema)).toBe([
|
||||
'{',
|
||||
' /** Absolute file path */',
|
||||
' path: string;',
|
||||
' limit?: number;',
|
||||
' opts?: {',
|
||||
' deep: boolean;',
|
||||
' };',
|
||||
'}',
|
||||
].join('\n'))
|
||||
})
|
||||
|
||||
it('is total: unsupported or hostile constructs degrade to unknown, never throw', () => {
|
||||
const cases: unknown[] = [
|
||||
undefined,
|
||||
null,
|
||||
42,
|
||||
'string-schema',
|
||||
{},
|
||||
{ type: 'integer' },
|
||||
{ type: 'null' },
|
||||
{ oneOf: [{ type: 'string' }] },
|
||||
{ $ref: '#/defs/x' },
|
||||
{ type: 'object', properties: 7 },
|
||||
{ type: 'object', properties: { bad: { $ref: 'x' } } },
|
||||
{ type: 'string', enum: [1, 2] },
|
||||
{ type: 'string', enum: [] },
|
||||
]
|
||||
for (const schema of cases) {
|
||||
expect(() => jsonSchemaToTs(schema), JSON.stringify(schema)).not.toThrow()
|
||||
}
|
||||
expect(jsonSchemaToTs({ type: 'integer' })).toBe('unknown')
|
||||
expect(jsonSchemaToTs({ oneOf: [] })).toBe('unknown')
|
||||
expect(jsonSchemaToTs({ type: 'object', properties: 7 })).toBe('Record<string, unknown>')
|
||||
expect(jsonSchemaToTs({ type: 'object', properties: { bad: { $ref: 'x' } }, required: ['bad'] })).toContain('bad: unknown;')
|
||||
// A non-string-only enum degrades to plain string; an empty one too.
|
||||
expect(jsonSchemaToTs({ type: 'string', enum: [1, 2] })).toBe('string')
|
||||
expect(jsonSchemaToTs({ type: 'string', enum: [] })).toBe('string')
|
||||
// A hostile required list only accepts string members.
|
||||
expect(jsonSchemaToTs({ type: 'object', properties: { a: { type: 'string' } }, required: [7] })).toContain('a?: string;')
|
||||
// A property VALUE that is not an object degrades to unknown (and can
|
||||
// carry no description).
|
||||
expect(jsonSchemaToTs({ type: 'object', properties: { weird: 42 } })).toContain('weird?: unknown;')
|
||||
})
|
||||
|
||||
it('escapes a comment-closer inside a description so the generated JSDoc cannot end early', () => {
|
||||
const rendered = jsonSchemaToTs({
|
||||
type: 'object',
|
||||
properties: { glob: { type: 'string', description: 'a pattern like packages/*/tool-*/ over here' } },
|
||||
})
|
||||
expect(rendered).not.toContain('tool-*/ over')
|
||||
expect(rendered).toContain(String.raw`tool-*\/ over`)
|
||||
})
|
||||
})
|
||||
|
||||
describe('renderToolsSdk', () => {
|
||||
const bash: ToolSchema = {
|
||||
name: 'bash',
|
||||
description: 'Run a shell command.',
|
||||
parameters: schemaSpecToJsonSchema({ command: { type: 'string', required: true } }) as unknown as Record<string, unknown>,
|
||||
}
|
||||
const exotic: ToolSchema = {
|
||||
name: 'my-mcp.tool',
|
||||
description: 'Exotic name.',
|
||||
parameters: schemaSpecToJsonSchema({}) as unknown as Record<string, unknown>,
|
||||
}
|
||||
|
||||
it('declares every tool in lexicographic order with quoted keys for exotic names', () => {
|
||||
const text = renderToolsSdk([exotic, bash])
|
||||
expect(text).toContain('declare const tools: {')
|
||||
expect(text.indexOf('bash(args:')).toBeGreaterThan(0)
|
||||
expect(text).toContain('"my-mcp.tool"(args:')
|
||||
expect(text.indexOf('bash(args:')).toBeLessThan(text.indexOf('"my-mcp.tool"(args:'))
|
||||
expect(text).toContain('): Promise<string>;')
|
||||
expect(text).toContain('/** Run a shell command. */')
|
||||
// The fixed instruction lines the model relies on.
|
||||
expect(text).toContain('erasable syntax only')
|
||||
expect(text).toContain('rejects with an `Error`')
|
||||
expect(text).toContain('sequentially, even under `Promise.all`')
|
||||
expect(text).toContain('JSON-serializable')
|
||||
})
|
||||
|
||||
it('is deterministic: same tool set, byte-identical text regardless of input order', () => {
|
||||
expect(renderToolsSdk([bash, exotic])).toBe(renderToolsSdk([exotic, bash]))
|
||||
// Equal names sort stably (the comparator's equal arm).
|
||||
expect(renderToolsSdk([bash, bash])).toBe(renderToolsSdk([bash, bash]))
|
||||
})
|
||||
|
||||
it('renders an empty declaration for an empty tool set', () => {
|
||||
expect(renderToolsSdk([])).toContain('declare const tools: {}')
|
||||
})
|
||||
})
|
||||
@@ -8,6 +8,12 @@
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../code-runtime/code-runtime"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
|
||||
@@ -37,7 +37,7 @@ The hooks **themselves** run in the agent's session workspace: for the agent-sco
|
||||
| `SessionStart` | `agent/session-start` (emit) | additionalContext → `agent.inject()` into the new session (cannot block) |
|
||||
| `UserPromptSubmit` | `agent/prompt-submit` (waterfall) | `deny` → `PromptDecision.block`; additionalContext-only → delegate via `next()` then fold context onto the downstream decision (a later listener can still block/rewrite) |
|
||||
| `PreToolUse` | `tools/pre-execute` (waterfall) | `deny` → `PreToolDecision.deny`; `ask` → `PreToolDecision.ask` |
|
||||
| `PostToolUse` | `tools/post-execute` (waterfall) | `deny` → `block` with feedback; additionalContext-only → delegate via `next()` then fold context onto the downstream decision |
|
||||
| `PostToolUse` | `tools/post-execute` (waterfall) | `deny` → `block` with feedback; additionalContext-only → delegate via `next()` then fold context onto the downstream decision (a Code Mode sub-call’s context is dropped by the run_code bridge — see [the pipeline doc](../../../docs/tool-execution-pipeline.md)) |
|
||||
| `Stop` | `agent/turn-continuation` (waterfall) | a blocking Stop hook forces `continue`, feeding its reason as next-step steering |
|
||||
| `SubagentStart` | `subagent/start` (emit) | additionalContext → `agent.inject()` into the live child |
|
||||
| `SubagentStop` | `subagent/end` (emit) | observe-only |
|
||||
|
||||
@@ -43,7 +43,7 @@ The hooks themselves run in the agent's session workspace: for the agent-scoped
|
||||
| `SessionStart` | `agent/session-start` (emit) | a plain-stdout hook's output → additionalContext → `agent.inject()` |
|
||||
| `UserPromptSubmit` | `agent/prompt-submit` (waterfall) | `block` (exit 2) → `PromptDecision.block`; additionalContext-only → delegate via `next()` then fold context onto the downstream decision |
|
||||
| `PreToolUse` | `tools/pre-execute` (waterfall) | `block` → `PreToolDecision.deny` (no `allow`/`ask`) |
|
||||
| `PostToolUse` | `tools/post-execute` (waterfall) | `block` → `block` with feedback; additionalContext-only → delegate via `next()` then fold context onto the downstream decision |
|
||||
| `PostToolUse` | `tools/post-execute` (waterfall) | `block` → `block` with feedback; additionalContext-only → delegate via `next()` then fold context onto the downstream decision (a Code Mode sub-call’s context is dropped by the run_code bridge — see [the pipeline doc](../../../docs/tool-execution-pipeline.md)) |
|
||||
| `Stop` | `agent/turn-continuation` (waterfall) | a blocking Stop hook forces `continue` with the reason as next-step steering |
|
||||
|
||||
A tool call's payload carries the real `tool_name` (the same value the matcher tests) and Codex's `tool_input: { command }` shape (the `command` arg when present, else `''`). The matcher subject is the tool name (`PreToolUse`/`PostToolUse`) or the session source (`SessionStart`); `UserPromptSubmit`/`Stop` ignore matchers.
|
||||
|
||||
@@ -8,9 +8,10 @@ The subagent seam: an agent delegating work to a child agent. Like the [bash](..
|
||||
| `subagent-inprocess/` | Shared in-process run driver (pure lib; registers nothing) | — |
|
||||
| `subagent-spawn/` | In-process backend: a fresh child agent | (registers on `ctx.subagents`) |
|
||||
| `subagent-fork/` | In-process backend: a child seeded with the parent's completed-turn prefix | (registers on `ctx.subagents`) |
|
||||
| `subagent-subprocess/` | Shared out-of-process machinery: env scrub, dispose ladder, isolated config dirs (pure lib; registers nothing) | — |
|
||||
| `subagent-acp/` | Out-of-process backend: a child agent in a spawned subprocess, driven over ACP | (registers on `ctx.subagents`) |
|
||||
| `tool-subagent/` | Model-facing `subagent` delegation tool over `ctx.subagents` | (registers on `ctx.tools`) |
|
||||
|
||||
The interface lives at `subagent/subagent/`. The in-process `subagent-spawn` / `subagent-fork` backends share the `subagent-inprocess` driver (a pure library — both depend on it, neither on the other), and the out-of-process `subagent-acp` backend ships alongside them here; the test-only `dsh-subagent-mock` (in [support](../support/README.md)) is separate. All **product** packages except the mock.
|
||||
The interface lives at `subagent/subagent/`. The in-process `subagent-spawn` / `subagent-fork` backends share the `subagent-inprocess` driver (a pure library — both depend on it, neither on the other), the out-of-process `subagent-acp` backend builds on the `subagent-subprocess` library (the credential env scrub, the dispose ladder, isolated config dirs) and ships alongside them here; the test-only `dsh-subagent-mock` (in [support](../support/README.md)) is separate. All **product** packages except the mock.
|
||||
|
||||
The proposal and design rationale: [docs/rfc/implemented/feature/2026-06-21-subagent-capability-seam.md](../../docs/rfc/implemented/feature/2026-06-21-subagent-capability-seam.md).
|
||||
|
||||
@@ -57,7 +57,7 @@ A spawn/transport/RPC failure resolves `error` (or `aborted` if a cancel was req
|
||||
|
||||
## Environment scrub
|
||||
|
||||
Credential-shaped ambient vars (`/KEY|SECRET|TOKEN/i`) are NOT forwarded to the child by default — the parent harness's own secrets must not leak into a spawned process implicitly. The child's OWN credentials are supplied explicitly via `config.env`, layered AFTER the scrub, so an intended `DEEPSEEK_API_KEY` survives while an incidental `AWS_SECRET_ACCESS_KEY` does not.
|
||||
The child env is built by [`buildChildEnv` from `@deepseek-ai/dsh-subagent-subprocess`](../subagent-subprocess/README.md) — the ambient env minus credential-shaped vars, with `config.env` layered on top after the scrub; the pattern and full semantics live there. For this backend that means the parent harness's own secrets never leak into the spawned agent implicitly, while the child's OWN `DEEPSEEK_API_KEY` is supplied deliberately via `config.env` and survives.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-subagent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-subagent-subprocess": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
@@ -35,6 +36,7 @@
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent-subprocess": "workspace:^",
|
||||
"@cordisjs/plugin-loader": "^1.0.0-rc.4",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@
|
||||
* @module @deepseek-ai/dsh-subagent-acp/run
|
||||
*/
|
||||
|
||||
import { spawn, type ChildProcess } from 'node:child_process'
|
||||
import { spawn } from 'node:child_process'
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { Readable, Writable } from 'node:stream'
|
||||
import {
|
||||
@@ -40,6 +40,7 @@ import {
|
||||
import { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { SubagentResult, SubagentRun, SubagentStartRequest, SubagentStopReason } from '@deepseek-ai/dsh-subagent'
|
||||
import { buildChildEnv, disposeChildProcess, spawnFailure } from '@deepseek-ai/dsh-subagent-subprocess'
|
||||
|
||||
/**
|
||||
* How the client answers a child's `session/request_permission`. The first cut
|
||||
@@ -110,31 +111,6 @@ export const DEFAULT_DISPOSE_EOF_GRACE_MS = 6_000
|
||||
/** Default grace between SIGTERM and SIGKILL on dispose (the `disposeGraceMs` config; mirrors the bash executor). */
|
||||
export const DEFAULT_DISPOSE_GRACE_MS = 3_000
|
||||
|
||||
/**
|
||||
* Credential-shaped ambient env vars are NOT forwarded to the child by default
|
||||
* (the parent harness's own `DEEPSEEK_API_KEY`/secrets must not leak into a
|
||||
* spawned process implicitly). Same pattern as the bash executor. The child
|
||||
* agent needs its OWN credentials to reach a model — those are supplied
|
||||
* explicitly via {@link AcpRunSpec.env}, which is layered on top AFTER the
|
||||
* scrub, so an intended `DEEPSEEK_API_KEY` survives while an incidental
|
||||
* `AWS_SECRET_ACCESS_KEY` does not.
|
||||
*/
|
||||
export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i
|
||||
|
||||
/**
|
||||
* The ambient env minus credential-shaped vars, plus the spec's explicit env.
|
||||
* @param extra - explicit vars layered on top AFTER the scrub, so a
|
||||
* credential-shaped name supplied deliberately still reaches the child.
|
||||
* @returns the environment to spawn the child with.
|
||||
*/
|
||||
export function buildChildEnv(extra: Record<string, string>): NodeJS.ProcessEnv {
|
||||
const env: NodeJS.ProcessEnv = {}
|
||||
for (const [key, value] of Object.entries(process.env)) {
|
||||
if (!SENSITIVE_ENV_PATTERN.test(key)) env[key] = value
|
||||
}
|
||||
return { ...env, ...extra }
|
||||
}
|
||||
|
||||
/**
|
||||
* Map an ACP {@link StopReason} to a harness {@link SubagentStopReason}.
|
||||
* @param reason - the terminal reason from the child's `session/prompt` response.
|
||||
@@ -196,24 +172,6 @@ function toError(value: unknown): Error {
|
||||
return value instanceof Error ? value : new Error(String(value))
|
||||
}
|
||||
|
||||
/** Resolve once the child process exits (any code/signal); immediate if gone. */
|
||||
function waitForExit(child: ChildProcess): Promise<void> {
|
||||
// Already-exited fast path: dispose guards on exitCode before calling, so in
|
||||
// tests the child is always still alive here.
|
||||
/* v8 ignore next */
|
||||
if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve()
|
||||
return new Promise<void>(resolve => child.once('exit', () => { resolve() }))
|
||||
}
|
||||
|
||||
/** Resolve `true` if the child exits within `ms`, `false` on timeout. */
|
||||
function exitsWithin(child: ChildProcess, ms: number): Promise<boolean> {
|
||||
return Promise.race([
|
||||
waitForExit(child).then(() => true),
|
||||
// `.unref()` so a pending grace timer never keeps the parent's loop alive.
|
||||
new Promise<boolean>(resolve => setTimeout(() => { resolve(false) }, ms).unref()),
|
||||
])
|
||||
}
|
||||
|
||||
/**
|
||||
* Start an out-of-process ACP child for `request` and return a {@link SubagentRun}.
|
||||
*
|
||||
@@ -254,13 +212,11 @@ export function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpec): Su
|
||||
env: buildChildEnv(spec.env),
|
||||
stdio: ['pipe', 'pipe', 'inherit'],
|
||||
})
|
||||
// A spawn-level failure (e.g. ENOENT for a bad command) is emitted as an
|
||||
// `error` event, NOT a thrown exception — without a listener Node treats it as
|
||||
// an unhandled error and crashes the parent. Capture it into a promise the
|
||||
// result path races, so a bad command settles `error` like any child failure.
|
||||
const spawnFailed = new Promise<Error>((resolve) => {
|
||||
child.once('error', (err) => { resolve(err) })
|
||||
})
|
||||
// Same-tick capture (the library's contract): a spawn-level failure (e.g.
|
||||
// ENOENT for a bad command) is an `error` EVENT that would crash the parent
|
||||
// unheard; the result path races this promise, so a bad command settles
|
||||
// `error` like any child failure.
|
||||
const spawnFailed = spawnFailure(child)
|
||||
|
||||
// Accumulate the child's streamed assistant text — the SubagentResult output.
|
||||
const output: string[] = []
|
||||
@@ -393,33 +349,19 @@ export function startAcpRun(request: SubagentStartRequest, spec: AcpRunSpec): Su
|
||||
},
|
||||
async dispose(): Promise<void> {
|
||||
request.signal?.removeEventListener('abort', onAbort)
|
||||
// Reach quiescence, not merely request it (dispose must AWAIT the child
|
||||
// actually stopping). If the child is already gone, nothing to do.
|
||||
if (child.exitCode !== null || child.signalCode !== null) return
|
||||
const eofGraceMs = spec.disposeEofGraceMs
|
||||
const graceMs = spec.disposeGraceMs
|
||||
// 1. Graceful: end the ACP request stream (stdin EOF) and let the child
|
||||
// quiesce ON ITS OWN. Our acp-agent has NO SIGTERM handler in a normal
|
||||
// session — it tears down via the server bridge's connection-close path
|
||||
// (conn.closed → per-agent dispose → final session/flush), driven by the
|
||||
// stdin EOF, NOT by a signal. A prompt response can resolve from a
|
||||
// turn/end BEFORE that post-turn flush lands, so the child still has
|
||||
// durable work owed when dispose runs. Give the EOF-driven quiesce a real
|
||||
// window — wider than a single signal-grace, since the child's own
|
||||
// teardown may itself be awaiting a signal-trapping grandchild (a bash
|
||||
// subprocess in its own SIGTERM→SIGKILL grace) plus a flush — and only
|
||||
// escalate if it overruns. Sending SIGTERM in the same tick (or too soon)
|
||||
// would default-terminate the child mid-flush, orphaning its nested work.
|
||||
child.stdin.end()
|
||||
if (await exitsWithin(child, eofGraceMs)) return
|
||||
// 2. SIGTERM, then escalate to SIGKILL if it still does not exit within the
|
||||
// grace period — a child that ignores EOF and traps SIGTERM must not
|
||||
// wedge dispose forever (the seam requires bounded quiescence).
|
||||
child.kill('SIGTERM')
|
||||
if (await exitsWithin(child, graceMs)) return
|
||||
// 3. Force-kill and await the (now-certain) exit.
|
||||
child.kill('SIGKILL')
|
||||
await waitForExit(child)
|
||||
// Quiescent teardown via the shared ladder (stdin EOF → SIGTERM →
|
||||
// SIGKILL, awaiting the actual exit). For THIS child the EOF tier is the
|
||||
// one that matters: our acp-agent has NO SIGTERM handler in a normal
|
||||
// session — it tears down via the server bridge's connection-close path
|
||||
// (conn.closed → per-agent dispose → final session/flush), driven by the
|
||||
// stdin EOF, NOT by a signal — and a prompt response can resolve from a
|
||||
// turn/end BEFORE that post-turn flush lands, so the child still has
|
||||
// durable work owed when dispose runs (hence the wide EOF grace; see
|
||||
// DEFAULT_DISPOSE_EOF_GRACE_MS).
|
||||
await disposeChildProcess(child, {
|
||||
disposeEofGraceMs: spec.disposeEofGraceMs,
|
||||
disposeGraceMs: spec.disposeGraceMs,
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,9 +6,10 @@ import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import { buildChildEnv, SENSITIVE_ENV_PATTERN } from '@deepseek-ai/dsh-subagent-subprocess'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import * as acp from '../src/index.ts'
|
||||
import { acpStopReason, acpContentText, buildChildEnv, DEFAULT_DISPOSE_EOF_GRACE_MS, DEFAULT_DISPOSE_GRACE_MS, SENSITIVE_ENV_PATTERN, startAcpRun, toAcpPrompt, type AcpRunSpec } from '../src/run.ts'
|
||||
import { acpStopReason, acpContentText, DEFAULT_DISPOSE_EOF_GRACE_MS, DEFAULT_DISPOSE_GRACE_MS, startAcpRun, toAcpPrompt, type AcpRunSpec } from '../src/run.ts'
|
||||
|
||||
/**
|
||||
* Keyless integration tests for the ACP subagent backend. Each spawns a REAL
|
||||
|
||||
@@ -25,6 +25,9 @@
|
||||
},
|
||||
{
|
||||
"path": "../subagent"
|
||||
},
|
||||
{
|
||||
"path": "../subagent-subprocess"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
40
packages/subagent/subagent-subprocess/README.md
Normal file
40
packages/subagent/subagent-subprocess/README.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# @deepseek-ai/dsh-subagent-subprocess
|
||||
|
||||
Shared machinery for **out-of-process subagent backends** — providers that spawn an external agent as a child process, such as the [ACP backend](../subagent-acp/README.md). A pure library (no provider, no registration, no Config): what every spawn-a-CLI-child backend needs to keep the parent deployment's credentials out of the child, tear the child down to quiescence, and isolate it from the host user's on-disk CLI state. Design rationale: [the Claude Code / Codex subagent backends RFC](../../../docs/rfc/proposed/feature/2026-07-07-claude-code-and-codex-subagent-backends.md).
|
||||
|
||||
Every tunable is a **parameter**: the dispose ladder takes its grace periods per call, the config-dir helper takes an optional pinned path. Defaults live in each consuming plugin's Config (defaulted, validated fields changeable from `cordis.yml`), never in this library.
|
||||
|
||||
## What it exports
|
||||
|
||||
### `SENSITIVE_ENV_PATTERN` / `buildChildEnv(extra)`
|
||||
|
||||
The credential env scrub (same pattern as the [bash executor](../../bash/bash-local/README.md)): the child env is the ambient env minus credential-shaped vars (`/KEY|SECRET|TOKEN/i`), with `extra` layered on top AFTER the scrub. `PATH`, `HOME`, `TMPDIR`, locale, and proxy vars survive, so the child CLI runs normally; the parent's own secrets never leak implicitly, while an explicitly supplied credential (the child's OWN key in a backend's `env` config) still reaches the child.
|
||||
|
||||
### `spawnFailure(child)`
|
||||
|
||||
Spawn-failure capture: a promise that resolves (never rejects) with the child's first `error` event. A spawn failure such as `ENOENT` is an event, not a thrown exception — without a listener Node crashes the parent process — so call this in the same tick as `spawn()` and race it in the run's result path; a bad command then settles as an ordinary child-level failure. For a child that spawns cleanly the promise never settles.
|
||||
|
||||
### `waitForExit(child)` / `exitsWithin(child, ms)`
|
||||
|
||||
Exit waits over a `ChildProcess`: resolve once the child exits by any code or signal (immediately if it is already gone), or race that against a timer (`true` = exited in time). The race cleans up after itself on both outcomes — the pending timer is `unref()`ed and cleared on exit, the exit listener removed on timeout — so repeated calls (the dispose ladder's tiers, a poll loop) never accumulate listeners on the child.
|
||||
|
||||
### `disposeChildProcess(child, graces)`
|
||||
|
||||
The three-tier dispose ladder. Resolves only once the child has ACTUALLY exited — quiescence reached, not merely requested (see [defensive patterns](../../../docs/defensive-patterns.md)):
|
||||
|
||||
1. stdin EOF (when stdin is piped), then wait `graces.disposeEofGraceMs` — a cooperative child quiesces on its own, its flushes and nested-subprocess teardown intact;
|
||||
2. `SIGTERM`, then wait `graces.disposeGraceMs`;
|
||||
3. `SIGKILL`, then await the now-certain exit — a child that ignores EOF and traps `SIGTERM` cannot wedge dispose forever.
|
||||
|
||||
The two graces (`DisposeLadderGraces`) come from the consuming plugin's `disposeEofGraceMs`/`disposeGraceMs` Config fields; the EOF window is deliberately a separate — usually wider — grace than the signal tier, since a cooperative child's EOF teardown may itself await a signal-trapping grandchild plus a final flush.
|
||||
|
||||
### `createIsolatedConfigDir(prefix, pinnedPath?)`
|
||||
|
||||
A per-run isolated config directory for an external CLI child (the target of `CLAUDE_CONFIG_DIR` / `CODEX_HOME`-style redirection), so child behavior is a function of deployment config alone — never of whatever `~/.claude` / `~/.codex`-style state exists on the host. Returns an `IsolatedConfigDir` handle: `path` goes into the child env, `remove()` runs on dispose.
|
||||
|
||||
- **Fresh (default)**: a private (0700) `mkdtemp` dir under the OS temp root; `remove()` deletes it best-effort (never rejects — a leftover temp dir beats a failed dispose) and is idempotent.
|
||||
- **Pinned** (`pinnedPath` set): the path is returned as-is — never created, never removed. A deployment that pins a directory to share child state across runs owns that directory's lifecycle.
|
||||
|
||||
## Testing
|
||||
|
||||
`tests/subagent-subprocess.spec.ts`: the env scrub and config-dir helpers run against the real process env and real filesystem (the rm-failure path injects its rejection at the fs boundary — a real recursive-rm failure is not portably provokable, and root ignores permission bits); the exit waits and the dispose ladder run against a scriptable fake child, driving each escalation tier deterministically. The [ACP backend suite](../subagent-acp/README.md) exercises the same ladder against real subprocesses (EOF-cooperative, EOF-ignoring, and SIGTERM-trapping children) end to end.
|
||||
30
packages/subagent/subagent-subprocess/package.json
Normal file
30
packages/subagent/subagent-subprocess/package.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-subagent-subprocess",
|
||||
"description": "Shared out-of-process subagent machinery: credential env scrub, spawn-failure capture, child-exit waits, the EOF-to-SIGTERM-to-SIGKILL dispose ladder, and isolated config dirs (pure lib; registers nothing)",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
219
packages/subagent/subagent-subprocess/src/index.ts
Normal file
219
packages/subagent/subagent-subprocess/src/index.ts
Normal file
@@ -0,0 +1,219 @@
|
||||
/**
|
||||
* Shared machinery for OUT-OF-PROCESS subagent backends — providers that spawn
|
||||
* an external agent as a child process and must keep the parent deployment's
|
||||
* credentials out of it, tear it down to quiescence, and isolate it from the
|
||||
* host user's on-disk CLI state. The pieces: the credential env scrub
|
||||
* ({@link SENSITIVE_ENV_PATTERN} / {@link buildChildEnv}), the spawn-failure
|
||||
* capture ({@link spawnFailure}), the child-exit waits ({@link waitForExit} /
|
||||
* {@link exitsWithin}), the stdin-EOF → SIGTERM → SIGKILL dispose ladder
|
||||
* ({@link disposeChildProcess}), and the per-run isolated config dir
|
||||
* ({@link createIsolatedConfigDir}).
|
||||
*
|
||||
* This package owns no provider and registers nothing; it is a pure library
|
||||
* the out-of-process backend packages depend on (the `subagent-inprocess`
|
||||
* shape, for the process boundary). Every tunable — the ladder's grace
|
||||
* periods, a pinned config dir — is a PARAMETER here: defaults belong in each
|
||||
* consuming plugin's Config, per the no-hardcoded-tunables rule.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-subagent-subprocess
|
||||
*/
|
||||
|
||||
import type { ChildProcess } from 'node:child_process'
|
||||
import { mkdtemp, rm } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
/**
|
||||
* Credential-shaped ambient env vars are NOT forwarded to a child by default
|
||||
* (the parent harness's own `DEEPSEEK_API_KEY`/secrets must not leak into a
|
||||
* spawned process implicitly). Same pattern as the bash executor. The child
|
||||
* agent needs its OWN credentials to reach a model — those are supplied
|
||||
* explicitly via the `extra` layer of {@link buildChildEnv}, which lands AFTER
|
||||
* the scrub, so an intended `DEEPSEEK_API_KEY` survives while an incidental
|
||||
* `AWS_SECRET_ACCESS_KEY` does not.
|
||||
*/
|
||||
export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i
|
||||
|
||||
/**
|
||||
* The ambient env minus credential-shaped vars, plus the caller's explicit
|
||||
* env. `PATH`, `HOME`, `TMPDIR`, locale, and proxy vars survive the scrub, so
|
||||
* a child CLI runs normally; only {@link SENSITIVE_ENV_PATTERN}-shaped names
|
||||
* are dropped.
|
||||
* @param extra - explicit vars layered on top AFTER the scrub, so a
|
||||
* credential-shaped name supplied deliberately still reaches the child.
|
||||
* @returns the environment to spawn the child with.
|
||||
*/
|
||||
export function buildChildEnv(extra: Record<string, string>): NodeJS.ProcessEnv {
|
||||
const env: NodeJS.ProcessEnv = {}
|
||||
for (const [key, value] of Object.entries(process.env)) {
|
||||
if (!SENSITIVE_ENV_PATTERN.test(key)) env[key] = value
|
||||
}
|
||||
return { ...env, ...extra }
|
||||
}
|
||||
|
||||
/**
|
||||
* Capture the child's spawn-level failure as a promise the run's result path
|
||||
* can race. A spawn failure (e.g. `ENOENT` for a bad command) is emitted as an
|
||||
* `error` EVENT, not a thrown exception — and without a listener Node treats
|
||||
* it as an unhandled error and crashes the parent process. Call this in the
|
||||
* SAME TICK as `spawn()`, so no window exists for the event to fire unheard.
|
||||
* @param child - the just-spawned child process.
|
||||
* @returns a promise that RESOLVES (never rejects) with the child's first
|
||||
* `error` event; for a child that spawns cleanly it never settles.
|
||||
*/
|
||||
export function spawnFailure(child: ChildProcess): Promise<Error> {
|
||||
return new Promise<Error>((resolve) => {
|
||||
child.once('error', (err) => { resolve(err) })
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve once the child process exits (any code/signal); immediate if it is
|
||||
* already gone.
|
||||
* @param child - the child process to await.
|
||||
*/
|
||||
export function waitForExit(child: ChildProcess): Promise<void> {
|
||||
if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve()
|
||||
return new Promise<void>(resolve => child.once('exit', () => { resolve() }))
|
||||
}
|
||||
|
||||
/**
|
||||
* Race the child's exit against a timer. Neither outcome leaves anything
|
||||
* behind on the child: the exit listener is removed on timeout and the timer
|
||||
* is cleared on exit, so repeated calls (the dispose ladder's tiers, a poll
|
||||
* loop) never accumulate listeners.
|
||||
* @param child - the child process to watch.
|
||||
* @param ms - the wait window in milliseconds.
|
||||
* @returns `true` if the child exits within `ms` (immediately if it is
|
||||
* already gone), `false` on timeout.
|
||||
*/
|
||||
export function exitsWithin(child: ChildProcess, ms: number): Promise<boolean> {
|
||||
if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve(true)
|
||||
return new Promise<boolean>((resolve) => {
|
||||
const onExit = (): void => {
|
||||
clearTimeout(timer)
|
||||
resolve(true)
|
||||
}
|
||||
// `.unref()` so a pending grace timer never keeps the parent's loop alive.
|
||||
const timer = setTimeout(() => {
|
||||
child.removeListener('exit', onExit)
|
||||
resolve(false)
|
||||
}, ms).unref()
|
||||
child.once('exit', onExit)
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* The two grace periods of the dispose ladder, supplied per call by the
|
||||
* consuming backend — each plugin carries them as defaulted, validated
|
||||
* `disposeEofGraceMs`/`disposeGraceMs` Config fields, so teardown timing is
|
||||
* deployment-tunable and this library hardcodes nothing.
|
||||
*/
|
||||
export interface DisposeLadderGraces {
|
||||
/**
|
||||
* Tier-1 window (ms): after stdin EOF, how long the child gets to quiesce
|
||||
* ON ITS OWN — flush durable state, tear down its own nested subprocesses —
|
||||
* before the parent escalates to `SIGTERM`. A separate (usually WIDER)
|
||||
* grace than {@link DisposeLadderGraces.disposeGraceMs}: a cooperative
|
||||
* child's EOF-driven teardown may itself be waiting on a signal-trapping
|
||||
* grandchild plus a final flush, needing more than one signal-grace of
|
||||
* headroom.
|
||||
*/
|
||||
disposeEofGraceMs: number
|
||||
/** Tier-2 window (ms): between `SIGTERM` and the `SIGKILL` escalation. */
|
||||
disposeGraceMs: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Tear a child process down to QUIESCENCE: resolves only once the child has
|
||||
* actually exited (or was already gone), never merely after requesting it.
|
||||
* Three-tier escalation —
|
||||
*
|
||||
* 1. stdin EOF (when stdin is piped), then wait `disposeEofGraceMs`: a
|
||||
* cooperative child quiesces on its own, its teardown and flushes intact;
|
||||
* 2. `SIGTERM`, then wait `disposeGraceMs`;
|
||||
* 3. `SIGKILL`, then await the (now-certain) exit — a child that ignores EOF
|
||||
* and traps `SIGTERM` must not wedge dispose forever.
|
||||
*
|
||||
* @param child - the child process to tear down.
|
||||
* @param graces - the two grace periods, from the consuming plugin's Config.
|
||||
*/
|
||||
export async function disposeChildProcess(child: ChildProcess, graces: DisposeLadderGraces): Promise<void> {
|
||||
// Already gone: nothing to reap.
|
||||
if (child.exitCode !== null || child.signalCode !== null) return
|
||||
// 1. Graceful: end the request stream (stdin EOF) and let the child quiesce
|
||||
// on its own. Sending SIGTERM in the same tick (or too soon) would
|
||||
// default-terminate a cooperative child mid-flush, orphaning its nested
|
||||
// work. A child spawned without a stdin pipe skips straight to the wait.
|
||||
child.stdin?.end()
|
||||
if (await exitsWithin(child, graces.disposeEofGraceMs)) return
|
||||
// 2. SIGTERM, escalating if the child still does not exit within the grace.
|
||||
child.kill('SIGTERM')
|
||||
if (await exitsWithin(child, graces.disposeGraceMs)) return
|
||||
// 3. Force-kill and await the (now-certain) exit.
|
||||
child.kill('SIGKILL')
|
||||
await waitForExit(child)
|
||||
}
|
||||
|
||||
/**
|
||||
* A per-run config directory handle for an external CLI child — the target of
|
||||
* `CLAUDE_CONFIG_DIR` / `CODEX_HOME`-style redirection. Hand {@link path} to
|
||||
* the child's environment; call {@link remove} on dispose.
|
||||
*/
|
||||
export interface IsolatedConfigDir {
|
||||
/** The directory to point the child at. */
|
||||
path: string
|
||||
/**
|
||||
* Best-effort cleanup: removes the directory (recursively) iff this handle
|
||||
* CREATED it — a pinned directory is never removed. Idempotent; never
|
||||
* rejects (a leftover dir under the OS temp root is preferable to a failed
|
||||
* dispose).
|
||||
*/
|
||||
remove(): Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* An isolated config dir for one child run, so the child's behavior is a
|
||||
* function of deployment config alone — never of whatever `~/.claude` /
|
||||
* `~/.codex`-style state happens to exist on the host machine. Two modes:
|
||||
*
|
||||
* - no `pinnedPath` (the default): creates a FRESH private (0700) `mkdtemp`
|
||||
* dir under the OS temp root; {@link IsolatedConfigDir.remove} deletes it
|
||||
* best-effort;
|
||||
* - `pinnedPath` set (a deployment deliberately sharing state across runs):
|
||||
* the pinned path is returned as-is — never created, never removed — the
|
||||
* deployment owns that directory's lifecycle.
|
||||
*
|
||||
* @param prefix - the `mkdtemp` name prefix for a fresh dir (e.g.
|
||||
* `dsh-subagent-codex-`); ignored when `pinnedPath` is set.
|
||||
* @param pinnedPath - a deployment-pinned directory to use instead of a
|
||||
* fresh one.
|
||||
* @returns the directory handle: `path` for the child env, `remove()` for
|
||||
* dispose.
|
||||
*/
|
||||
export async function createIsolatedConfigDir(prefix: string, pinnedPath?: string): Promise<IsolatedConfigDir> {
|
||||
if (pinnedPath !== undefined) {
|
||||
return {
|
||||
path: pinnedPath,
|
||||
remove(): Promise<void> {
|
||||
// A pinned dir is deployment-owned state (config the user asked to
|
||||
// persist across runs); removing it here would destroy it. No-op.
|
||||
return Promise.resolve()
|
||||
},
|
||||
}
|
||||
}
|
||||
const path = await mkdtemp(join(tmpdir(), prefix))
|
||||
return {
|
||||
path,
|
||||
async remove(): Promise<void> {
|
||||
try {
|
||||
await rm(path, { recursive: true, force: true })
|
||||
} catch {
|
||||
// Best-effort by contract: swallows rm failures (EACCES/EBUSY-style —
|
||||
// e.g. the dead child left an unreadable entry behind). The dir lives
|
||||
// under the OS temp root, which reclaims it; failing dispose over
|
||||
// cleanup would be worse than a leftover temp dir.
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,326 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { EventEmitter } from 'node:events'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { mkdtemp, rm, stat, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import type { ChildProcess } from 'node:child_process'
|
||||
import {
|
||||
buildChildEnv,
|
||||
createIsolatedConfigDir,
|
||||
disposeChildProcess,
|
||||
exitsWithin,
|
||||
SENSITIVE_ENV_PATTERN,
|
||||
spawnFailure,
|
||||
waitForExit,
|
||||
} from '../src/index.ts'
|
||||
|
||||
// `rm` is wrapped (real-passthrough by default) so ONE test can inject a
|
||||
// rejection deterministically. A real recursive-rm failure is not portably
|
||||
// provokable — permission tricks (a chmod-000 subtree) fail only for
|
||||
// unprivileged users and are ignored by root — so this is the fs boundary
|
||||
// the testing policy sanctions mocking; everything else stays the real fs.
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>()
|
||||
return { ...actual, rm: vi.fn(actual.rm) }
|
||||
})
|
||||
|
||||
/**
|
||||
* Unit tests for the shared out-of-process machinery. The env scrub and the
|
||||
* isolated-config-dir helpers run against the REAL process env and REAL
|
||||
* filesystem (one exception: the rm-failure path injects its rejection at the
|
||||
* mocked fs boundary, see above); the exit waits and the dispose ladder run
|
||||
* against a scriptable fake child so each escalation tier's timing is driven
|
||||
* deterministically (the ACP backend's suite exercises the same ladder
|
||||
* against real subprocesses end to end).
|
||||
*/
|
||||
|
||||
/** What fells a scripted {@link FakeChild}. */
|
||||
type LethalTrigger = 'eof' | NodeJS.Signals
|
||||
|
||||
/** Per-scenario script for a {@link FakeChild}. */
|
||||
interface FakeChildScript {
|
||||
/**
|
||||
* The one trigger that makes the child exit (SIGKILL always does,
|
||||
* uncatchable, like a real process). Omitted: only SIGKILL fells it.
|
||||
*/
|
||||
diesOn?: LethalTrigger
|
||||
/** Delay (ms) between the lethal trigger and the exit event. */
|
||||
delayMs?: number
|
||||
/** `false` models a child spawned without a stdin pipe. */
|
||||
stdin?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* A scriptable stand-in for a ChildProcess carrying exactly the surface the
|
||||
* helpers read: `exitCode`/`signalCode`, `stdin.end()`, `kill()`, and the
|
||||
* `exit` event.
|
||||
*/
|
||||
class FakeChild extends EventEmitter {
|
||||
exitCode: number | null = null
|
||||
signalCode: NodeJS.Signals | null = null
|
||||
readonly kills: NodeJS.Signals[] = []
|
||||
stdinEnded = false
|
||||
readonly stdin: { end: () => void } | null
|
||||
|
||||
constructor(private readonly script: FakeChildScript = {}) {
|
||||
super()
|
||||
this.stdin = script.stdin === false
|
||||
? null
|
||||
: { end: () => { this.stdinEnded = true; this.maybeDie('eof') } }
|
||||
}
|
||||
|
||||
kill(signal: NodeJS.Signals): boolean {
|
||||
this.kills.push(signal)
|
||||
this.maybeDie(signal)
|
||||
return true
|
||||
}
|
||||
|
||||
private maybeDie(trigger: LethalTrigger): void {
|
||||
// SIGKILL is uncatchable — it always fells the child; any other trigger
|
||||
// only when the scenario scripts it as the lethal one.
|
||||
if (trigger !== 'SIGKILL' && this.script.diesOn !== trigger) return
|
||||
setTimeout(() => {
|
||||
if (trigger === 'eof') this.exitCode = 0
|
||||
else this.signalCode = trigger
|
||||
this.emit('exit', this.exitCode, this.signalCode)
|
||||
}, this.script.delayMs ?? 0)
|
||||
}
|
||||
}
|
||||
|
||||
/** The helpers take a real ChildProcess; the fake carries the read surface. */
|
||||
function asChild(fake: FakeChild): ChildProcess {
|
||||
return fake as unknown as ChildProcess
|
||||
}
|
||||
|
||||
describe('buildChildEnv / SENSITIVE_ENV_PATTERN', () => {
|
||||
it('drops credential-shaped ambient vars (KEY/SECRET/TOKEN, case-insensitive)', () => {
|
||||
process.env.DSH_PROC_TEST_API_KEY = 'leak'
|
||||
process.env.dsh_proc_test_secret = 'leak'
|
||||
process.env.DSH_PROC_TEST_TOKEN = 'leak'
|
||||
try {
|
||||
const env = buildChildEnv({})
|
||||
expect(env.DSH_PROC_TEST_API_KEY).toBeUndefined()
|
||||
expect(env.dsh_proc_test_secret).toBeUndefined()
|
||||
expect(env.DSH_PROC_TEST_TOKEN).toBeUndefined()
|
||||
} finally {
|
||||
delete process.env.DSH_PROC_TEST_API_KEY
|
||||
delete process.env.dsh_proc_test_secret
|
||||
delete process.env.DSH_PROC_TEST_TOKEN
|
||||
}
|
||||
})
|
||||
|
||||
it('forwards normal ambient vars', () => {
|
||||
expect(SENSITIVE_ENV_PATTERN.test('PATH')).toBe(false)
|
||||
expect(buildChildEnv({}).PATH).toBe(process.env.PATH)
|
||||
})
|
||||
|
||||
it('layers extras AFTER the scrub, so a deliberate credential-shaped name survives', () => {
|
||||
process.env.DSH_PROC_TEST_EXTRA_TOKEN = 'ambient-leak'
|
||||
try {
|
||||
const env = buildChildEnv({ DSH_PROC_TEST_EXTRA_TOKEN: 'explicit' })
|
||||
// The ambient value was scrubbed; ONLY the explicit opt-in reaches the child.
|
||||
expect(env.DSH_PROC_TEST_EXTRA_TOKEN).toBe('explicit')
|
||||
} finally {
|
||||
delete process.env.DSH_PROC_TEST_EXTRA_TOKEN
|
||||
}
|
||||
})
|
||||
|
||||
it('an extra overrides the ambient value of a non-credential var', () => {
|
||||
process.env.DSH_PROC_TEST_PLAIN = 'ambient'
|
||||
try {
|
||||
expect(buildChildEnv({ DSH_PROC_TEST_PLAIN: 'override' }).DSH_PROC_TEST_PLAIN).toBe('override')
|
||||
} finally {
|
||||
delete process.env.DSH_PROC_TEST_PLAIN
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('spawnFailure', () => {
|
||||
it('resolves (never rejects) with the first error event', async () => {
|
||||
const fake = new FakeChild()
|
||||
const failure = spawnFailure(asChild(fake))
|
||||
const err = new Error('spawn ENOENT')
|
||||
fake.emit('error', err)
|
||||
await expect(failure).resolves.toBe(err)
|
||||
})
|
||||
|
||||
it('never settles for a child that spawns cleanly and exits', async () => {
|
||||
const fake = new FakeChild({ diesOn: 'SIGTERM' })
|
||||
const failure = spawnFailure(asChild(fake))
|
||||
fake.kill('SIGTERM')
|
||||
await waitForExit(asChild(fake))
|
||||
// A clean lifecycle emits `exit`, never `error` — the capture stays
|
||||
// pending forever, so a race against it is decided by the other arms.
|
||||
const settled = await Promise.race([
|
||||
failure.then(() => 'settled'),
|
||||
new Promise<string>(resolve => setTimeout(() => { resolve('pending') }, 30)),
|
||||
])
|
||||
expect(settled).toBe('pending')
|
||||
})
|
||||
})
|
||||
|
||||
describe('waitForExit / exitsWithin', () => {
|
||||
it('resolves immediately for a child that already exited by code', async () => {
|
||||
const fake = new FakeChild()
|
||||
fake.exitCode = 0
|
||||
await expect(waitForExit(asChild(fake))).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('resolves immediately for a child that already died by signal', async () => {
|
||||
const fake = new FakeChild()
|
||||
fake.signalCode = 'SIGTERM'
|
||||
await expect(waitForExit(asChild(fake))).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('resolves on the exit event of a live child', async () => {
|
||||
const fake = new FakeChild({ diesOn: 'SIGTERM', delayMs: 5 })
|
||||
const exited = waitForExit(asChild(fake))
|
||||
fake.kill('SIGTERM')
|
||||
await expect(exited).resolves.toBeUndefined()
|
||||
expect(fake.signalCode).toBe('SIGTERM')
|
||||
})
|
||||
|
||||
it('exitsWithin resolves true immediately for an already-exited child (no listener attached)', async () => {
|
||||
const fake = new FakeChild()
|
||||
fake.exitCode = 0
|
||||
await expect(exitsWithin(asChild(fake), 1000)).resolves.toBe(true)
|
||||
expect(fake.listenerCount('exit')).toBe(0)
|
||||
})
|
||||
|
||||
it('exitsWithin resolves true when the child exits inside the window', async () => {
|
||||
const fake = new FakeChild({ diesOn: 'SIGTERM', delayMs: 5 })
|
||||
fake.kill('SIGTERM')
|
||||
await expect(exitsWithin(asChild(fake), 1000)).resolves.toBe(true)
|
||||
// The once-listener fired and the grace timer was cleared — nothing lingers.
|
||||
expect(fake.listenerCount('exit')).toBe(0)
|
||||
})
|
||||
|
||||
it('exitsWithin resolves false on timeout for a child that never exits', async () => {
|
||||
const fake = new FakeChild() // nothing short of SIGKILL fells it; no signal sent
|
||||
await expect(exitsWithin(asChild(fake), 20)).resolves.toBe(false)
|
||||
// The timeout arm removed its exit listener: repeated waits (a poll loop,
|
||||
// the ladder's tiers) never accumulate listeners on the same child.
|
||||
expect(fake.listenerCount('exit')).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('disposeChildProcess', () => {
|
||||
it('returns immediately for an already-exited child (no EOF, no signals)', async () => {
|
||||
const fake = new FakeChild()
|
||||
fake.exitCode = 0
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 1000, disposeGraceMs: 1000 })
|
||||
expect(fake.stdinEnded).toBe(false)
|
||||
expect(fake.kills).toEqual([])
|
||||
})
|
||||
|
||||
it('returns immediately for a child already dead by signal', async () => {
|
||||
const fake = new FakeChild()
|
||||
fake.signalCode = 'SIGKILL'
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 1000, disposeGraceMs: 1000 })
|
||||
expect(fake.stdinEnded).toBe(false)
|
||||
expect(fake.kills).toEqual([])
|
||||
})
|
||||
|
||||
it('tier 1: a cooperative child quiesces on stdin EOF — no signal is ever sent', async () => {
|
||||
const fake = new FakeChild({ diesOn: 'eof', delayMs: 5 })
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 1000, disposeGraceMs: 1000 })
|
||||
expect(fake.stdinEnded).toBe(true)
|
||||
expect(fake.kills).toEqual([])
|
||||
expect(fake.exitCode).toBe(0)
|
||||
})
|
||||
|
||||
it('tier 2: a child that ignores EOF but honors SIGTERM dies on the middle rung', async () => {
|
||||
const fake = new FakeChild({ diesOn: 'SIGTERM', delayMs: 5 })
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 20, disposeGraceMs: 1000 })
|
||||
expect(fake.stdinEnded).toBe(true)
|
||||
expect(fake.kills).toEqual(['SIGTERM'])
|
||||
expect(fake.signalCode).toBe('SIGTERM')
|
||||
})
|
||||
|
||||
it('tier 3: a SIGTERM-trapping child is SIGKILLed, and dispose resolves only after the exit', async () => {
|
||||
const fake = new FakeChild({ delayMs: 5 }) // only SIGKILL fells it
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 20, disposeGraceMs: 20 })
|
||||
expect(fake.kills).toEqual(['SIGTERM', 'SIGKILL'])
|
||||
// Quiescence, not a request: at resolution the child has ACTUALLY exited
|
||||
// (the exit event landed, despite the scripted post-SIGKILL delay).
|
||||
expect(fake.signalCode).toBe('SIGKILL')
|
||||
})
|
||||
|
||||
it('walks the ladder for a child spawned without a stdin pipe', async () => {
|
||||
const fake = new FakeChild({ stdin: false, diesOn: 'SIGTERM', delayMs: 5 })
|
||||
await disposeChildProcess(asChild(fake), { disposeEofGraceMs: 20, disposeGraceMs: 1000 })
|
||||
expect(fake.kills).toEqual(['SIGTERM'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('createIsolatedConfigDir', () => {
|
||||
it('creates a fresh private mkdtemp dir under the OS temp root', async () => {
|
||||
const dir = await createIsolatedConfigDir('dsh-subagent-subprocess-test-')
|
||||
try {
|
||||
expect(dir.path.startsWith(join(tmpdir(), 'dsh-subagent-subprocess-test-'))).toBe(true)
|
||||
const st = await stat(dir.path)
|
||||
expect(st.isDirectory()).toBe(true)
|
||||
// Private (0700) per the defensive-patterns temp-dir rule.
|
||||
expect(st.mode & 0o777).toBe(0o700)
|
||||
} finally {
|
||||
await dir.remove()
|
||||
}
|
||||
})
|
||||
|
||||
it('creates a distinct dir per call (per-run isolation)', async () => {
|
||||
const a = await createIsolatedConfigDir('dsh-subagent-subprocess-test-')
|
||||
const b = await createIsolatedConfigDir('dsh-subagent-subprocess-test-')
|
||||
try {
|
||||
expect(a.path).not.toBe(b.path)
|
||||
} finally {
|
||||
await a.remove()
|
||||
await b.remove()
|
||||
}
|
||||
})
|
||||
|
||||
it('remove() deletes a fresh dir recursively and is idempotent', async () => {
|
||||
const dir = await createIsolatedConfigDir('dsh-subagent-subprocess-test-')
|
||||
await writeFile(join(dir.path, 'settings.json'), '{}')
|
||||
await dir.remove()
|
||||
expect(existsSync(dir.path)).toBe(false)
|
||||
// Second remove: nothing left to delete, still resolves.
|
||||
await expect(dir.remove()).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('returns a pinned dir verbatim and NEVER removes it', async () => {
|
||||
const pinned = await mkdtemp(join(tmpdir(), 'dsh-subagent-subprocess-pinned-'))
|
||||
try {
|
||||
const dir = await createIsolatedConfigDir('ignored-prefix-', pinned)
|
||||
expect(dir.path).toBe(pinned)
|
||||
await dir.remove()
|
||||
// The deployment owns a pinned dir's lifecycle — remove() must not touch it.
|
||||
expect(existsSync(pinned)).toBe(true)
|
||||
} finally {
|
||||
await rm(pinned, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('does not create a missing pinned path (the deployment owns its lifecycle)', async () => {
|
||||
const missing = join(tmpdir(), `dsh-subagent-subprocess-missing-${process.pid}`)
|
||||
const dir = await createIsolatedConfigDir('ignored-prefix-', missing)
|
||||
expect(dir.path).toBe(missing)
|
||||
expect(existsSync(missing)).toBe(false)
|
||||
await dir.remove()
|
||||
expect(existsSync(missing)).toBe(false)
|
||||
})
|
||||
|
||||
it('remove() is best-effort: an rm rejection resolves instead of rejecting', async () => {
|
||||
const dir = await createIsolatedConfigDir('dsh-subagent-subprocess-locked-')
|
||||
try {
|
||||
// The swallow contract is error-kind agnostic; EACCES stands in for the
|
||||
// family (EBUSY, a vanished mount, …) that best-effort must absorb.
|
||||
vi.mocked(rm).mockRejectedValueOnce(Object.assign(new Error('EACCES: permission denied'), { code: 'EACCES' }))
|
||||
await expect(dir.remove()).resolves.toBeUndefined()
|
||||
// The injected rejection consumed the only rm call — nothing was deleted.
|
||||
expect(existsSync(dir.path)).toBe(true)
|
||||
} finally {
|
||||
await rm(dir.path, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
})
|
||||
11
packages/subagent/subagent-subprocess/tsconfig.json
Normal file
11
packages/subagent/subagent-subprocess/tsconfig.json
Normal file
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": []
|
||||
}
|
||||
@@ -6,7 +6,7 @@ Three layers, importable separately:
|
||||
|
||||
- **`runScenario` (harness)** — boots the real agent bin as a subprocess via tsx (unbuilt, Loader path), drives it over ACP JSON-RPC stdio from a deterministic `input.json` script, tees raw stdout for the golden + purity check, and harvests every persisted session JSONL (parent + subagent children, primary-first) after a graceful stdin-EOF shutdown. Parameterized by `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath` — absolute paths; the subprocess cwd is a temp dir outside the repo).
|
||||
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), and the composable `scrubRequestHeaders` (header bulk → `{{system}}`/`{{tools}}`, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
|
||||
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record-mode fixture write-back, the per-suite header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin, non-pinning fixtures header-scrubbed). Must be called at vitest collection time.
|
||||
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record-mode fixture write-back, the per-header-class pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, pinning fixtures well-formed, non-pinning fixtures header-scrubbed). Must be called at vitest collection time.
|
||||
|
||||
A consuming `*.snapshot.ts` is the scenario table plus one factory call:
|
||||
|
||||
@@ -26,11 +26,13 @@ defineAcpSnapshotSuite({
|
||||
tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)),
|
||||
},
|
||||
snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'),
|
||||
scenarios: SCENARIOS, // exactly one entry sets pinsHeader
|
||||
scenarios: SCENARIOS, // exactly one entry per header class sets pinsHeader
|
||||
mode: process.env.DSH_SNAPSHOT === 'record' ? 'record' : 'replay',
|
||||
})
|
||||
```
|
||||
|
||||
A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode scenarios are the template.
|
||||
|
||||
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config RFC](../../../docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. Fixture roles, record/replay semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
|
||||
|
||||
Constraints: `suite.ts` imports vitest, so the package is importable only inside a vitest run (the harness and normalizers have no such dependency but ship from the same entry). ACP-specific by design — the harness speaks the SDK's `ClientSideConnection`. Permission round-trips are scriptable: `InputScript.permissionAnswers` is a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) the client maps to the agent-issued `optionId` at answer time; an absent or exhausted queue answers `cancelled`, and a kind the request never offered rejects the run (the agent is answered `cancelled`, so a tolerant agent cannot absorb the scenario bug).
|
||||
|
||||
@@ -166,6 +166,15 @@ export interface RunOptions {
|
||||
* start from an empty workspace.
|
||||
*/
|
||||
workspaceDir?: string
|
||||
/**
|
||||
* Alternate LIVE config path for the boot (absolute), overriding
|
||||
* {@link AgentUnderTest.configPath} for this run. A scenario needing a
|
||||
* differently-composed tree (the Code Mode scenarios) ships an overlay
|
||||
* whose basename still ends in `cordis.yml`, so the bin's replay swap
|
||||
* resolves the sibling `*cordis.snapshot.yml` the same way it does for
|
||||
* the default.
|
||||
*/
|
||||
configPath?: string
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -211,7 +220,7 @@ export async function runScenario(input: InputScript, opts: RunOptions): Promise
|
||||
|
||||
child = spawn(
|
||||
process.execPath,
|
||||
['--import', tsxLoader, opts.agent.binScript, opts.agent.configPath],
|
||||
['--import', tsxLoader, opts.agent.binScript, opts.configPath ?? opts.agent.configPath],
|
||||
{ cwd, env, stdio: ['pipe', 'pipe', 'pipe'] },
|
||||
)
|
||||
|
||||
|
||||
@@ -11,13 +11,14 @@
|
||||
* before comparing).
|
||||
*
|
||||
* Request-header content (the composed system prompt + tool schemas riding on
|
||||
* `request/header` events) is pinned by exactly ONE scenario per suite — the
|
||||
* one with `pinsHeader` — and scrubbed to `{{system}}`/`{{tools}}` tokens in
|
||||
* every other fixture and compare, so a prompt or tool-schema edit churns one
|
||||
* committed line instead of every fixture. A per-run uniformity guard keeps
|
||||
* the single pin sound: every live header must equal the pinned one, and no
|
||||
* header-delta may appear outside the pinning scenario (see the
|
||||
* pinned-header RFC,
|
||||
* `request/header` events) is pinned by exactly ONE scenario per HEADER CLASS
|
||||
* — scenarios that boot the same config compose the same header; each class's
|
||||
* `pinsHeader` scenario commits it verbatim — and scrubbed to
|
||||
* `{{system}}`/`{{tools}}` tokens in every other fixture and compare, so a
|
||||
* prompt or tool-schema edit churns one committed line per class instead of
|
||||
* every fixture. A per-run uniformity guard keeps each pin sound: every live
|
||||
* header must equal its class's pinned one, and no header-delta may appear
|
||||
* outside a pinning scenario (see the pinned-header RFC,
|
||||
* docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md).
|
||||
*
|
||||
* `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the
|
||||
@@ -80,27 +81,38 @@ export interface Scenario {
|
||||
* Whether THIS scenario's fixtures keep the full request-header content (the
|
||||
* composed system prompt and tool schema list on `request/header` /
|
||||
* `request/header-delta` events) and compare it verbatim. Exactly one
|
||||
* scenario per suite pins it; every other scenario stores and compares that
|
||||
* content as `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}),
|
||||
* scenario per HEADER CLASS ({@link headerClass}) pins it; every other
|
||||
* scenario of that class stores and compares that content as
|
||||
* `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}),
|
||||
* so a system prompt or tool-schema change shows up as ONE committed-fixture
|
||||
* diff, not one per scenario. One pin suffices because header composition is
|
||||
* suite-uniform (parent, spawn child, and fork child all compose the same
|
||||
* prompt-modulo-cwd and the same tools) — and that premise is ASSERTED, not
|
||||
* assumed: every non-pinning run's live headers must equal the pinned
|
||||
* fixture's (normalized), so a session-dependent header (say, a restricted
|
||||
* subagent toolset) fails loud until it gets its own pinning scenario.
|
||||
* diff per class, not one per scenario. One pin per class suffices because
|
||||
* header composition is class-uniform (parent, spawn child, and fork child
|
||||
* all compose the same prompt-modulo-cwd and the same tools) — and that
|
||||
* premise is ASSERTED, not assumed: every non-pinning run's live headers
|
||||
* must equal its class's pinned fixture's (normalized), so a
|
||||
* session-dependent header (say, a restricted subagent toolset) fails loud
|
||||
* until it gets its own pinning scenario.
|
||||
* Defaults to false.
|
||||
*/
|
||||
pinsHeader?: boolean
|
||||
/**
|
||||
* Whether this scenario intentionally composes a different live request
|
||||
* header from the suite's pinned baseline while still storing scrubbed header
|
||||
* content in its fixture. Use only for cwd/project-sensitive context fixtures
|
||||
* whose header difference is the behavior under test; the fixture guard still
|
||||
* rejects committed full header content unless {@link pinsHeader} is true.
|
||||
* Defaults to false.
|
||||
* Which header-composition class this scenario belongs to. Scenarios that
|
||||
* boot the same config compose the same header; each class has exactly one
|
||||
* {@link pinsHeader} scenario, and the uniformity guard compares every
|
||||
* other member against ITS class's pin. Defaults to `'default'`; a
|
||||
* scenario booting an alternate config ({@link configPath}) whose tool
|
||||
* list or prompt sections differ by construction carries its own class.
|
||||
*/
|
||||
variesHeader?: boolean
|
||||
headerClass?: string
|
||||
/**
|
||||
* Alternate LIVE config path (absolute) this scenario boots instead of
|
||||
* {@link AgentUnderTest.configPath} — an overlay composing a different
|
||||
* tree (its basename must still end in `cordis.yml` so the bin's replay
|
||||
* swap finds the sibling `*cordis.snapshot.yml`). A scenario whose
|
||||
* overlay changes the composed header also needs its own
|
||||
* {@link headerClass}.
|
||||
*/
|
||||
configPath?: string
|
||||
}
|
||||
|
||||
/** One suite's inputs: the agent to boot, where its fixtures live, and its scenario table. */
|
||||
@@ -192,10 +204,11 @@ export function headerDeltaCount(rawLog: string): number {
|
||||
/**
|
||||
* Register the suite: one `describe` per scenario (the golden/log compares and
|
||||
* the header-uniformity guard) plus the fixture guard block (no orphan
|
||||
* scenario dirs, required files present, exactly one pin, non-pinning fixtures
|
||||
* header-scrubbed). Must run at vitest collection time — it calls
|
||||
* `describe`/`it`. Throws immediately if no scenario pins the header (the
|
||||
* uniformity guard would have nothing to compare against).
|
||||
* scenario dirs, required files present, exactly one pin per header class,
|
||||
* pinning fixtures well-formed, non-pinning fixtures header-scrubbed). Must
|
||||
* run at vitest collection time — it calls `describe`/`it`. Throws
|
||||
* immediately if any header class lacks a pinning scenario or carries two
|
||||
* (the uniformity guard needs exactly one comparison anchor per class).
|
||||
*
|
||||
* @param options The agent, snapshots directory, scenario table, and mode.
|
||||
*/
|
||||
@@ -203,9 +216,23 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
const { agent, snapshotsDir, scenarios, mode } = options
|
||||
const RECORDING = mode === 'record'
|
||||
|
||||
/** The suite's single header-pinning scenario. Guarded here (and by a meta-test) so the pin cannot silently vanish. */
|
||||
const pinningScenario = scenarios.find(s => s.pinsHeader === true)
|
||||
if (pinningScenario === undefined) throw new Error('acp-snapshot: no scenario pins the request-header content')
|
||||
/** The class a scenario's header composition belongs to (see {@link Scenario.headerClass}). */
|
||||
const classOf = (scenario: Scenario): string => scenario.headerClass ?? 'default'
|
||||
|
||||
/** Each header class's single pinning scenario. Guarded here (and by meta-tests) so a pin cannot silently vanish or split. */
|
||||
const pinningByClass = new Map<string, Scenario>()
|
||||
for (const scenario of scenarios) {
|
||||
if (scenario.pinsHeader !== true) continue
|
||||
const cls = classOf(scenario)
|
||||
const existing = pinningByClass.get(cls)
|
||||
if (existing) throw new Error(`acp-snapshot: header class "${cls}" pinned by both ${existing.name} and ${scenario.name}`)
|
||||
pinningByClass.set(cls, scenario)
|
||||
}
|
||||
for (const scenario of scenarios) {
|
||||
if (!pinningByClass.has(classOf(scenario))) {
|
||||
throw new Error(`acp-snapshot: no scenario pins the request-header content of class "${classOf(scenario)}" (needed by ${scenario.name})`)
|
||||
}
|
||||
}
|
||||
|
||||
for (const scenario of scenarios) {
|
||||
describe(`snapshot: ${scenario.name}`, () => {
|
||||
@@ -226,6 +253,9 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
// replays from its own script. In RECORD they are harvested, not read.
|
||||
...!RECORDING && childSessions > 0 ? { childFiles: childFixturePaths(dir, childSessions) } : {},
|
||||
...existsSync(workspaceDir) ? { workspaceDir } : {},
|
||||
// A scenario booting an overlay tree passes its own live config; the
|
||||
// bin's replay swap derives the sibling `*cordis.snapshot.yml` from it.
|
||||
...scenario.configPath !== undefined ? { configPath: scenario.configPath } : {},
|
||||
})
|
||||
|
||||
// Scrub every volatile id the run produced: the ACP server-issued session
|
||||
@@ -286,19 +316,22 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
}
|
||||
}
|
||||
|
||||
// Header-uniformity guard: the single pin is sound only while every
|
||||
// session in the suite composes the SAME header and keeps it for the
|
||||
// whole run. Assert both halves live. (1) Every request/header the run
|
||||
// produced (parent, spawn child, fork child, initial or resume) must
|
||||
// equal the pinned fixture's header after each side is normalized
|
||||
// against its own volatile values. (2) No request/header-delta may
|
||||
// appear at all — a mid-run header change diverges from the pin by
|
||||
// construction, and its content would be invisible under the scrub. If
|
||||
// either fails, either the header changed (update the pin: re-record or
|
||||
// hand-edit the pinning scenario's fixture) or composition became
|
||||
// session-dependent by design (give the divergent shape its own
|
||||
// pinning scenario).
|
||||
if (scenario.pinsHeader !== true && scenario.variesHeader !== true) {
|
||||
// Header-uniformity guard: a class's single pin is sound only while
|
||||
// every session in that class composes the SAME header and keeps it
|
||||
// for the whole run. Assert both halves live. (1) Every
|
||||
// request/header the run produced (parent, spawn child, fork child,
|
||||
// initial or resume) must equal the CLASS's pinned fixture's header
|
||||
// after each side is normalized against its own volatile values.
|
||||
// (2) No request/header-delta may appear at all — a mid-run header
|
||||
// change diverges from the pin by construction, and its content
|
||||
// would be invisible under the scrub. If either fails, either the
|
||||
// header changed (update the pin: re-record or hand-edit the pinning
|
||||
// scenario's fixture) or composition became session-dependent by
|
||||
// design (give the divergent shape its own pinning scenario and
|
||||
// class).
|
||||
if (scenario.pinsHeader !== true) {
|
||||
/* v8 ignore next -- construction guarantees the pin exists; a miss would fail the one-header assertion loudly. */
|
||||
const pinningScenario = pinningByClass.get(classOf(scenario)) ?? scenario
|
||||
const pinnedFixture = await readFile(join(snapshotsDir, pinningScenario.name, 'session.jsonl'), 'utf8')
|
||||
const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture))
|
||||
expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`)
|
||||
@@ -356,11 +389,35 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
}
|
||||
})
|
||||
|
||||
it('exactly one scenario pins the request-header content', () => {
|
||||
// Zero pins would drop the prompt/schema surface from the suite entirely;
|
||||
// two would split it. One pin per suite is the design (pinned-header RFC);
|
||||
// WHICH scenario pins is the scenario table's reviewable choice.
|
||||
expect(scenarios.filter(s => s.pinsHeader === true).map(s => s.name)).toEqual([pinningScenario.name])
|
||||
it('exactly one scenario pins the request-header content of each header class', () => {
|
||||
// Zero pins would drop a class's prompt/schema surface from the suite
|
||||
// entirely; two would split it. One pin per class is the design
|
||||
// (pinned-header RFC); WHICH scenario pins is the scenario table's
|
||||
// reviewable choice.
|
||||
const pins = new Map<string, string[]>()
|
||||
for (const scenario of scenarios.filter(s => s.pinsHeader === true)) {
|
||||
const cls = classOf(scenario)
|
||||
pins.set(cls, [...pins.get(cls) ?? [], scenario.name])
|
||||
}
|
||||
expect(Object.fromEntries([...pins].map(([cls, names]) => [cls, names.length]))).toEqual(
|
||||
Object.fromEntries([...pinningByClass.keys()].map(cls => [cls, 1])))
|
||||
for (const scenario of scenarios) {
|
||||
expect(pinningByClass.has(classOf(scenario)), `class "${classOf(scenario)}" (scenario ${scenario.name}) has a pin`).toBe(true)
|
||||
}
|
||||
})
|
||||
|
||||
it('every pinning fixture carries exactly one request/header and no deltas', async () => {
|
||||
// The live uniformity guard runs only in NON-pinning scenarios, so a
|
||||
// class made of just its pinning scenario would otherwise accept a
|
||||
// re-recorded pin with several headers or a mid-run header-delta —
|
||||
// shapes the pin design cannot represent. Assert the committed pins
|
||||
// directly.
|
||||
for (const scenario of pinningByClass.values()) {
|
||||
const fixture = await readFile(join(snapshotsDir, scenario.name, 'session.jsonl'), 'utf8')
|
||||
const headers = normalizedHeaders(fixture, fixtureContext(fixture))
|
||||
expect(headers.length, `${scenario.name}: a pinning fixture must carry exactly one request/header`).toBe(1)
|
||||
expect(headerDeltaCount(fixture), `${scenario.name}: a pinning fixture must carry no request/header-delta`).toBe(0)
|
||||
}
|
||||
})
|
||||
|
||||
it('committed fixtures carry request-header content ONLY in the pinning scenario', async () => {
|
||||
|
||||
@@ -34,12 +34,18 @@ const AGENT = {
|
||||
const REPLAY_DIR = fileURLToPath(new URL('./fixtures/suite', import.meta.url))
|
||||
const RECORD_SRC = fileURLToPath(new URL('./fixtures/record-suite', import.meta.url))
|
||||
|
||||
// The replay suite doubles as the header-CLASS coverage: every scenario names
|
||||
// the same explicit class (the record suite exercises the 'default' fallback),
|
||||
// and plain-turn boots through a per-scenario configPath override (the same
|
||||
// dummy path the agent default carries — the plumbing, not the composition,
|
||||
// is what this suite can exercise; the real overlay boot is the acp-agent
|
||||
// example's code-mode scenarios).
|
||||
const REPLAY_SCENARIOS: Scenario[] = [
|
||||
{ name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true },
|
||||
{ name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1 },
|
||||
{ name: 'no-model', hasModelTurn: false, recorded: false },
|
||||
{ name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false },
|
||||
{ name: 'authored-error', hasModelTurn: true, recorded: false, overridden: true },
|
||||
{ name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'main' },
|
||||
{ name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1, headerClass: 'main', configPath: AGENT.configPath },
|
||||
{ name: 'no-model', hasModelTurn: false, recorded: false, headerClass: 'main' },
|
||||
{ name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false, headerClass: 'main' },
|
||||
{ name: 'authored-error', hasModelTurn: true, recorded: false, overridden: true, headerClass: 'main' },
|
||||
]
|
||||
|
||||
const RECORD_SCENARIOS: Scenario[] = [
|
||||
@@ -70,7 +76,7 @@ describe('defineAcpSnapshotSuite: record mode', () => {
|
||||
})
|
||||
|
||||
describe('defineAcpSnapshotSuite: registration contract', () => {
|
||||
it('throws when no scenario pins the request-header content', () => {
|
||||
it("throws when a scenario's header class has no pinning scenario", () => {
|
||||
expect(() => {
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
@@ -78,7 +84,33 @@ describe('defineAcpSnapshotSuite: registration contract', () => {
|
||||
scenarios: [{ name: 'pinless', hasModelTurn: true, recorded: true }],
|
||||
mode: 'replay',
|
||||
})
|
||||
}).toThrow(/no scenario pins/)
|
||||
}).toThrow(/no scenario pins the request-header content of class "default"/)
|
||||
// A pinned class does not cover a DIFFERENT class's members.
|
||||
expect(() => {
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
snapshotsDir: REPLAY_DIR,
|
||||
scenarios: [
|
||||
{ name: 'pinned', hasModelTurn: true, recorded: true, pinsHeader: true },
|
||||
{ name: 'classless-orphan', hasModelTurn: true, recorded: true, headerClass: 'other' },
|
||||
],
|
||||
mode: 'replay',
|
||||
})
|
||||
}).toThrow(/class "other" \(needed by classless-orphan\)/)
|
||||
})
|
||||
|
||||
it('throws when two scenarios pin the same header class', () => {
|
||||
expect(() => {
|
||||
defineAcpSnapshotSuite({
|
||||
agent: AGENT,
|
||||
snapshotsDir: REPLAY_DIR,
|
||||
scenarios: [
|
||||
{ name: 'first-pin', hasModelTurn: true, recorded: true, pinsHeader: true },
|
||||
{ name: 'second-pin', hasModelTurn: true, recorded: true, pinsHeader: true },
|
||||
],
|
||||
mode: 'replay',
|
||||
})
|
||||
}).toThrow(/header class "default" pinned by both first-pin and second-pin/)
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
@@ -36,6 +36,7 @@
|
||||
"@deepseek-ai/dsh-acp": "^0.0.1",
|
||||
"@deepseek-ai/dsh-agent-core": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"@deepseek-ai/dsh-user-interaction": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6",
|
||||
"schemastery": "^3.17.0"
|
||||
@@ -46,6 +47,7 @@
|
||||
"@deepseek-ai/dsh-app-boot": "workspace:^",
|
||||
"@deepseek-ai/dsh-acp": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-core": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
"@deepseek-ai/dsh-user-interaction": "workspace:^",
|
||||
|
||||
@@ -34,6 +34,7 @@ import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import * as acp from '@deepseek-ai/dsh-acp'
|
||||
import * as agentCore from '@deepseek-ai/dsh-agent-core'
|
||||
import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import UserInteractionService from '@deepseek-ai/dsh-user-interaction'
|
||||
|
||||
@@ -45,7 +46,8 @@ export const name = 'acp-agent'
|
||||
* pre-created agent — ACP creates agents at `session/new`); `persona` is the
|
||||
* deployment persona (forwarded to the system-prompt plugin); `toolOrder` is
|
||||
* the explicit model-facing tool order (forwarded to the system-prompt plugin);
|
||||
* `persistenceRoot` is the JSONL backend's directory.
|
||||
* `tools` is the tool registry's config (its presentation `mode`, forwarded
|
||||
* through agent-core); `persistenceRoot` is the JSONL backend's directory.
|
||||
*/
|
||||
export interface Config {
|
||||
/** Model name for ACP-created agents (must have a registered adapter). */
|
||||
@@ -54,6 +56,8 @@ export interface Config {
|
||||
persona?: string
|
||||
/** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */
|
||||
toolOrder?: string[]
|
||||
/** Tool-registry config — its presentation `mode` (forwarded through agent-core; see dsh-tools). */
|
||||
tools?: ToolsConfig
|
||||
/** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */
|
||||
persistenceRoot?: string
|
||||
/** Skill registry, local-provider, and model-facing consumer config forwarded to agent-core. */
|
||||
@@ -67,6 +71,7 @@ export const Config: z<Config> = z.object({
|
||||
// order" (the owning dsh-system-prompt schema does the same), while
|
||||
// schemastery's native [] default would read as an invalid configured list.
|
||||
toolOrder: z.array(z.string()).default(undefined as unknown as string[]),
|
||||
tools: ToolRegistry.Config,
|
||||
persistenceRoot: z.string().default('./.sessions'),
|
||||
skills: agentCore.SkillConfigSchema,
|
||||
})
|
||||
@@ -82,6 +87,7 @@ export function apply(ctx: Context, config: Config): void {
|
||||
ctx.plugin(agentCore, {
|
||||
...config.persona !== undefined ? { persona: config.persona } : {},
|
||||
...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {},
|
||||
...config.tools !== undefined ? { tools: config.tools } : {},
|
||||
...config.skills !== undefined ? { skills: config.skills } : {},
|
||||
})
|
||||
ctx.plugin(UserInteractionService)
|
||||
|
||||
@@ -38,6 +38,7 @@
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-agent-core": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tool-ask-user": "^0.0.1",
|
||||
"@deepseek-ai/dsh-user-interaction": "^0.0.1",
|
||||
@@ -54,6 +55,7 @@
|
||||
"@deepseek-ai/dsh-agent-core": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
"@deepseek-ai/dsh-tool-ask-user": "workspace:^",
|
||||
"@deepseek-ai/dsh-user-interaction": "workspace:^",
|
||||
|
||||
@@ -43,6 +43,7 @@ import ConsoleExporter from '@cordisjs/plugin-logger-console'
|
||||
import z from 'schemastery'
|
||||
import { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import ToolRegistry, { type Config as ToolsConfig } from '@deepseek-ai/dsh-tools'
|
||||
import * as agentCore from '@deepseek-ai/dsh-agent-core'
|
||||
import SessionPersistenceJsonl from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import UserInteractionService from '@deepseek-ai/dsh-user-interaction'
|
||||
@@ -68,6 +69,8 @@ export interface Config {
|
||||
persona?: string
|
||||
/** Explicit model-facing tool order (the system-prompt plugin's `toolOrder` config; see dsh-system-prompt). */
|
||||
toolOrder?: string[]
|
||||
/** Tool-registry config — its presentation `mode` (forwarded through agent-core; see dsh-tools). */
|
||||
tools?: ToolsConfig
|
||||
/** Directory the JSONL session backend writes under. Defaults to `./.sessions`. */
|
||||
persistenceRoot?: string
|
||||
/** stdin-chat banner printed once on start. Defaults to `'ready.'`. */
|
||||
@@ -89,6 +92,7 @@ export const Config: z<Config> = z.object({
|
||||
// order" (the owning dsh-system-prompt schema does the same), while
|
||||
// schemastery's native [] default would read as an invalid configured list.
|
||||
toolOrder: z.array(z.string()).default(undefined as unknown as string[]),
|
||||
tools: ToolRegistry.Config,
|
||||
persistenceRoot: z.string().default('./.sessions'),
|
||||
welcome: z.string().default('ready.'),
|
||||
skills: agentCore.SkillConfigSchema,
|
||||
@@ -107,6 +111,7 @@ export function apply(ctx: Context, config: Config): void {
|
||||
ctx.plugin(agentCore, {
|
||||
...config.persona !== undefined ? { persona: config.persona } : {},
|
||||
...config.toolOrder !== undefined ? { toolOrder: config.toolOrder } : {},
|
||||
...config.tools !== undefined ? { tools: config.tools } : {},
|
||||
agents: [{
|
||||
id: AgentId('main'),
|
||||
model: config.model,
|
||||
|
||||
13
packages/workflow/README.md
Normal file
13
packages/workflow/README.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# workflow/ — dynamic-workflow capability family
|
||||
|
||||
The workflow seam: a model-written JavaScript orchestration script that fans out subagents at scale (phases, structured per-agent results, concurrency caps), modeled on Claude Code's dynamic workflows. A capability seam (see [capability seams](../../docs/rfc/implemented/architecture/2026-06-13-capability-seams.md)) in the bash shape: ONE engine implementation per context registers as `ctx.workflows`; the model-facing tool consumes it.
|
||||
|
||||
| Package | Role | ctx key |
|
||||
|---|---|---|
|
||||
| `workflow/` | Abstract workflow seam: service base class + run vocabulary + `workflow/*` events | `ctx.workflows` |
|
||||
| `workflow-workerthread/` | `node:worker_threads` engine: one worker per run; the script's vm context lives inside the worker, `agent()` bridges to `ctx.subagents` over the message port | (provides `ctx.workflows`) |
|
||||
| `tool-workflow/` | Model-facing `workflow` tool over `ctx.workflows` | (registers on `ctx.tools`) |
|
||||
|
||||
The interface lives at `workflow/workflow/`. The engine's `agent()` hook rides the [subagent seam](../subagent/README.md) (any registered provider; the shipped examples use `spawn`), and `agent({ schema })` rides the structured-output support the in-process backends implement. The worker thread isolates the SCRIPT — the host never blocks on it, and a cancelled run's post-grace termination is real — but it is NOT a security boundary; an isolated-vm/separate-process engine (actual sandboxing) swaps in behind the same interface if that ever matters.
|
||||
|
||||
The proposal, decisions, and deferred work: [docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md](../../docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md).
|
||||
22
packages/workflow/tool-workflow/README.md
Normal file
22
packages/workflow/tool-workflow/README.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# @deepseek-ai/dsh-tool-workflow
|
||||
|
||||
The model-facing **`workflow` tool**: run a JavaScript orchestration script that fans out subagents, and return the script's final value. Pure schema + lifecycle shaping over [`ctx.workflows`](../workflow/README.md) — script parsing, execution, caps, and cancellation live behind the seam, so a hardened engine swaps in without touching what the model sees.
|
||||
|
||||
## What the model sees
|
||||
|
||||
Two parameters: `script` (required — the full `export const meta = {...}` + body text; the tool DESCRIPTION carries the complete authoring contract: hooks, semantics, the supported schema subset) and `args` (optional JSON object exposed to the script as the `args` global; a bare list is wrapped as a field, a deliberate deviation from Claude Code's any-JSON `args` so the wire schema stays honest). The plugin also contributes a `tool:<toolName>` system-prompt section carrying the usage policy — use the tool only on an explicit user ask for a workflow / large orchestration; prefer plain subagent calls for one or two delegations — per the convention that tool guidance ships with the tool plugin, never in the deployment persona.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
Collection is SYNCHRONOUS this cut (like [`dsh-tool-subagent`](../../subagent/tool-subagent/README.md)): `execute` starts a run and awaits `run.result` inside a `try/finally` that always disposes the run, so the script and its children reach quiescence on every path. `exec.signal` is bridged to `run.cancel()` (including the already-aborted-before-start case). A non-`completed` stop reason maps to an `isError` result reporting the reason — never partial output as success; a parse/meta failure thrown synchronously by `start()` becomes an `isError` the model can correct from. The completed result renders the meta name, the agent count, and the return value as JSON, truncated at `maxResultChars` with an explicit notice.
|
||||
|
||||
## Render intent
|
||||
|
||||
Decided up front (per the [render-intent RFC](../../../docs/rfc/implemented/architecture/2026-07-02-tool-render-intent-union.md)): a `generic` card titled `workflow: <meta.name>`, the name sniffed TEXTUALLY from `args.script` (presentation must be a pure function of args, so it cannot ask the engine to parse); the script text rides as `rawInput`. The result keeps the generic card.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `toolName` | `workflow` | The model-facing tool name to register. |
|
||||
| `maxResultChars` | `50000` | Rendered-result ceiling; longer JSON is truncated with a notice. |
|
||||
46
packages/workflow/tool-workflow/package.json
Normal file
46
packages/workflow/tool-workflow/package.json
Normal file
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-tool-workflow",
|
||||
"description": "Model-facing workflow tool: run a JavaScript orchestration script over ctx.workflows",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"@deepseek-ai/dsh-workflow": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow-workerthread": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
215
packages/workflow/tool-workflow/src/index.ts
Normal file
215
packages/workflow/tool-workflow/src/index.ts
Normal file
@@ -0,0 +1,215 @@
|
||||
/**
|
||||
* The model-facing `workflow` tool: run a JavaScript orchestration script that
|
||||
* fans out subagents, and return the script's final value. Pure schema +
|
||||
* lifecycle shaping — script parsing, execution, caps, and cancellation live
|
||||
* behind `ctx.workflows` (`@deepseek-ai/dsh-workflow`), so a hardened engine
|
||||
* swaps in without touching what the model sees.
|
||||
*
|
||||
* Collection is SYNCHRONOUS this cut (like `dsh-tool-subagent`): `execute`
|
||||
* starts a run and awaits `run.result` inside a `try/finally` that always
|
||||
* disposes the run, so the script and its children are torn down on every
|
||||
* path. A non-`completed` stop reason maps to an `isError` tool result (by
|
||||
* throwing) rather than returning partial output as success. Background
|
||||
* collection is deferred to the cross-tool background redesign.
|
||||
*
|
||||
* Render intent (decided up front, per the render-intent RFC): a `generic`
|
||||
* card whose title carries the workflow's `meta.name`, read directly from the
|
||||
* call's `meta` parameter — presentation is a pure function of `args`.
|
||||
*
|
||||
* Usage policy ships with the tool as a `tool:<toolName>` system-prompt
|
||||
* section (explicit-ask-only guidance) — tool guidance lives in tool plugins,
|
||||
* never in the deployment persona.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-tool-workflow
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolCallView, ToolResultView } from '@deepseek-ai/dsh-tools'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { WorkflowResult, WorkflowRun } from '@deepseek-ai/dsh-workflow'
|
||||
// Declaration merge only: makes ctx.systemPrompt visible for the section registration.
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
|
||||
export const name = 'tool-workflow'
|
||||
export const inject = ['tools', 'workflows', 'systemPrompt']
|
||||
|
||||
/** Config: the model-facing tool name plus result rendering caps. */
|
||||
export interface Config {
|
||||
/** The model-facing tool name to register (default `workflow`). */
|
||||
toolName?: string
|
||||
/** Rendered-result ceiling, in characters: a longer JSON value is truncated with a notice (default 50000). */
|
||||
maxResultChars?: number
|
||||
}
|
||||
|
||||
export const Config: z<Config> = z.object({
|
||||
toolName: z.string().default('workflow'),
|
||||
maxResultChars: z.natural().min(1).default(50_000),
|
||||
})
|
||||
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/**
|
||||
* The script-authoring contract, embedded in the tool description. This IS the
|
||||
* model-facing spec: the meta block, the hooks and their exact semantics, and
|
||||
* the supported schema subset.
|
||||
*/
|
||||
const DESCRIPTION = `Run a JavaScript workflow script that orchestrates subagents at scale. Use this for work that fans out across many independent pieces — an audit over many files, a migration, multi-angle research, adversarial verification of findings — where you write the orchestration as a script instead of delegating turn by turn.
|
||||
|
||||
The workflow's identity rides the \`meta\` parameter as JSON: required \`name\` (short kebab-case) and \`description\` strings, optional \`whenToUse\` string and \`phases\` array (\`{title, detail?, model?}\`). The \`script\` parameter is the plain JavaScript body ONLY (NOT TypeScript, and NO \`export const meta\` statement — meta is a parameter, not code), running with top-level await; end with \`return <value>\` — the value must be JSON-serializable and is this tool's result.
|
||||
|
||||
Script-body hooks:
|
||||
- \`agent(prompt, opts?): Promise<any>\` — run one subagent to completion. Without \`opts.schema\` it resolves to the child's final text; with \`opts.schema\` (an object-rooted JSON Schema using ONLY type/properties/required/additionalProperties/items/enum/const — no oneOf/pattern/format/numeric bounds) it resolves to the validated object. Resolves \`null\` when the child fails (filter with \`.filter(Boolean)\`). Other opts: \`label\` (display), \`phase\` (progress group), \`model\` (override). Anything else (\`effort\`/\`isolation\`/\`agentType\`) is rejected loudly.
|
||||
- \`pipeline(items, ...stages): Promise<any[]>\` — run each item through the stages independently with NO barrier between stages (prefer this for multi-stage work). Each stage receives \`(prev, item, index)\`. An ordinary stage throw drops that ITEM to \`null\` and skips its remaining stages.
|
||||
- \`parallel(thunks): Promise<any[]>\` — run zero-argument functions concurrently and await ALL of them (a barrier; use only when a stage genuinely needs every prior result together). A throwing thunk resolves to \`null\`.
|
||||
- \`phase(title)\` — start a progress phase; \`log(message)\` — narrate progress; \`args\` — the tool call's \`args\` input, verbatim.
|
||||
|
||||
Misused hooks (bad arguments, unknown options, unsupported schemas, tripped caps) throw errors that ALWAYS kill the script — they never dissolve into a per-item \`null\`.
|
||||
|
||||
Constraints: concurrency and total-agent caps apply; no filesystem, network, timers, or Node.js APIs are provided — the agents do the work, the script only coordinates them. The run executes in the foreground: this call returns when the whole script finishes.`
|
||||
|
||||
type WorkflowCallArgs = {
|
||||
script: string
|
||||
meta: { name: string; description: string; whenToUse?: string; phases?: { title: string; detail?: string; model?: string }[] }
|
||||
args?: Record<string, unknown>
|
||||
}
|
||||
|
||||
/** The pending-state card: a generic card titled by the workflow's meta name. */
|
||||
function presentWorkflowCall(args: WorkflowCallArgs): ToolCallView {
|
||||
return {
|
||||
card: 'generic',
|
||||
title: `workflow: ${args.meta.name}`,
|
||||
rawInput: args.script,
|
||||
}
|
||||
}
|
||||
|
||||
/** The completed-state card: keep the pending title; render the result content as-is. */
|
||||
function presentWorkflowResult(args: WorkflowCallArgs, result: { content: ContentBlock[]; isError: boolean }): ToolResultView {
|
||||
void args
|
||||
void result
|
||||
return { card: 'generic' }
|
||||
}
|
||||
|
||||
/** A non-`completed` stop reason means the script did not finish cleanly. */
|
||||
function stopReasonError(result: WorkflowResult): string | undefined {
|
||||
switch (result.stopReason) {
|
||||
case 'completed':
|
||||
return undefined
|
||||
case 'cancelled':
|
||||
return `workflow run was cancelled${result.error !== undefined ? ` (${result.error})` : ''}`
|
||||
case 'error':
|
||||
return `workflow run failed: ${result.error ?? 'unknown error'}`
|
||||
/* v8 ignore start -- defensive: WorkflowStopReason is a closed union, exhaustive by construction; a future variant fails here loudly */
|
||||
default:
|
||||
return `workflow run ended abnormally (${String(result.stopReason satisfies never)})`
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
|
||||
/** Render the run's outcome text: the meta name, agent count, and the JSON value (capped). */
|
||||
function renderResult(run: WorkflowRun, result: WorkflowResult, maxChars: number): string {
|
||||
// The engine returns JSON data (null for a valueless script), so stringify never yields undefined.
|
||||
const rendered = JSON.stringify(result.value, null, 2)
|
||||
const clipped = rendered.length > maxChars
|
||||
? `${rendered.slice(0, maxChars)}\n… [truncated: ${rendered.length - maxChars} more characters]`
|
||||
: rendered
|
||||
return `workflow "${run.meta.name}" completed (${result.agentsStarted} agent${result.agentsStarted === 1 ? '' : 's'}).\nReturn value:\n${clipped}`
|
||||
}
|
||||
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
// schemastery (the exported Config schema) has already filled the defaulted
|
||||
// fields; the assertion records that resolution, not a hidden fallback.
|
||||
const { toolName, maxResultChars } = config as ResolvedConfig
|
||||
// Usage policy ships with the tool (the master convention: tool guidance
|
||||
// lives in tool plugins as prompt sections, not in the deployment persona).
|
||||
ctx.systemPrompt.section({
|
||||
name: `tool:${toolName}`,
|
||||
order: 115,
|
||||
text: `Use the ${toolName} tool ONLY when the user explicitly asks for a workflow or for large multi-agent orchestration: you write a JavaScript script (the tool description documents the exact format) that fans work out across many subagents with phases and structured results. For one or two delegations, prefer plain subagent calls.`,
|
||||
})
|
||||
ctx.tools.register(defineTool({
|
||||
name: toolName,
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
script: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
description: 'The plain-JS workflow script body (top-level await allowed; NO `export const meta` statement; end with `return <json-value>`).',
|
||||
},
|
||||
meta: {
|
||||
type: 'object',
|
||||
required: true,
|
||||
description: 'The workflow identity block (plain JSON — never code).',
|
||||
properties: {
|
||||
name: { type: 'string', required: true, description: 'Short kebab-case workflow name.' },
|
||||
description: { type: 'string', required: true, description: 'One-line description of what the workflow does.' },
|
||||
whenToUse: { type: 'string', description: 'Optional guidance on when this workflow applies.' },
|
||||
phases: {
|
||||
type: 'array',
|
||||
description: 'Optional phase declarations matched by phase() calls.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
title: { type: 'string', required: true, description: 'The phase title phase() calls match by exact string.' },
|
||||
detail: { type: 'string', description: 'Optional one-line description of the phase.' },
|
||||
model: { type: 'string', description: 'Optional model override this phase is expected to use.' },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
args: {
|
||||
type: 'object',
|
||||
description: 'Optional JSON input exposed to the script as the `args` global (wrap a bare list as a field, e.g. {"files": [...]}).',
|
||||
},
|
||||
},
|
||||
async execute(args, exec): Promise<ContentBlock[]> {
|
||||
const parent = exec.agent
|
||||
if (!parent) {
|
||||
// The loop sets `exec.agent` for every model-driven call; its absence
|
||||
// means a non-agent caller invoked the tool directly, which has no
|
||||
// parent to attribute the children to. Fail loud rather than guess.
|
||||
throw new Error('workflow tool requires a calling agent (exec.agent was undefined)')
|
||||
}
|
||||
|
||||
// Meta/body validation failures (META_INVALID/SCRIPT_PARSE) throw
|
||||
// synchronously here and become isError results via the registry — the
|
||||
// model sees the violation list and can correct the call.
|
||||
const run: WorkflowRun = ctx.workflows.start({
|
||||
script: args.script,
|
||||
meta: args.meta,
|
||||
...args.args !== undefined ? { args: args.args } : {},
|
||||
parent,
|
||||
...exec.signal ? { signal: exec.signal } : {},
|
||||
})
|
||||
|
||||
// Bridge the tool's abort signal to the run: if the parent step is
|
||||
// aborted while the script is in flight, cancel the whole run. The
|
||||
// engine also receives `signal` directly, but an explicit bridge keeps
|
||||
// the tool's contract local (and covers an engine that ignores it).
|
||||
const onAbort = (): void => { run.cancel('parent step aborted') }
|
||||
exec.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
// `addEventListener` does NOT fire for a signal already aborted before
|
||||
// this line — cancel explicitly in that case.
|
||||
if (exec.signal?.aborted) run.cancel('parent step aborted')
|
||||
|
||||
try {
|
||||
const result = await run.result
|
||||
const error = stopReasonError(result)
|
||||
if (error !== undefined) {
|
||||
// Map a non-clean finish to an isError result (the registry turns a
|
||||
// throw into an isError). Report the reason, not partial output.
|
||||
throw new Error(error)
|
||||
}
|
||||
return [{ type: 'text', text: renderResult(run, result, maxResultChars) }]
|
||||
} finally {
|
||||
exec.signal?.removeEventListener('abort', onAbort)
|
||||
// Always reach run quiescence — never leak a live script or children.
|
||||
await run.dispose()
|
||||
}
|
||||
},
|
||||
presentCall: args => presentWorkflowCall(args),
|
||||
presentResult: (args, result) => presentWorkflowResult(args, result),
|
||||
}))
|
||||
}
|
||||
254
packages/workflow/tool-workflow/tests/tool-workflow.spec.ts
Normal file
254
packages/workflow/tool-workflow/tests/tool-workflow.spec.ts
Normal file
@@ -0,0 +1,254 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolExecutionResult } from '@deepseek-ai/dsh-tools'
|
||||
import { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { WorkflowRunId, WorkflowService } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowResult, WorkflowRun, WorkflowStartRequest } from '@deepseek-ai/dsh-workflow'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import WorkerWorkflowEngine from '@deepseek-ai/dsh-workflow-workerthread'
|
||||
import * as toolWorkflow from '../src/index.ts'
|
||||
|
||||
/** A controllable engine standing in behind ctx.workflows (the tool's only seam). */
|
||||
class StubEngine extends WorkflowService {
|
||||
requests: WorkflowStartRequest[] = []
|
||||
cancels: string[] = []
|
||||
disposed = 0
|
||||
settle!: (result: WorkflowResult) => void
|
||||
startError: Error | undefined
|
||||
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
if (this.startError) throw this.startError
|
||||
this.requests.push(request)
|
||||
const result = new Promise<WorkflowResult>((resolve) => { this.settle = resolve })
|
||||
request.signal?.addEventListener('abort', () => {
|
||||
this.settle({ value: null, stopReason: 'cancelled', error: 'signal', agentsStarted: 0 })
|
||||
}, { once: true })
|
||||
return {
|
||||
id: WorkflowRunId('run-1'),
|
||||
meta: { name: 'stub-flow', description: 'd' },
|
||||
result,
|
||||
cancel: (reason?: string) => {
|
||||
this.cancels.push(reason ?? 'cancelled')
|
||||
this.settle({ value: null, stopReason: 'cancelled', ...reason !== undefined ? { error: reason } : {}, agentsStarted: 0 })
|
||||
},
|
||||
dispose: () => {
|
||||
this.disposed += 1
|
||||
return Promise.resolve()
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function setup(config?: { toolName?: string; maxResultChars?: number }) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(StubEngine)
|
||||
await ctx.plugin(toolWorkflow, config ?? {})
|
||||
const engine = ctx.workflows as StubEngine
|
||||
const parent = { id: AgentId('caller'), options: {} } as unknown as Agent
|
||||
return { ctx, engine, parent }
|
||||
}
|
||||
|
||||
const SCRIPT = 'return 1'
|
||||
const META = { name: 'audit', description: 'd' }
|
||||
|
||||
function execute(ctx: Context, args: unknown, extra?: { agent?: Agent; signal?: AbortSignal }): Promise<ToolExecutionResult> {
|
||||
return ctx.tools.execute({
|
||||
callId: CallId('call-1'),
|
||||
name: 'workflow',
|
||||
arguments: args,
|
||||
...extra?.agent ? { agent: extra.agent } : {},
|
||||
...extra?.signal ? { signal: extra.signal } : {},
|
||||
})
|
||||
}
|
||||
|
||||
describe('dsh-tool-workflow', () => {
|
||||
it('starts a run with the script/args/parent/signal and renders the completed value', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const controller = new AbortController()
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META, args: { files: ['a.ts'] } }, { agent: parent, signal: controller.signal })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
expect(engine.requests[0]).toMatchObject({ script: SCRIPT, meta: META, args: { files: ['a.ts'] }, parent })
|
||||
expect(engine.requests[0]!.signal).toBe(controller.signal)
|
||||
engine.settle({ value: { findings: [1, 2] }, stopReason: 'completed', agentsStarted: 7 })
|
||||
const result = await pending
|
||||
expect(result.isError).toBe(false)
|
||||
const rendered = (result.content[0] as { text: string }).text
|
||||
expect(rendered).toContain('workflow "stub-flow" completed (7 agents)')
|
||||
expect(rendered).toContain('"findings"')
|
||||
expect(engine.disposed).toBe(1)
|
||||
})
|
||||
|
||||
it('maps a non-completed stop reason to an isError result (and still disposes)', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
engine.settle({ value: null, stopReason: 'error', error: 'script threw: boom', agentsStarted: 2 })
|
||||
const result = await pending
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('workflow run failed: script threw: boom')
|
||||
expect(engine.disposed).toBe(1)
|
||||
})
|
||||
|
||||
it('reports a cancelled run distinctly (with and without a reason)', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
engine.settle({ value: null, stopReason: 'cancelled', error: 'user', agentsStarted: 0 })
|
||||
const result = await pending
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('workflow run was cancelled (user)')
|
||||
|
||||
const bare = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(2) })
|
||||
engine.settle({ value: null, stopReason: 'cancelled', agentsStarted: 0 })
|
||||
expect(((await bare).content[0] as { text: string }).text.trim().endsWith('cancelled')).toBe(true)
|
||||
})
|
||||
|
||||
it('an error result without a message renders the unknown-error fallback', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
engine.settle({ value: null, stopReason: 'error', agentsStarted: 0 })
|
||||
expect(((await pending).content[0] as { text: string }).text).toContain('unknown error')
|
||||
})
|
||||
|
||||
it('cancels the run when exec.signal aborts MID-FLIGHT (the abort bridge)', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const controller = new AbortController()
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent, signal: controller.signal })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
controller.abort()
|
||||
const result = await pending
|
||||
expect(result.isError).toBe(true)
|
||||
expect(engine.cancels).toContain('parent step aborted')
|
||||
expect(engine.disposed).toBe(1)
|
||||
})
|
||||
|
||||
it('a synchronous engine start throw (meta/parse failure) becomes an isError result', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
engine.startError = new Error('invalid meta: meta.name must be a non-empty string')
|
||||
const result = await execute(ctx, { script: 'nope', meta: { name: '', description: 'd' } }, { agent: parent })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('meta.name must be a non-empty string')
|
||||
})
|
||||
|
||||
it('requires a calling agent (fails loud without exec.agent)', async () => {
|
||||
const { ctx, engine } = await setup()
|
||||
const result = await execute(ctx, { script: SCRIPT, meta: META })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('requires a calling agent')
|
||||
expect(engine.requests.length).toBe(0)
|
||||
})
|
||||
|
||||
it('validates its own arguments via the schema DSL (missing script)', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
const result = await execute(ctx, {}, { agent: parent })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(result.error?.code).toBe('INVALID_ARGS')
|
||||
})
|
||||
|
||||
it('cancels the run when exec.signal is ALREADY aborted at call time', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
const result = await execute(ctx, { script: SCRIPT, meta: META }, { agent: parent, signal: controller.signal })
|
||||
expect(result.isError).toBe(true)
|
||||
expect(engine.cancels).toContain('parent step aborted')
|
||||
expect(engine.disposed).toBe(1)
|
||||
})
|
||||
|
||||
it('truncates an oversized rendered value with a notice (maxResultChars)', async () => {
|
||||
const { ctx, engine, parent } = await setup({ maxResultChars: 40 })
|
||||
const pending = execute(ctx, { script: SCRIPT, meta: META }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBe(1) })
|
||||
engine.settle({ value: { blob: 'x'.repeat(500) }, stopReason: 'completed', agentsStarted: 1 })
|
||||
const rendered = ((await pending).content[0] as { text: string }).text
|
||||
expect(rendered).toContain('[truncated:')
|
||||
expect(rendered.length).toBeLessThan(400)
|
||||
})
|
||||
|
||||
it('registers under a configured toolName and unregisters on fiber dispose (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(StubEngine)
|
||||
const fiber = await ctx.plugin(toolWorkflow, { toolName: 'orchestrate' })
|
||||
expect(ctx.tools.get('orchestrate')).toBeDefined()
|
||||
expect(ctx.tools.get('workflow')).toBeUndefined()
|
||||
// The usage-policy prompt section rides the same registration: present
|
||||
// under the CONFIGURED name (its guidance names the tool it describes)…
|
||||
const sections = (await ctx.systemPrompt.assemble()).sections
|
||||
const section = sections.find(s => s.name === 'tool:orchestrate')
|
||||
expect(section?.text).toContain('orchestrate')
|
||||
expect(sections.some(s => s.name === 'tool:workflow')).toBe(false)
|
||||
await fiber.dispose()
|
||||
expect(ctx.tools.get('orchestrate')).toBeUndefined()
|
||||
// …and gone with the fiber — a reload must not leak a stale section.
|
||||
expect((await ctx.systemPrompt.assemble()).sections.some(s => s.name === 'tool:orchestrate')).toBe(false)
|
||||
})
|
||||
|
||||
it('presents a generic pending card titled by the meta name, with the script as rawInput', async () => {
|
||||
const { ctx } = await setup()
|
||||
const tool = ctx.tools.get('workflow')!
|
||||
const view = tool.presentCall!({ script: SCRIPT, meta: META })
|
||||
expect(view).toMatchObject({ card: 'generic', title: 'workflow: audit', rawInput: SCRIPT })
|
||||
})
|
||||
|
||||
it('presentResult keeps the generic card; presentation is pure and replay-safe on malformed args', async () => {
|
||||
const { ctx } = await setup()
|
||||
const tool = ctx.tools.get('workflow')!
|
||||
expect(tool.presentResult!({ script: SCRIPT, meta: META }, { content: [], isError: false })).toEqual({ card: 'generic' })
|
||||
// defineTool soft-validates presentation args: a malformed logged shape
|
||||
// (wrong fields entirely, or a call missing its meta) falls back to
|
||||
// undefined instead of throwing mid-replay.
|
||||
expect(tool.presentCall!({ not: 'the schema' })).toBeUndefined()
|
||||
expect(tool.presentCall!({ script: SCRIPT })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('has the namespace-plugin export shape (no stray default)', () => {
|
||||
expect('default' in toolWorkflow).toBe(false)
|
||||
expect(toolWorkflow.name).toBe('tool-workflow')
|
||||
expect(toolWorkflow.inject).toEqual(['tools', 'workflows', 'systemPrompt'])
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolWorkflow) as Record<string, unknown>
|
||||
expect(unwrapped).toBe(toolWorkflow)
|
||||
expect(typeof unwrapped.apply).toBe('function')
|
||||
})
|
||||
|
||||
describe('composition with the REAL worker-thread engine (the mock above must stay honest)', () => {
|
||||
it('an abort releases the tool even when the script parks on a promise no hook owns', async () => {
|
||||
// Regression for the review-found turn wedge: the tool awaits
|
||||
// run.result BEFORE its disposing finally, the registry and the loop
|
||||
// await the tool — so if cancellation could not settle result (a script
|
||||
// parked on `await new Promise(() => {})`), an aborted turn stayed
|
||||
// wedged forever. The seam now guarantees result settles within the
|
||||
// grace of cancel(); this drives that guarantee through the real
|
||||
// registry + real tool + real engine.
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { disposeGraceMs: 30 })
|
||||
await ctx.plugin(toolWorkflow, {})
|
||||
const parent = { id: AgentId('caller'), options: {} } as unknown as Agent
|
||||
const controller = new AbortController()
|
||||
const pending = execute(ctx, {
|
||||
script: 'await new Promise(() => {})\nreturn 1',
|
||||
meta: { name: 'stuck', description: 'parks forever' },
|
||||
}, { agent: parent, signal: controller.signal })
|
||||
// Give the run a beat to start (past its synchronous slice), then abort.
|
||||
await new Promise(resolve => setTimeout(resolve, 20))
|
||||
controller.abort('user abort')
|
||||
const result = await pending
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('cancelled')
|
||||
})
|
||||
})
|
||||
})
|
||||
36
packages/workflow/tool-workflow/tsconfig.json
Normal file
36
packages/workflow/tool-workflow/tsconfig.json
Normal file
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
},
|
||||
{
|
||||
"path": "../../core/system-prompt"
|
||||
},
|
||||
{
|
||||
"path": "../workflow"
|
||||
}
|
||||
]
|
||||
}
|
||||
49
packages/workflow/workflow-workerthread/README.md
Normal file
49
packages/workflow/workflow-workerthread/README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# @deepseek-ai/dsh-workflow-workerthread
|
||||
|
||||
The [`WorkflowService`](../workflow/README.md) implementation, on **`node:worker_threads`**: each run gets its OWN worker thread (one run = one worker, no pooling — a run is heavyweight, so the ~tens-of-ms thread spin-up is noise), the script executes in a vm context INSIDE that worker with the workflow hooks injected, and every `agent()` call bridges back over the message port to [`ctx.subagents`](../../subagent/README.md) on the host. Child agents are I/O-bound LLM loops and stay on the host event loop; the thread isolates the SCRIPT, the only part that can spin synchronously.
|
||||
|
||||
## Trust premise: what the thread buys (and what it does not)
|
||||
|
||||
Workflow scripts are **model-written** — the same trust level as the model's existing bash access — so this engine defends against **buggy** scripts, never hostile ones. A worker thread is NOT a security boundary: the vm context inside it is escapable by construction (`node:vm` shares object machinery with its surrounding realm, so a script can reach the `Function` constructor via `globalThis.constructor.constructor` and from it `process` and every Node builtin), and an escapee holds the same process privileges as the host — Node's permission model is process-wide. The absent globals are API surface that keeps honest scripts portable, not walls. What the thread concretely buys:
|
||||
|
||||
- **The host never blocks**: `start()` returns without running any script code on the host; a synchronous spin anywhere in the script occupies the worker's loop, not the harness's.
|
||||
- **Termination is real**: a script that outlives its post-cancel grace is `worker.terminate()`d — nothing of it survives `dispose()`, where an in-process engine could only abandon the spin on its own loop.
|
||||
- **No ambient credentials**: the worker spawns with an EMPTY environment (`env: {}` plus hermetic `execArgv`, the same stance as `dsh-code-runtime-worker`; the unbuilt dev shape forwards exactly one loader variable, `TSX_TSCONFIG_PATH` — path plumbing, not a secret), so an escapee reading `process.env` finds no harness secrets — ambient-channel hardening only; the process-wide privileges above (fs and the rest) remain, so a genuine sandbox is still the engine swap.
|
||||
- **Serialization by construction**: everything crossing the thread is structured-clone data, and plain JSON before that — the `materializeFromRealm` walk rejects loud what JSON cannot carry, which is also what makes every postMessage hop total.
|
||||
|
||||
What the seam guarantees regardless, because benign scripts hit these constantly: `result` never rejects, a dropped hook promise never becomes an unhandled rejection, values JSON cannot carry are rejected **loud** instead of silently mangled, and hook misuse is fatal instead of dissolving into a per-item `null`. Genuine sandboxing (containing what an escaped script may touch) remains an isolated-vm/separate-process engine swap behind the seam, still deferred.
|
||||
|
||||
## The script contract it executes
|
||||
|
||||
- **Meta as DATA** (`validateMeta`, host-side): the workflow's identity arrives on the start request as plain JSON (the tool carries it as its schema-validated `meta` parameter — never as script text) and is shape-validated loud, every violation named (`name`/`description` required; unknown fields rejected). The engine deliberately evaluates NO script text to obtain meta: an evaluated meta literal could smuggle getters that run on the host outside any vm timeout — the exact spin the worker thread exists to isolate. A body that still opens with a Claude Code-style `export const meta` statement is rejected with a pointed `SCRIPT_PARSE` message.
|
||||
- **Hooks**: `agent(prompt, {label, phase, schema, model})` (schema = the [structured-output subset](../../core/tools/README.md), forwarded as `outputSchema`; result = validated object, or final text without a schema; a failed child resolves `null`), `parallel(thunks)`, `pipeline(items, ...stages)` with NO cross-stage barrier and `(prev, item, index)` stage callbacks, `phase(title)`, `log(message)`, and the `args` global. Anything else — `effort`/`isolation`/`agentType`, unknown options, malformed arguments, schemas outside the subset — throws a FATAL `WorkflowError` that `parallel`/`pipeline` re-throw rather than nulling (see the seam README's failure discipline).
|
||||
- **No ambient APIs**: no timers, filesystem, or Node APIs are injected into the context (absence is API surface, not containment — see the trust premise).
|
||||
|
||||
## How a run executes
|
||||
|
||||
`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (`src/worker.ts` unbuilt via an explicit tsx `execArgv`; the sibling `lib/worker.js` bundle when built) with the meta, body, `args`, and worker-side limits as `workerData`.
|
||||
|
||||
Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**: `agent()` sends `child-start` and the host starts the child on `ctx.subagents` (parent attribution, the shared per-run abort signal, `outputSchema`/`model` pass-through), replying with the child id, its settlement (a JSON projection; an infrastructure REJECTION crosses as `child-failed` and stays the fatal `AGENT_RESULT`), and dispose acks. Observer narration (`phase`/`log`/`agent-start`/`agent-end`) crosses as messages and re-emits as the seam's `workflow/*` events. A **ready→go handshake** gates the body: a cancellation racing worker boot arrives before `go`, so a run cancelled before start never executes the body at all.
|
||||
|
||||
## The value boundary
|
||||
|
||||
Values LEAVING the script (hook options/schemas, the script's return) are materialized by `materializeFromRealm`: a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested `undefined`), copying into plain containers via `defineProperty` so a `"__proto__"` key becomes a data property, never a prototype mutation. Getters are read ordinarily — the RESULT is what crosses; a read that throws fails loud; both run in the WORKER, never on the host. Values ENTERING the realm (`args`, `agent()` results, hook promises and their failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; `args` is cloned once at start so a script scribbling on it cannot mutate the caller's object. One script-visible consequence: an error thrown by a hook is built OUTSIDE the script's vm context, so `e instanceof Error` inside the script is `false` — branch on `e.name`/`e.code` instead (the combinators recognize fatality by `instanceof` against their own realm's class, which a script-built object can never pass, so fatal-vs-null cannot be forged or dissolved).
|
||||
|
||||
## Cancellation, death, disposal
|
||||
|
||||
Per-run limits: a concurrency semaphore (`maxConcurrentAgents`), a total-`agent()` cap (`maxTotalAgents`), and a per-call item cap (`maxItemsPerCall`), all config. `cancel()` posts the cancel to the worker (its hooks start throwing `CANCELLED`; the script dies at its next await) and cancels every host-side child NOW on **both seam channels** — the shared request signal aborts AND each registered child's explicit `cancel()` is called host-side, because the seam leaves a provider free to honor either channel and a worker wedged in a synchronous spin could not relay its own per-child cancel RPCs (those later land as idempotent no-ops). The grace then arms: a run still unsettled `disposeGraceMs` later force-settles `cancelled` and the worker is **terminated**. A cancellation that lands before the body runs (the ready→go handshake) reports `cancelled` without executing anything; a worker `result` racing an in-flight host cancellation reports `cancelled` too (first-wins settlement — the seam-visible result had not settled when cancellation was requested); post-cancel `phase`/`log` narration is suppressed host-side, while cancelled children still deliver their paired `agent-end`.
|
||||
|
||||
A worker that dies unexpectedly (an OOM, a script reaching `process.exit` through the documented vm escape) settles the run `stopReason: 'error'` with the exit diagnostics — or `'cancelled'` when a cancel was in flight — and the host-side child registry is what winds every surviving child down. `dispose()` = cancel + immediate host-driven disposal of every registered child (a wedged worker can relay no dispose RPC, so child teardown overlaps the grace instead of starting after it; the worker's own dispose RPCs join the same per-child disposal) + bounded wait (result, then child-registry quiescence, capped by the grace) + unconditional `worker.terminate()`: the thread never outlives its run. Once a run settles, stray children a script fired without awaiting are cancelled too, and `dispose()` waits for their disposal (bounded by the grace) before returning. `agent-start`/`agent-end` pairing is host-guaranteed the same way: forwarded starts live in a ledger, worker-reported ends pair them on the graceful paths, and the termination paths (grace force-settle, worker death) synthesize the missing ends (outcome `cancelled`) before the run settles — a start still in flight across the force-settle can surface after `workflow/end`, immediately paired the same way.
|
||||
|
||||
**Engine-specific limitations**: worker startup is paid per run; on a termination path `agentsStarted` reports the HOST-observed count (accepted `child-start`s — calls still queued worker-side for a concurrency slot are unknowable then); and a returned promise or thenable resolves per JavaScript semantics BEFORE materialization — that is what makes an un-awaited `return agent('x')` work — with the value-boundary guard applying to the resolution.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `provider` | `spawn` | The `ctx.subagents` provider children run on (host-side). |
|
||||
| `maxConcurrentAgents` | `0` (auto) | Concurrent `agent()` ceiling; `0` resolves to `min(16, max(1, cores - 2))`. |
|
||||
| `maxTotalAgents` | `1000` | Total `agent()` calls one run may start (runaway-loop backstop). |
|
||||
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()`/`pipeline()` call. |
|
||||
| `syncTimeoutMs` | `5000` | vm timeout for the script's initial synchronous slice (in the worker). |
|
||||
| `disposeGraceMs` | `5000` | How long a cancelled run may stay unsettled before force-settle + terminate; also bounds `dispose()`. |
|
||||
56
packages/workflow/workflow-workerthread/package.json
Normal file
56
packages/workflow/workflow-workerthread/package.json
Normal file
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-workflow-workerthread",
|
||||
"description": "worker-thread workflow engine: executes model-written orchestration scripts off the host event loop, bridging agent() calls back to ctx.subagents",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./worker": {
|
||||
"types": "./lib/types/worker.d.ts",
|
||||
"default": "./lib/worker.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/worker.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-subagent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"@deepseek-ai/dsh-workflow": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent-spawn": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6",
|
||||
"tsx": "^4.19.2"
|
||||
}
|
||||
}
|
||||
473
packages/workflow/workflow-workerthread/src/host.ts
Normal file
473
packages/workflow/workflow-workerthread/src/host.ts
Normal file
@@ -0,0 +1,473 @@
|
||||
/**
|
||||
* The host half of one worker-engine run: spawn the Worker, bridge its child
|
||||
* RPC onto `ctx.subagents`, fan its observer messages into the engine's
|
||||
* events, and own cancellation, the settle-within-grace guarantee, and child
|
||||
* cleanup. The worker's lifetime IS the run's lifetime: `dispose()` always
|
||||
* ends with `worker.terminate()`, so no thread outlives its run.
|
||||
*
|
||||
* The run's `result` promise settles exactly once, from whichever of these
|
||||
* lands first: the worker's `result` message (a host-side cancellation in
|
||||
* flight overrides a non-cancelled report — the seam-visible result had not
|
||||
* settled when cancellation was requested), an unexpected worker death
|
||||
* (`error`/`messageerror`/premature `exit` → `stopReason: 'error'`, or
|
||||
* `'cancelled'` when a cancel was in flight), or the post-cancel grace timer
|
||||
* (a script that never settles is force-settled `cancelled` and its worker
|
||||
* terminated — the real kill an in-process engine could not perform).
|
||||
*
|
||||
* Children live in a host-side registry (callId → run): the worker drives
|
||||
* their disposal by RPC on the graceful path, `dispose()` host-drives every
|
||||
* registered child's disposal immediately (a wedged worker can relay no
|
||||
* dispose RPC, and child teardown must overlap the grace, not start after
|
||||
* it), and the registry is what lets the host abort and dispose every
|
||||
* survivor when the worker dies or is terminated mid-flight. The three
|
||||
* paths share ONE disposal per child (memoized by callId; the seam's
|
||||
* dispose() is idempotent anyway, the memo keeps the bookkeeping and the
|
||||
* containment warn single). Lifecycle pairing is host-guaranteed the same
|
||||
* way: every forwarded `agent-start` lives in a ledger, and a start the
|
||||
* dead or terminated worker never paired is closed by a synthesized
|
||||
* `agent-end` (outcome `'cancelled'`) before the run settles. On a
|
||||
* termination path `agentsStarted` reports the
|
||||
* HOST-observed count (accepted `child-start` messages) — `agent()` calls
|
||||
* still queued worker-side for a concurrency slot are unknowable then; the
|
||||
* worker's own count rides the result message on every graceful path.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/host
|
||||
*/
|
||||
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import type { WorkerOptions } from 'node:worker_threads'
|
||||
import type { Context } from 'cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import type { SubagentRun } from '@deepseek-ai/dsh-subagent'
|
||||
import type { WorkflowAgentEndInfo, WorkflowAgentInfo, WorkflowMeta, WorkflowResult, WorkflowRun, WorkflowRunId } from '@deepseek-ai/dsh-workflow'
|
||||
import { renderThrown } from './realm.ts'
|
||||
import type { ExecutionObserver } from './runtime.ts'
|
||||
import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
|
||||
import type { HostToWorkerPayloads, WorkerToHostMessage } from './protocol.ts'
|
||||
import type { ChildStartRequest, WorkerInit } from './types.ts'
|
||||
|
||||
/**
|
||||
* Resolve the worker entry and spawn options for the current runtime shape.
|
||||
* Unbuilt (tsx demos, vitest — `import.meta.url` points into `src/`), the
|
||||
* entry is the TypeScript sibling and the worker needs the tsx loader
|
||||
* registered explicitly: a worker thread inherits no transform pipeline from
|
||||
* vitest (vite transforms in-process, not via a node loader), and passing
|
||||
* execArgv explicitly also shields the worker from any loader flags the
|
||||
* parent was started with. Built (`lib/index.js`), the entry is the sibling
|
||||
* bundle the package tsdown config emits and no loader is needed (execArgv
|
||||
* pinned empty — hermetic, like the environment).
|
||||
*
|
||||
* Both shapes spawn with an EMPTY environment (`env: {}`): the documented vm
|
||||
* escape reaches `process`, and the harness's ambient credentials
|
||||
* (`DEEPSEEK_API_KEY` et al.) must not ride along — the same stance as
|
||||
* `dsh-code-runtime-worker`, stronger than the scrubbed env the
|
||||
* defensive-patterns rule requires for spawned commands (a shell needs PATH;
|
||||
* this worker needs nothing). Sole exception: the unbuilt shape forwards
|
||||
* `TSX_TSCONFIG_PATH` when the parent carries it (loader plumbing the paths
|
||||
* map depends on outside the repo cwd, not a secret). This closes the
|
||||
* AMBIENT channel only — an escapee still holds process-wide privileges
|
||||
* like fs access (the README's trust premise stands).
|
||||
* @param init - the run payload, passed as `workerData`.
|
||||
* @returns the entry URL and the Worker options to spawn it with.
|
||||
*/
|
||||
function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOptions } {
|
||||
/* v8 ignore next 3 -- the built-output arm: tests always run unbuilt (src/); the built-worker e2e exercises this shape for real */
|
||||
if (!import.meta.url.endsWith('.ts')) {
|
||||
return { entry: new URL('./worker.js', import.meta.url), options: { workerData: init, env: {}, execArgv: [] } }
|
||||
}
|
||||
// Lazy tsx resolution: only the unbuilt shape needs it, so the built
|
||||
// bundle never requires tsx to be installed. TSX_TSCONFIG_PATH is the one
|
||||
// variable forwarded through the scrub: tsx finds a tsconfig by searching
|
||||
// UP from the worker's cwd, and a parent running with its cwd outside the
|
||||
// repo (the ACP snapshot harness pins the tsconfig through this exact
|
||||
// variable) would otherwise lose the dsh-* paths map and resolve workspace
|
||||
// imports to unbuilt lib/ bundles. Loader plumbing, not a secret.
|
||||
return {
|
||||
entry: new URL('./worker.ts', import.meta.url),
|
||||
options: {
|
||||
workerData: init,
|
||||
env: process.env.TSX_TSCONFIG_PATH === undefined ? {} : { TSX_TSCONFIG_PATH: process.env.TSX_TSCONFIG_PATH },
|
||||
execArgv: ['--import', fileURLToPath(import.meta.resolve('tsx'))],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* One live worker-engine run — the seam's {@link WorkflowRun}, returned by
|
||||
* `start()` directly. Owns the Worker, the child registry, and the result
|
||||
* settlement; `result` never rejects. `meta` is this handle's OWN clone
|
||||
* (event payloads carry separate clones), so a consumer mutating it corrupts
|
||||
* nothing.
|
||||
*/
|
||||
export class WorkerRun implements WorkflowRun {
|
||||
/** Settles exactly once with the run's outcome; never rejects. */
|
||||
readonly result: Promise<WorkflowResult>
|
||||
private settleResolve!: (result: WorkflowResult) => void
|
||||
private settled = false
|
||||
private cancelReason: string | undefined
|
||||
private graceTimer: NodeJS.Timeout | undefined
|
||||
private readonly worker: Worker
|
||||
/** Set on `exit`: the thread is gone, so posting has nowhere to go. */
|
||||
private workerGone = false
|
||||
/** Accepted `child-start` messages — the terminate-path `agentsStarted` (see module doc). */
|
||||
private hostStarted = 0
|
||||
/** Live children by callId; an entry leaves ONLY after its dispose settles (quiescence = empty). */
|
||||
private readonly children = new Map<number, SubagentRun>()
|
||||
/** In-flight child disposals by callId — the memo that gives every path (worker RPC, dispose(), reap) ONE shared disposal per child. */
|
||||
private readonly childDisposals = new Map<number, Promise<void>>()
|
||||
/** Started-but-not-ended agents by seq — the pairing ledger the HOST guarantees (see {@link endAgent}). */
|
||||
private readonly liveAgents = new Map<number, WorkflowAgentInfo>()
|
||||
private readonly quiescenceWaiters: (() => void)[] = []
|
||||
/** The per-run abort fanout every child start request carries. */
|
||||
private readonly controller = new AbortController()
|
||||
private disposed: Promise<void> | undefined
|
||||
|
||||
constructor(
|
||||
private readonly ctx: Context,
|
||||
readonly id: WorkflowRunId,
|
||||
readonly meta: WorkflowMeta,
|
||||
private readonly parent: Agent,
|
||||
init: WorkerInit,
|
||||
private readonly provider: string,
|
||||
private readonly disposeGraceMs: number,
|
||||
private readonly observer: ExecutionObserver,
|
||||
signal: AbortSignal | undefined,
|
||||
) {
|
||||
this.result = new Promise<WorkflowResult>((resolve) => { this.settleResolve = resolve })
|
||||
// workerData rides the structured clone: args are plain JSON by the seam
|
||||
// contract, so the clone is total and doubles as the caller-isolation
|
||||
// copy (a clone failure throws loud out of start()).
|
||||
const { entry, options } = resolveWorkerSpawn(init)
|
||||
this.worker = new Worker(entry, options)
|
||||
this.worker.on('message', (message: WorkerToHostMessage) => { this.onMessage(message) })
|
||||
this.worker.on('error', (error) => { this.onWorkerDeath(`workflow worker failed: ${renderThrown(error)}`) })
|
||||
/* v8 ignore next -- messageerror: not constructible from the engine's own protocol (every payload is JSON data) */
|
||||
this.worker.on('messageerror', (error) => { this.onWorkerDeath(`workflow worker message failed to deserialize: ${renderThrown(error)}`) })
|
||||
this.worker.on('exit', (code) => {
|
||||
this.workerGone = true
|
||||
this.onWorkerDeath(`workflow worker exited before the run settled (exit code ${code})`)
|
||||
})
|
||||
if (signal?.aborted) {
|
||||
this.cancel('workflow start signal already aborted')
|
||||
} else {
|
||||
signal?.addEventListener('abort', () => { this.cancel('workflow signal aborted') }, { once: true })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel the run: the worker is told (its hooks start throwing and the
|
||||
* script dies at its next await), every host-side child is cancelled NOW on
|
||||
* BOTH seam channels — the shared request signal aborts and each registered
|
||||
* child's explicit `cancel()` is called (the seam leaves a provider free to
|
||||
* honor either, and a worker wedged in a synchronous spin could not relay
|
||||
* its own per-child cancel RPCs until far too late) — and the grace timer
|
||||
* arms: a run still unsettled `disposeGraceMs` later force-settles
|
||||
* `cancelled` and its worker is TERMINATED. Idempotent; the first reason
|
||||
* wins.
|
||||
* @param reason - human-readable cause (default `'workflow cancelled'`).
|
||||
*/
|
||||
cancel(reason?: string): void {
|
||||
// A settled run has nothing left to cancel: without this guard the
|
||||
// ordinary consumer path (await result, then dispose -> cancel) would arm
|
||||
// a grace timer nothing ever clears, pinning the run and its Worker
|
||||
// closure until the grace expires - a bounded leak per completed run.
|
||||
if (this.settled || this.cancelReason !== undefined) return
|
||||
this.cancelReason = reason ?? 'workflow cancelled'
|
||||
this.post(HostToWorkerType.Cancel, { reason: this.cancelReason })
|
||||
this.controller.abort(this.cancelReason)
|
||||
// The explicit channel is driven host-side, not left to the worker: a
|
||||
// provider honoring only run.cancel() must not wait on a wedged worker's
|
||||
// ChildCancel relay (those later RPCs land as idempotent no-ops).
|
||||
for (const run of this.children.values()) run.cancel(this.cancelReason)
|
||||
this.graceTimer = setTimeout(() => {
|
||||
// The worker may no longer speak (it is about to be terminated): pair
|
||||
// every stranded start before the run settles, so ends precede
|
||||
// workflow/end.
|
||||
this.endStrandedAgents()
|
||||
this.settleResult(this.cancelledResult(this.hostStarted))
|
||||
void this.worker.terminate()
|
||||
}, this.disposeGraceMs)
|
||||
// unref'd: an armed grace timer must never hold the process open.
|
||||
this.graceTimer.unref()
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel + bounded settle + termination. Host-drives every registered
|
||||
* child's disposal IMMEDIATELY — a wedged worker can relay no dispose RPC,
|
||||
* and deferring child teardown to the post-terminate reap would spend the
|
||||
* whole grace waiting for a quiescence that cannot start, then return with
|
||||
* the disposals still in flight — so child disposal overlaps the same
|
||||
* grace the worker gets to settle (the worker's own dispose RPCs join the
|
||||
* shared per-child disposal). Waits (at most the grace) for the result and
|
||||
* child quiescence, then terminates the worker unconditionally — the
|
||||
* thread never outlives its run — and reaps whatever children remain
|
||||
* (their disposal is contained, not awaited past the grace, the same
|
||||
* abandonment the seam documents for a slow-disposing child). Idempotent;
|
||||
* safe on every path.
|
||||
* @returns resolves when the run's resources are released or abandoned.
|
||||
*/
|
||||
dispose(): Promise<void> {
|
||||
this.disposed ??= (async () => {
|
||||
this.cancel('workflow disposed')
|
||||
for (const [callId, run] of [...this.children]) void this.disposeChild(callId, run)
|
||||
await Promise.race([
|
||||
(async () => {
|
||||
await this.result
|
||||
await this.childQuiescence()
|
||||
})(),
|
||||
sleep(this.disposeGraceMs),
|
||||
])
|
||||
await this.worker.terminate()
|
||||
this.reapChildren('workflow disposed')
|
||||
})()
|
||||
return this.disposed
|
||||
}
|
||||
|
||||
/** Post one message to the worker (payload looked up from the tag's map entry), tolerating a thread that is already gone. */
|
||||
private post<T extends HostToWorkerType>(type: T, payload: HostToWorkerPayloads[T]): void {
|
||||
if (this.workerGone) return
|
||||
try {
|
||||
this.worker.postMessage({ type, ...payload })
|
||||
} catch (error: unknown) {
|
||||
// Only a teardown race can land here (every engine message is JSON
|
||||
// data, so serialization cannot fail); there is nothing left to
|
||||
// deliver to — log and move on.
|
||||
/* v8 ignore next -- postMessage teardown race (a throw between exit and its event): not constructible in-process */
|
||||
this.ctx.logger.warn(`workflow-workerthread: postMessage failed: ${renderThrown(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
private onMessage(message: WorkerToHostMessage): void {
|
||||
switch (message.type) {
|
||||
case WorkerToHostType.Ready:
|
||||
this.post(HostToWorkerType.Go, {})
|
||||
break
|
||||
case WorkerToHostType.Phase:
|
||||
// Post-cancel narration is suppressed host-side: worker-side the
|
||||
// hooks throw once the cancel message is PROCESSED, but narration
|
||||
// already in flight (or emitted while the cancel crossed the
|
||||
// boundary) must not reach observers — nothing is emitted after
|
||||
// cancel() returns.
|
||||
if (this.cancelReason === undefined) this.observer.phase(message.title)
|
||||
break
|
||||
case WorkerToHostType.Log:
|
||||
if (this.cancelReason === undefined) this.observer.log(message.message)
|
||||
break
|
||||
case WorkerToHostType.AgentStart:
|
||||
this.liveAgents.set(message.info.seq, message.info)
|
||||
this.observer.agentStart(message.info)
|
||||
break
|
||||
case WorkerToHostType.AgentEnd:
|
||||
// NOT suppressed on cancel: cancelled children report their paired
|
||||
// agent-end with outcome 'cancelled'. The gate (with the termination
|
||||
// paths' synthesis) is what makes the one-pair-per-started-child
|
||||
// contract hold on every stop path.
|
||||
this.endAgent(message.info)
|
||||
break
|
||||
case WorkerToHostType.ChildStart:
|
||||
this.onChildStart(message.callId, message.request)
|
||||
break
|
||||
case WorkerToHostType.ChildCancel:
|
||||
this.children.get(message.callId)?.cancel(message.reason)
|
||||
break
|
||||
case WorkerToHostType.ChildDispose:
|
||||
this.onChildDispose(message.callId)
|
||||
break
|
||||
case WorkerToHostType.Result:
|
||||
this.onResult(message.result)
|
||||
break
|
||||
/* v8 ignore next 2 -- closed engine-owned union; the arm only makes adding a message type a compile error */
|
||||
default:
|
||||
assertNever(message, 'worker-to-host message')
|
||||
}
|
||||
}
|
||||
|
||||
private onChildStart(callId: number, request: ChildStartRequest): void {
|
||||
if (this.cancelReason !== undefined) {
|
||||
// The worker's start raced our cancel: refuse — a child must never
|
||||
// start on an already-aborted signal (a provider subscribing only to
|
||||
// future abort events would never observe it).
|
||||
this.post(HostToWorkerType.ChildStartError, { callId, rendered: `workflow run cancelled: ${this.cancelReason}` })
|
||||
return
|
||||
}
|
||||
this.hostStarted += 1
|
||||
let run: SubagentRun
|
||||
try {
|
||||
run = this.ctx.subagents.start(this.provider, {
|
||||
prompt: [{ type: 'text', text: request.prompt }],
|
||||
parent: this.parent,
|
||||
signal: this.controller.signal,
|
||||
...request.schema !== undefined ? { outputSchema: request.schema } : {},
|
||||
...request.model !== undefined ? { agentOptions: { model: request.model } } : {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
|
||||
return
|
||||
}
|
||||
this.children.set(callId, run)
|
||||
this.post(HostToWorkerType.ChildStarted, { callId, childId: run.id })
|
||||
run.result.then(
|
||||
(result) => {
|
||||
this.post(HostToWorkerType.ChildSettled, {
|
||||
callId,
|
||||
result: {
|
||||
output: result.output,
|
||||
...result.structured !== undefined ? { structured: result.structured } : {},
|
||||
stopReason: result.stopReason,
|
||||
},
|
||||
})
|
||||
},
|
||||
(error: unknown) => { this.post(HostToWorkerType.ChildFailed, { callId, rendered: renderThrown(error) }) },
|
||||
)
|
||||
}
|
||||
|
||||
private onChildDispose(callId: number): void {
|
||||
const run = this.children.get(callId)
|
||||
if (run === undefined) {
|
||||
// Already disposed host-side (a dispose() drive or a death reap beat
|
||||
// the RPC) — the ack is still owed (the worker-side wrapper awaits it).
|
||||
this.post(HostToWorkerType.ChildDisposed, { callId })
|
||||
return
|
||||
}
|
||||
// disposeChild never rejects (containment is inside), so the ack always follows.
|
||||
void this.disposeChild(callId, run).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
|
||||
}
|
||||
|
||||
/**
|
||||
* Start (or join) one registered child's disposal; the registry entry
|
||||
* leaves when it settles. Memoized per callId: the worker's dispose RPC,
|
||||
* the dispose() host drive, and the reap can all land on the same child —
|
||||
* the child's `dispose()` runs once and every caller awaits that one
|
||||
* settlement. A rejection is contained (the subagent seam's dispose() is
|
||||
* not supposed to reject, but a backend that does anyway must not break
|
||||
* quiescence): logged, and the child still leaves the registry.
|
||||
* @param callId - the child's registry key.
|
||||
* @param run - the registered child (the caller looked it up).
|
||||
* @returns resolves when the disposal settled either way; never rejects.
|
||||
*/
|
||||
private disposeChild(callId: number, run: SubagentRun): Promise<void> {
|
||||
let disposal = this.childDisposals.get(callId)
|
||||
if (disposal === undefined) {
|
||||
disposal = run.dispose().then(
|
||||
() => { this.finishChild(callId) },
|
||||
(error: unknown) => {
|
||||
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)
|
||||
this.finishChild(callId)
|
||||
},
|
||||
)
|
||||
this.childDisposals.set(callId, disposal)
|
||||
}
|
||||
return disposal
|
||||
}
|
||||
|
||||
/** Drop a child from the registry (and its disposal memo), releasing quiescence waiters at zero. */
|
||||
private finishChild(callId: number): void {
|
||||
this.children.delete(callId)
|
||||
this.childDisposals.delete(callId)
|
||||
if (this.children.size === 0) {
|
||||
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolves once the child registry is empty (every disposal settled). */
|
||||
private childQuiescence(): Promise<void> {
|
||||
if (this.children.size === 0) return Promise.resolve()
|
||||
return new Promise((resolve) => { this.quiescenceWaiters.push(resolve) })
|
||||
}
|
||||
|
||||
/** Abort + dispose every registered child (worker death / final teardown); disposal is contained, not awaited. */
|
||||
private reapChildren(reason: string): void {
|
||||
this.controller.abort(this.cancelReason ?? reason)
|
||||
for (const [callId, run] of [...this.children]) {
|
||||
run.cancel(this.cancelReason ?? reason)
|
||||
void this.disposeChild(callId, run)
|
||||
}
|
||||
}
|
||||
|
||||
private onResult(result: WorkflowResult): void {
|
||||
// The worker's settle-reap already child-cancel()s every stray; this
|
||||
// abort fires the seam signal too, for providers that only honor the
|
||||
// request signal (both channels, on every path).
|
||||
if (this.cancelReason === undefined) this.controller.abort('workflow settled')
|
||||
if (this.cancelReason !== undefined && result.stopReason !== 'cancelled') {
|
||||
// The script settled while our cancel was crossing the thread boundary
|
||||
// — the seam-visible result had NOT settled when cancellation was
|
||||
// requested, so report cancelled (the vm drive()'s post-settle check,
|
||||
// relocated to the receiving side of the race).
|
||||
this.settleResult(this.cancelledResult(result.agentsStarted))
|
||||
return
|
||||
}
|
||||
this.settleResult(result)
|
||||
}
|
||||
|
||||
/** An unexpected worker death (or the expected exit after termination). */
|
||||
private onWorkerDeath(message: string): void {
|
||||
// Whatever the worker left behind must not leak — abort + dispose it all.
|
||||
if (this.children.size > 0) this.reapChildren('workflow worker gone')
|
||||
// The thread is gone: no more worker-authored agent-ends can arrive —
|
||||
// pair every stranded start (a start that crossed between the grace
|
||||
// force-settle and this exit included) before the run settles.
|
||||
this.endStrandedAgents()
|
||||
// settleResult no-ops on an already-settled run (the expected exit after
|
||||
// a dispose's terminate lands here too).
|
||||
if (this.cancelReason !== undefined) {
|
||||
this.settleResult(this.cancelledResult(this.hostStarted))
|
||||
return
|
||||
}
|
||||
this.settleResult({ value: null, stopReason: 'error', error: message, agentsStarted: this.hostStarted })
|
||||
}
|
||||
|
||||
/**
|
||||
* The single agent-end emission gate: forwards `end` iff its start is still
|
||||
* unpaired in the ledger, so every forwarded `workflow/agent-start` gets
|
||||
* EXACTLY one `workflow/agent-end` — the worker's own report where it can
|
||||
* speak, a host-synthesized one where it cannot ({@link endStrandedAgents}).
|
||||
* @param end - the settlement to emit (worker-reported or synthesized).
|
||||
*/
|
||||
private endAgent(end: WorkflowAgentEndInfo): void {
|
||||
/* v8 ignore next -- a real end still in flight across the grace force-settle: not orderable in-process */
|
||||
if (!this.liveAgents.delete(end.seq)) return
|
||||
this.observer.agentEnd(end)
|
||||
}
|
||||
|
||||
/**
|
||||
* Synthesize the missing `agent-end` for every started-but-unpaired agent,
|
||||
* outcome `'cancelled'`: the reap cancels every child, and a real
|
||||
* settlement racing the force-settle loses to the cancellation — the same
|
||||
* first-wins override {@link onResult} applies to the run's own result.
|
||||
* Called where the worker can no longer speak (the grace force-settle,
|
||||
* worker death), BEFORE settleResult, so the paired ends reach observers
|
||||
* before `workflow/end`.
|
||||
*/
|
||||
private endStrandedAgents(): void {
|
||||
for (const info of [...this.liveAgents.values()]) {
|
||||
this.endAgent({ ...info, outcome: 'cancelled' })
|
||||
}
|
||||
}
|
||||
|
||||
private cancelledResult(agentsStarted: number): WorkflowResult {
|
||||
// cancel() is the only writer of cancelReason and every caller checks it
|
||||
// first; the fallback guards the type, not a reachable path.
|
||||
/* v8 ignore next */
|
||||
const reason = this.cancelReason ?? 'workflow cancelled'
|
||||
return { value: null, stopReason: 'cancelled', error: `workflow run cancelled: ${reason}`, agentsStarted }
|
||||
}
|
||||
|
||||
/** First settle wins; disarms the grace timer. */
|
||||
private settleResult(result: WorkflowResult): void {
|
||||
if (this.settled) return
|
||||
this.settled = true
|
||||
clearTimeout(this.graceTimer)
|
||||
this.settleResolve(result)
|
||||
}
|
||||
}
|
||||
|
||||
/** A plain timer sleep (the dispose grace); unref'd so it never holds the process open. */
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const timer = setTimeout(resolve, ms)
|
||||
timer.unref()
|
||||
})
|
||||
}
|
||||
203
packages/workflow/workflow-workerthread/src/index.ts
Normal file
203
packages/workflow/workflow-workerthread/src/index.ts
Normal file
@@ -0,0 +1,203 @@
|
||||
/**
|
||||
* The `node:worker_threads` workflow engine: the {@link WorkflowService}
|
||||
* implementation. Runs each script in its OWN worker thread (one run = one
|
||||
* worker, no pooling — a run is heavyweight, so thread spin-up is noise): the
|
||||
* body executes in a vm context INSIDE the worker with the workflow hooks
|
||||
* injected, and `agent()` calls bridge back to `ctx.subagents` over the
|
||||
* message port — child agents are I/O-bound LLM loops and stay on the host
|
||||
* event loop; the thread isolates the SCRIPT, the only part that can spin
|
||||
* synchronously.
|
||||
*
|
||||
* TRUST PREMISE: scripts are MODEL-WRITTEN — the same trust level as the
|
||||
* model's existing bash access — so this engine defends against BUGGY
|
||||
* scripts, never hostile ones. A worker thread is NOT a security boundary:
|
||||
* the vm context inside it is escapable by construction, and an escapee
|
||||
* holds the same process privileges as the host (Node's permission model is
|
||||
* process-wide); genuine sandboxing (isolated-vm, a separate process) is an
|
||||
* engine swap behind the seam. What the thread buys, concretely:
|
||||
*
|
||||
* - `start()` never blocks the host: the script's initial synchronous slice
|
||||
* (and any later synchronous spin) occupies the WORKER's event loop, not
|
||||
* the harness's.
|
||||
* - Termination is REAL: a script that outlives its post-cancel grace is
|
||||
* `worker.terminate()`d — nothing of the script survives `dispose()`,
|
||||
* where an in-process engine could only abandon the spin on its own loop.
|
||||
* - The value boundary is serialization by construction: everything crossing
|
||||
* the thread is structured-clone data (and plain JSON before that, by the
|
||||
* materialization walk in ./realm.ts).
|
||||
*
|
||||
* Engine-specific limitations: worker startup (~tens of ms) is paid per run;
|
||||
* on a termination path `agentsStarted` reports the host-observed child
|
||||
* count (calls still queued worker-side for a slot are unknowable — see
|
||||
* ./host.ts); and a worker that dies unexpectedly (an OOM, a script reaching
|
||||
* `process.exit` through the documented vm escape) settles the run
|
||||
* `stopReason: 'error'` with the exit diagnostics.
|
||||
*
|
||||
* Plugin export shape: a default-exported {@link WorkflowService} subclass
|
||||
* (the class-based service form, like `dsh-bash-local`).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { availableParallelism } from 'node:os'
|
||||
import * as vm from 'node:vm'
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import WorkflowService, { WorkflowError, WorkflowRunId } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowRun, WorkflowRunInfo, WorkflowStartRequest } from '@deepseek-ai/dsh-workflow'
|
||||
import { WorkerRun } from './host.ts'
|
||||
import { validateMeta } from './meta.ts'
|
||||
import type { WorkerInit, WorkerLimits } from './types.ts'
|
||||
|
||||
export { validateMeta } from './meta.ts'
|
||||
export { HostToWorkerType, WorkerToHostType } from './protocol.ts'
|
||||
export type { HostToWorkerMessage, HostToWorkerPayloads, WorkerToHostMessage, WorkerToHostPayloads } from './protocol.ts'
|
||||
export { materializeFromRealm, MaterializeError } from './realm.ts'
|
||||
export { WorkflowExecution, type ExecutionObserver } from './runtime.ts'
|
||||
export { requireParentPort, runWorkerSession } from './session.ts'
|
||||
export type {
|
||||
ChildHandle,
|
||||
ChildPort,
|
||||
ChildResult,
|
||||
ChildStartRequest,
|
||||
WorkerInit,
|
||||
WorkerLimits,
|
||||
} from './types.ts'
|
||||
|
||||
/** Plugin config (all optional — `static Config` supplies the defaults). */
|
||||
export interface Config {
|
||||
/** The `ctx.subagents` provider children run on (default `spawn`). */
|
||||
provider?: string
|
||||
/** Concurrent `agent()` ceiling; `0` (the default) auto-resolves to `min(16, max(1, cores - 2))`. */
|
||||
maxConcurrentAgents?: number
|
||||
/** Total `agent()` calls one run may start — the runaway-loop backstop (default 1000). */
|
||||
maxTotalAgents?: number
|
||||
/** Items accepted by a single `parallel()`/`pipeline()` call (default 4096). */
|
||||
maxItemsPerCall?: number
|
||||
/** vm timeout for the script's initial synchronous slice, inside the worker (default 5000 ms). */
|
||||
syncTimeoutMs?: number
|
||||
/**
|
||||
* How long after a cancellation an unsettled script may keep running before
|
||||
* the run force-settles `cancelled` and its worker is TERMINATED (default
|
||||
* 5000 ms); also bounds `dispose()`.
|
||||
*/
|
||||
disposeGraceMs?: number
|
||||
}
|
||||
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/** A body that still carries the Claude Code-style meta header (meta rides the seam as data here). */
|
||||
const META_STATEMENT = /^\s*export\s+const\s+meta\b/
|
||||
|
||||
/**
|
||||
* Parse-check the body with the SAME wrapper the worker-side runtime
|
||||
* compiles, so `start()` keeps the seam's synchronous `SCRIPT_PARSE` throw
|
||||
* (the worker's own compile happens a thread away, after `start()` returned).
|
||||
* One redundant parse per run, bought deliberately for the contract. A body
|
||||
* opening with `export const meta` gets a pointed message instead of the
|
||||
* wrapper's bare SyntaxError — the model's likeliest authoring slip.
|
||||
*/
|
||||
function assertBodyParses(body: string, name: string): void {
|
||||
if (META_STATEMENT.test(body)) {
|
||||
throw new WorkflowError('workflow meta rides the `meta` request field, not the script: remove the `export const meta = {...}` statement from the body', 'SCRIPT_PARSE')
|
||||
}
|
||||
try {
|
||||
// Parse only — the script object is discarded, nothing executes.
|
||||
void new vm.Script(`(async () => {\n${body}\n})()`, { filename: `workflow:${name}`, lineOffset: -1 })
|
||||
} catch (error: unknown) {
|
||||
throw new WorkflowError(`workflow script does not parse: ${String(error)}`, 'SCRIPT_PARSE', { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-thread engine service. `start()` validates the script up front
|
||||
* (meta + a host-side body parse) and returns a {@link WorkflowRun} whose
|
||||
* `result` never rejects; the `workflow/*` events fire around the run per
|
||||
* the seam contract.
|
||||
*/
|
||||
export class WorkerWorkflowEngine extends WorkflowService {
|
||||
static inject = ['subagents']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
provider: z.string().default('spawn'),
|
||||
maxConcurrentAgents: z.natural().default(0),
|
||||
maxTotalAgents: z.natural().min(1).default(1000),
|
||||
maxItemsPerCall: z.natural().min(1).default(4096),
|
||||
syncTimeoutMs: z.natural().min(1).default(5000),
|
||||
disposeGraceMs: z.natural().default(5000),
|
||||
})
|
||||
|
||||
private readonly config: ResolvedConfig
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
// schemastery (static Config) has already filled the defaulted fields;
|
||||
// the assertion records that resolution, not a hidden fallback.
|
||||
this.config = config as ResolvedConfig
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and execute a workflow script in a fresh worker thread. Throws
|
||||
* {@link WorkflowError} synchronously (`META_INVALID` for a malformed meta
|
||||
* block, `SCRIPT_PARSE` for a body that does not compile) for a request
|
||||
* that cannot begin; once a run is returned, every failure resolves through
|
||||
* `result.stopReason` instead.
|
||||
* @param request - the script body, its meta data and `args`, the parent
|
||||
* agent, and an optional cancel signal.
|
||||
* @returns the live run (its `result` resolves when the script settles).
|
||||
*/
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
const meta = validateMeta(request.meta)
|
||||
assertBodyParses(request.script, meta.name)
|
||||
const id = WorkflowRunId(randomUUID())
|
||||
// The event payloads and the run handle get SEPARATE meta clones: a
|
||||
// listener mutating its snapshot must not corrupt the holder's view.
|
||||
const info: WorkflowRunInfo = { id, meta: structuredClone(meta) }
|
||||
const limits: WorkerLimits = {
|
||||
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
|
||||
? Math.min(16, Math.max(1, availableParallelism() - 2))
|
||||
: this.config.maxConcurrentAgents,
|
||||
maxTotalAgents: this.config.maxTotalAgents,
|
||||
maxItemsPerCall: this.config.maxItemsPerCall,
|
||||
syncTimeoutMs: this.config.syncTimeoutMs,
|
||||
}
|
||||
const init: WorkerInit = {
|
||||
meta,
|
||||
body: request.script,
|
||||
...request.args !== undefined ? { args: request.args } : {},
|
||||
limits,
|
||||
}
|
||||
const workerRun = new WorkerRun(
|
||||
this.ctx,
|
||||
id,
|
||||
structuredClone(meta),
|
||||
request.parent,
|
||||
init,
|
||||
this.config.provider,
|
||||
this.config.disposeGraceMs,
|
||||
{
|
||||
phase: (title) => { this.emitWorkflowEvent('workflow/phase', info, title) },
|
||||
log: (message) => { this.emitWorkflowEvent('workflow/log', info, message) },
|
||||
agentStart: (agent) => { this.emitWorkflowEvent('workflow/agent-start', info, agent) },
|
||||
agentEnd: (agent) => { this.emitWorkflowEvent('workflow/agent-end', info, agent) },
|
||||
},
|
||||
request.signal,
|
||||
)
|
||||
|
||||
this.emitWorkflowEvent('workflow/start', info)
|
||||
// `workflow/end` fires as the (never-rejecting) result settles, with the
|
||||
// outcome DATA only — the value stays with the run's holder.
|
||||
void workerRun.result.then((settled) => {
|
||||
this.emitWorkflowEvent('workflow/end', info, {
|
||||
stopReason: settled.stopReason,
|
||||
...settled.error !== undefined ? { error: settled.error } : {},
|
||||
agentsStarted: settled.agentsStarted,
|
||||
})
|
||||
})
|
||||
|
||||
return workerRun
|
||||
}
|
||||
}
|
||||
|
||||
export default WorkerWorkflowEngine
|
||||
85
packages/workflow/workflow-workerthread/src/meta.ts
Normal file
85
packages/workflow/workflow-workerthread/src/meta.ts
Normal file
@@ -0,0 +1,85 @@
|
||||
/**
|
||||
* Meta validation: check the caller-provided {@link WorkflowMeta} DATA against
|
||||
* the shape contract and reject everything else loud, every violation named.
|
||||
* Meta arrives as plain JSON through the seam (the model-facing tool carries
|
||||
* it as a schema-validated object parameter) — the engine never evaluates
|
||||
* script text to obtain it, so no script-controlled code can run on the host
|
||||
* here (an evaluated meta literal could smuggle getters that spin the host
|
||||
* outside any vm timeout, the exact escape the worker thread exists to
|
||||
* prevent).
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/meta
|
||||
*/
|
||||
|
||||
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowMeta, WorkflowPhase } from '@deepseek-ai/dsh-workflow'
|
||||
|
||||
/** Collect shape violations for a meta value (plain JSON data by the seam contract). */
|
||||
function validateMetaShape(meta: unknown): { meta?: WorkflowMeta; violations: string[] } {
|
||||
const violations: string[] = []
|
||||
if (typeof meta !== 'object' || meta === null || Array.isArray(meta)) {
|
||||
return { violations: ['meta must be an object'] }
|
||||
}
|
||||
const record = meta as Record<string, unknown>
|
||||
const known = new Set(['name', 'description', 'whenToUse', 'phases'])
|
||||
for (const key of Object.keys(record)) {
|
||||
if (!known.has(key)) violations.push(`meta.${key} is not a recognized field (name/description/whenToUse/phases)`)
|
||||
}
|
||||
if (typeof record.name !== 'string' || record.name.length === 0) violations.push('meta.name must be a non-empty string')
|
||||
if (typeof record.description !== 'string' || record.description.length === 0) violations.push('meta.description must be a non-empty string')
|
||||
if (record.whenToUse !== undefined && typeof record.whenToUse !== 'string') violations.push('meta.whenToUse must be a string')
|
||||
const phases: WorkflowPhase[] = []
|
||||
if (record.phases !== undefined) {
|
||||
if (!Array.isArray(record.phases)) {
|
||||
violations.push('meta.phases must be an array')
|
||||
} else {
|
||||
record.phases.forEach((phase, index) => {
|
||||
if (typeof phase !== 'object' || phase === null || Array.isArray(phase)) {
|
||||
violations.push(`meta.phases[${index}] must be an object`)
|
||||
return
|
||||
}
|
||||
const entry = phase as Record<string, unknown>
|
||||
for (const key of Object.keys(entry)) {
|
||||
if (!['title', 'detail', 'model'].includes(key)) violations.push(`meta.phases[${index}].${key} is not a recognized field`)
|
||||
}
|
||||
if (typeof entry.title !== 'string' || entry.title.length === 0) violations.push(`meta.phases[${index}].title must be a non-empty string`)
|
||||
if (entry.detail !== undefined && typeof entry.detail !== 'string') violations.push(`meta.phases[${index}].detail must be a string`)
|
||||
if (entry.model !== undefined && typeof entry.model !== 'string') violations.push(`meta.phases[${index}].model must be a string`)
|
||||
if (violations.length === 0) {
|
||||
phases.push({
|
||||
title: entry.title as string,
|
||||
...entry.detail !== undefined ? { detail: entry.detail as string } : {},
|
||||
...entry.model !== undefined ? { model: entry.model as string } : {},
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
if (violations.length > 0) return { violations }
|
||||
return {
|
||||
violations,
|
||||
meta: {
|
||||
name: record.name as string,
|
||||
description: record.description as string,
|
||||
...record.whenToUse !== undefined ? { whenToUse: record.whenToUse as string } : {},
|
||||
...record.phases !== undefined ? { phases } : {},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a caller-provided meta value against the {@link WorkflowMeta}
|
||||
* contract. Throws `META_INVALID` naming every violation (unknown fields,
|
||||
* missing/mistyped `name`/`description`, malformed `phases`); the returned
|
||||
* meta is a NORMALIZED copy built from the validated fields, so the engine
|
||||
* never aliases the caller's object.
|
||||
* @param value - the meta data from the start request (plain JSON by the seam contract).
|
||||
* @returns the validated, normalized meta block.
|
||||
*/
|
||||
export function validateMeta(value: unknown): WorkflowMeta {
|
||||
const { meta, violations } = validateMetaShape(value)
|
||||
if (meta === undefined) {
|
||||
throw new WorkflowError(`invalid meta: ${violations.join('; ')}`, 'META_INVALID')
|
||||
}
|
||||
return meta
|
||||
}
|
||||
114
packages/workflow/workflow-workerthread/src/protocol.ts
Normal file
114
packages/workflow/workflow-workerthread/src/protocol.ts
Normal file
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* The host⇄worker wire protocol: one string-valued enum of message tags per
|
||||
* direction, a payload map giving each tag its parameters (the single source
|
||||
* of truth), and the message unions derived from them. Everything in a
|
||||
* payload is plain JSON data by construction (the runtime materializes
|
||||
* script values before they reach a message; the host projects seam results
|
||||
* down to their JSON fields), so the structured-clone hop never meets a
|
||||
* value it cannot carry.
|
||||
*
|
||||
* Both directions are CLOSED (engine-owned): each side switches on `type`
|
||||
* and ends with `assertNever` — an unknown message is a protocol bug, never
|
||||
* something to skip silently. Senders go through a generic
|
||||
* `post(type, payload)` whose payload parameter is looked up from the map,
|
||||
* so a tag/payload mismatch is a compile error at the call site.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/protocol
|
||||
*/
|
||||
|
||||
import type { WorkflowAgentEndInfo, WorkflowAgentInfo, WorkflowResult } from '@deepseek-ai/dsh-workflow'
|
||||
import type { ChildResult, ChildStartRequest } from './types.ts'
|
||||
|
||||
/** Message tags the worker sends the host (the wire values are the tag strings). */
|
||||
export enum WorkerToHostType {
|
||||
/** The startup handshake: the session is listening and awaits {@link HostToWorkerType.Go}. */
|
||||
Ready = 'ready',
|
||||
/** Observer narration: a `phase(title)` call. */
|
||||
Phase = 'phase',
|
||||
/** Observer narration: a `log(message)` call. */
|
||||
Log = 'log',
|
||||
/** Observer lifecycle: one `agent()` call started a child. */
|
||||
AgentStart = 'agent-start',
|
||||
/** Observer lifecycle: one `agent()` call settled. */
|
||||
AgentEnd = 'agent-end',
|
||||
/** Child RPC: start a child on the host (answered by ChildStarted or ChildStartError). */
|
||||
ChildStart = 'child-start',
|
||||
/** Child RPC: cancel a started child (fire-and-forget). */
|
||||
ChildCancel = 'child-cancel',
|
||||
/** Child RPC: dispose a started child (answered by ChildDisposed). */
|
||||
ChildDispose = 'child-dispose',
|
||||
/** The run's single terminal result. */
|
||||
Result = 'result',
|
||||
}
|
||||
|
||||
/** The payload each worker→host tag carries. */
|
||||
export interface WorkerToHostPayloads {
|
||||
/** Ready carries nothing. */
|
||||
[WorkerToHostType.Ready]: Record<never, never>
|
||||
/** The phase title, verbatim. */
|
||||
[WorkerToHostType.Phase]: { title: string }
|
||||
/** The logged message, verbatim. */
|
||||
[WorkerToHostType.Log]: { message: string }
|
||||
/** The call's sequence number, label, phase, and child id. */
|
||||
[WorkerToHostType.AgentStart]: { info: WorkflowAgentInfo }
|
||||
/** The call identity plus its outcome. */
|
||||
[WorkerToHostType.AgentEnd]: { info: WorkflowAgentEndInfo }
|
||||
/** The RPC correlation id and the prompt plus validated options. */
|
||||
[WorkerToHostType.ChildStart]: { callId: number; request: ChildStartRequest }
|
||||
/** The RPC correlation id and the cancel reason (undefined = unspecified). */
|
||||
[WorkerToHostType.ChildCancel]: { callId: number; reason: string | undefined }
|
||||
/** The RPC correlation id of the child to dispose. */
|
||||
[WorkerToHostType.ChildDispose]: { callId: number }
|
||||
/** The run's terminal outcome. */
|
||||
[WorkerToHostType.Result]: { result: WorkflowResult }
|
||||
}
|
||||
|
||||
/** Message tags the host sends the worker (the wire values are the tag strings). */
|
||||
export enum HostToWorkerType {
|
||||
/** Releases the startup gate: run the script body. */
|
||||
Go = 'go',
|
||||
/** Cancel the run: hooks start throwing and the script dies at its next await. */
|
||||
Cancel = 'cancel',
|
||||
/** Child RPC reply: the start succeeded (exactly one of ChildStarted/ChildStartError per ChildStart). */
|
||||
ChildStarted = 'child-started',
|
||||
/** Child RPC reply: the start was refused or threw. */
|
||||
ChildStartError = 'child-start-error',
|
||||
/** Child RPC: a started child's result RESOLVED (its JSON projection). */
|
||||
ChildSettled = 'child-settled',
|
||||
/** Child RPC: a started child's result REJECTED (an infrastructure fault, rendered). */
|
||||
ChildFailed = 'child-failed',
|
||||
/** Child RPC reply: a requested disposal completed. */
|
||||
ChildDisposed = 'child-disposed',
|
||||
}
|
||||
|
||||
/** The payload each host→worker tag carries. */
|
||||
export interface HostToWorkerPayloads {
|
||||
/** Go carries nothing. */
|
||||
[HostToWorkerType.Go]: Record<never, never>
|
||||
/** The cancel reason, canonical for the whole run. */
|
||||
[HostToWorkerType.Cancel]: { reason: string }
|
||||
/** The RPC correlation id and the child agent's id (minted by the subagent seam). */
|
||||
[HostToWorkerType.ChildStarted]: { callId: number; childId: string }
|
||||
/** The RPC correlation id and the rendered start failure. */
|
||||
[HostToWorkerType.ChildStartError]: { callId: number; rendered: string }
|
||||
/** The RPC correlation id and the child's terminal result projection. */
|
||||
[HostToWorkerType.ChildSettled]: { callId: number; result: ChildResult }
|
||||
/** The RPC correlation id and the rendered infrastructure fault. */
|
||||
[HostToWorkerType.ChildFailed]: { callId: number; rendered: string }
|
||||
/** The RPC correlation id of the completed disposal. */
|
||||
[HostToWorkerType.ChildDisposed]: { callId: number }
|
||||
}
|
||||
|
||||
/**
|
||||
* One worker→host message of tag `T`; unparameterized, the closed union over
|
||||
* every tag (a discriminated union — `switch` on `type` narrows).
|
||||
*/
|
||||
export type WorkerToHostMessage<T extends WorkerToHostType = WorkerToHostType> =
|
||||
{ [K in T]: { type: K } & WorkerToHostPayloads[K] }[T]
|
||||
|
||||
/**
|
||||
* One host→worker message of tag `T`; unparameterized, the closed union over
|
||||
* every tag (a discriminated union — `switch` on `type` narrows).
|
||||
*/
|
||||
export type HostToWorkerMessage<T extends HostToWorkerType = HostToWorkerType> =
|
||||
{ [K in T]: { type: K } & HostToWorkerPayloads[K] }[T]
|
||||
174
packages/workflow/workflow-workerthread/src/realm.ts
Normal file
174
packages/workflow/workflow-workerthread/src/realm.ts
Normal file
@@ -0,0 +1,174 @@
|
||||
/**
|
||||
* The engine's value boundary: copy script-realm values into plain JSON data
|
||||
* — loud about everything JSON cannot carry — and render thrown script
|
||||
* values to failure text. The script runs in a vm context INSIDE the worker
|
||||
* thread, so "host" here means the worker-side JavaScript around that
|
||||
* context; everything that later crosses the thread boundary is JSON by this
|
||||
* walk, which is what makes the postMessage hop total.
|
||||
*
|
||||
* TRUST PREMISE (everything in this module hangs on it): workflow scripts are
|
||||
* MODEL-WRITTEN, the same trust level as the model's existing bash access, so
|
||||
* this boundary guards against BUGGY scripts, not hostile ones. It rejects
|
||||
* loud what JSON would silently mangle — functions, symbols, bigints,
|
||||
* non-finite numbers, nested `undefined`, cycles, sparse arrays, exotic
|
||||
* prototypes — because accepted-then-ignored is this repo's banned failure
|
||||
* mode. It does NOT defend against adversarial values: the walk reads
|
||||
* properties ordinarily (a getter runs, and whatever it returns is what
|
||||
* crosses), {@link renderThrown} reads `stack`/`message`/`String()` directly,
|
||||
* and a proxy is walked through its traps. A hostile script gains nothing
|
||||
* worth defending here — the vm context inside the worker is escapable by
|
||||
* construction, so hostile-value containment would be cost without a threat
|
||||
* model (what the worker thread DOES buy is that a spin occupies the
|
||||
* worker's loop, not the host's, and termination is real).
|
||||
*
|
||||
* The host→realm direction needs no machinery at all: hooks hand the script
|
||||
* plain values of the worker realm, prototypes included — the script is
|
||||
* trusted. One consequence is documented in the engine README: an error
|
||||
* thrown by a hook is built OUTSIDE the script's vm context, so an in-script
|
||||
* `instanceof Error` check is false; read `name`/`code`/`message` instead.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/realm
|
||||
*/
|
||||
|
||||
/** Thrown by {@link materializeFromRealm}; the caller wraps it into the right `WorkflowError` code. */
|
||||
export class MaterializeError extends Error {
|
||||
constructor(public readonly path: string, public readonly reason: string) {
|
||||
super(`${path}: ${reason}`)
|
||||
this.name = 'MaterializeError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a thrown value to failure text without ever throwing: prefer the
|
||||
* `stack` (host or realm — a realm error's `stack` is a plain string read),
|
||||
* fall back to `message`, then `String()`. Reading those properties MAY run
|
||||
* script code (a getter, `toString`) — accepted under the module's trust
|
||||
* premise; if that code itself throws, a fixed label is returned instead.
|
||||
* @param error - the thrown value, of any shape and any realm.
|
||||
* @returns human-readable text for the failure report; prefers the stack.
|
||||
*/
|
||||
export function renderThrown(error: unknown): string {
|
||||
try {
|
||||
const stack = (error as { stack?: unknown } | null | undefined)?.stack
|
||||
if (typeof stack === 'string' && stack.length > 0) return stack
|
||||
const message = (error as { message?: unknown } | null | undefined)?.message
|
||||
if (typeof message === 'string' && message.length > 0) return message
|
||||
return String(error)
|
||||
} catch {
|
||||
// A throwing accessor/toString on the thrown value — rendering must be
|
||||
// total (drive()'s never-reject contract), so fall back to a fixed label.
|
||||
return '[unrenderable thrown value]'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether an object's prototype chain is data-shaped: `null`, or a prototype
|
||||
* whose own prototype is `null` (the realm's `Object.prototype` — which we
|
||||
* cannot compare by identity across realms). A `Date`/`Map`/class instance
|
||||
* has a longer chain and is rejected.
|
||||
*/
|
||||
function hasPlainPrototype(value: object): boolean {
|
||||
const proto: unknown = Object.getPrototypeOf(value)
|
||||
if (proto === null) return true
|
||||
return Object.getPrototypeOf(proto) === null
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy `value` (typically from the vm realm) into plain host JSON data.
|
||||
* Throws {@link MaterializeError} naming the offending path for anything JSON
|
||||
* cannot carry losslessly. Properties are read ordinarily — a getter runs and
|
||||
* its RESULT is materialized; a read that throws surfaces as a
|
||||
* {@link MaterializeError} carrying the rendered failure. `undefined` is
|
||||
* accepted only at the ROOT (a script with no `return` value) — the caller
|
||||
* decides what it means; an `undefined` nested INSIDE a container is a
|
||||
* violation.
|
||||
* @param value - the realm value to materialize.
|
||||
* @param root - the path label for the root value (error messages).
|
||||
* @returns the host-realm copy (plain objects/arrays/scalars only).
|
||||
*/
|
||||
export function materializeFromRealm(value: unknown, root = 'value'): unknown {
|
||||
if (value === undefined) return undefined
|
||||
try {
|
||||
return materialize(value, root, new Set())
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof MaterializeError) throw error
|
||||
// A property read ran script code that threw; total-ize it so callers can
|
||||
// keep the narrow MaterializeError contract.
|
||||
throw new MaterializeError(root, `reading the value threw: ${renderThrown(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
function materialize(value: unknown, path: string, seen: Set<object>): unknown {
|
||||
switch (typeof value) {
|
||||
case 'boolean':
|
||||
case 'string':
|
||||
return value
|
||||
case 'number': {
|
||||
if (!Number.isFinite(value)) throw new MaterializeError(path, 'non-finite numbers are not JSON data')
|
||||
return value
|
||||
}
|
||||
case 'bigint':
|
||||
throw new MaterializeError(path, 'bigints are not JSON data')
|
||||
case 'function':
|
||||
throw new MaterializeError(path, 'functions cannot cross the workflow value boundary')
|
||||
case 'symbol':
|
||||
throw new MaterializeError(path, 'symbols cannot cross the workflow value boundary')
|
||||
case 'undefined':
|
||||
throw new MaterializeError(path, 'undefined is not JSON data')
|
||||
case 'object':
|
||||
break
|
||||
}
|
||||
if (value === null) return null
|
||||
const objectValue: object = value
|
||||
if (seen.has(objectValue)) throw new MaterializeError(path, 'circular references are not JSON data')
|
||||
seen.add(objectValue)
|
||||
try {
|
||||
if (Array.isArray(objectValue)) return materializeArray(objectValue, path, seen)
|
||||
return materializeObject(objectValue, path, seen)
|
||||
} finally {
|
||||
seen.delete(objectValue)
|
||||
}
|
||||
}
|
||||
|
||||
function materializeArray(value: unknown[], path: string, seen: Set<object>): unknown[] {
|
||||
const out: unknown[] = []
|
||||
for (let index = 0; index < value.length; index++) {
|
||||
if (!(index in value)) throw new MaterializeError(`${path}[${index}]`, 'sparse arrays are not JSON data')
|
||||
out.push(materialize(value[index], `${path}[${index}]`, seen))
|
||||
}
|
||||
// Own enumerable props beyond the indices (e.g. `arr.total = 3`) would be
|
||||
// silently dropped by JSON — reject them instead.
|
||||
for (const key of Object.keys(value)) {
|
||||
const index = Number(key)
|
||||
if (!Number.isInteger(index) || index < 0 || index >= value.length) {
|
||||
throw new MaterializeError(`${path}.${key}`, 'arrays with non-index properties are not JSON data')
|
||||
}
|
||||
}
|
||||
if (Object.getOwnPropertySymbols(value).length > 0) {
|
||||
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow value boundary')
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function materializeObject(value: object, path: string, seen: Set<object>): Record<string, unknown> {
|
||||
if (!hasPlainPrototype(value)) {
|
||||
throw new MaterializeError(path, 'only plain objects and arrays are JSON data (exotic prototype)')
|
||||
}
|
||||
if (Object.getOwnPropertySymbols(value).length > 0) {
|
||||
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow value boundary')
|
||||
}
|
||||
const out: Record<string, unknown> = {}
|
||||
// Object.keys = own enumerable string keys, matching JSON.stringify's
|
||||
// property selection exactly (non-enumerable props never reach JSON output).
|
||||
for (const key of Object.keys(value)) {
|
||||
// defineProperty, never assignment: a "__proto__" key must become an OWN
|
||||
// data property of the copy, not a prototype mutation.
|
||||
Object.defineProperty(out, key, {
|
||||
value: materialize((value as Record<string, unknown>)[key], `${path}.${key}`, seen),
|
||||
enumerable: true,
|
||||
writable: true,
|
||||
configurable: true,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
522
packages/workflow/workflow-workerthread/src/runtime.ts
Normal file
522
packages/workflow/workflow-workerthread/src/runtime.ts
Normal file
@@ -0,0 +1,522 @@
|
||||
/**
|
||||
* Per-run execution state for the engine's THREAD side: the script's vm
|
||||
* context and its injected hooks (`agent`/`parallel`/`pipeline`/`phase`/
|
||||
* `log`/`args`), the concurrency semaphore and caps, cancellation, and the
|
||||
* drive loop that turns a script settlement into a {@link WorkflowResult}.
|
||||
* Children are started by RPC to the host through a {@link ChildPort}, so
|
||||
* this module never touches a cordis context — it runs inside the worker
|
||||
* thread.
|
||||
*
|
||||
* Value boundary (the trust premise lives in ./realm.ts): values ENTERING the
|
||||
* worker-side host code from the script (hook options, schemas, the return
|
||||
* value) are materialized by `materializeFromRealm` — a plain walk that
|
||||
* rejects loud everything JSON cannot carry, which also makes every value
|
||||
* safe for the later postMessage hop. Values ENTERING the realm (`args`,
|
||||
* `agent()` results, hook promises and their failures, combinator arrays) are
|
||||
* handed over DIRECTLY as worker-realm values: the script is model-written
|
||||
* and trusted, so outer prototypes are not a leak. `args` is cloned once at
|
||||
* start so a script scribbling on it cannot mutate the session's init object
|
||||
* (a benign-bug guard; the postMessage clone already isolated the caller).
|
||||
*
|
||||
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
|
||||
* unsupported options/schemas, tripped caps, host start refusals and child
|
||||
* result rejections, cancellation) ALWAYS propagate through
|
||||
* `parallel`/`pipeline` — recognized by `instanceof` against this realm's
|
||||
* class, which a script inside the vm context cannot forge — and the per-item
|
||||
* `null` is reserved for child-run failures and ordinary in-stage script
|
||||
* errors. Every hook-returned promise gets a no-op rejection consumer, so a
|
||||
* dropped promise cannot surface an unhandled rejection (which would kill the
|
||||
* worker and read as an engine fault).
|
||||
*
|
||||
* There is deliberately NO worker-side abandon channel: a script that never
|
||||
* settles after a cancel simply never posts a result, and the HOST enforces
|
||||
* the settles-within-grace guarantee by force-settling `cancelled` and
|
||||
* terminating the worker — the real kill an in-process engine could not have.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/runtime
|
||||
*/
|
||||
|
||||
import * as vm from 'node:vm'
|
||||
import { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import { assertSupportedOutputSchema, OutputSchemaError } from '@deepseek-ai/dsh-tools'
|
||||
import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
|
||||
import { isFatalWorkflowError, WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import type {
|
||||
WorkflowAgentEndInfo,
|
||||
WorkflowAgentInfo,
|
||||
WorkflowMeta,
|
||||
WorkflowResult,
|
||||
} from '@deepseek-ai/dsh-workflow'
|
||||
import { materializeFromRealm, MaterializeError, renderThrown } from './realm.ts'
|
||||
import type { ChildHandle, ChildPort, WorkerLimits } from './types.ts'
|
||||
|
||||
/** The observers the execution reports progress through (the session posts them to the host). */
|
||||
export interface ExecutionObserver {
|
||||
phase(title: string): void
|
||||
log(message: string): void
|
||||
agentStart(info: WorkflowAgentInfo): void
|
||||
agentEnd(info: WorkflowAgentEndInfo): void
|
||||
}
|
||||
|
||||
/** The `agent()` options the script may pass; everything else rejects loud. */
|
||||
const SUPPORTED_AGENT_OPTIONS = new Set(['label', 'phase', 'schema', 'model'])
|
||||
/** Deferred Claude Code options we name explicitly in the rejection message. */
|
||||
const DEFERRED_AGENT_OPTIONS = new Set(['effort', 'isolation', 'agentType'])
|
||||
|
||||
/** Flatten a child's final output blocks to text (the non-schema `agent()` result). */
|
||||
function outputText(blocks: ContentBlock[]): string {
|
||||
return blocks
|
||||
.filter((block): block is Extract<ContentBlock, { type: 'text' }> => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
/** A short display label derived from the prompt when the script passes none. */
|
||||
function defaultLabel(prompt: string): string {
|
||||
const newline = prompt.indexOf('\n')
|
||||
const line = newline === -1 ? prompt : prompt.slice(0, newline)
|
||||
return line.length <= 48 ? line : `${line.slice(0, 47)}…`
|
||||
}
|
||||
|
||||
/**
|
||||
* One live script execution inside the worker. Constructed per run by the
|
||||
* session; `drive()` is called exactly once and NEVER rejects — every failure
|
||||
* becomes a {@link WorkflowResult} with a non-`completed` stop reason.
|
||||
*/
|
||||
export class WorkflowExecution {
|
||||
/** 1-based count of `agent()` calls started (the `agentsStarted` result field). */
|
||||
private started = 0
|
||||
private activeSlots = 0
|
||||
private readonly slotWaiters: { resolve(): void; reject(error: unknown): void }[] = []
|
||||
private cancelReason: string | undefined
|
||||
private cancelError: WorkflowError | undefined
|
||||
private readonly controller = new AbortController()
|
||||
private currentPhase: string | undefined
|
||||
private readonly context: vm.Context
|
||||
private readonly compiled: vm.Script
|
||||
|
||||
constructor(
|
||||
meta: WorkflowMeta,
|
||||
body: string,
|
||||
args: unknown,
|
||||
private readonly limits: WorkerLimits,
|
||||
private readonly observer: ExecutionObserver,
|
||||
private readonly children: ChildPort,
|
||||
) {
|
||||
// Compile FIRST: a body syntax error must throw out of the constructor
|
||||
// before any realm state exists. The host pre-parses the identical
|
||||
// wrapper, so under one Node version this throw is unreachable in
|
||||
// production — the session still maps it to an error result defensively.
|
||||
// lineOffset compensates for the wrapper line, so stack traces carry the
|
||||
// script's own line numbers.
|
||||
try {
|
||||
this.compiled = new vm.Script(`(async () => {\n${body}\n})()`, {
|
||||
filename: `workflow:${meta.name}`,
|
||||
lineOffset: -1,
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
throw new WorkflowError(`workflow script does not parse: ${String(error)}`, 'SCRIPT_PARSE', { cause: error })
|
||||
}
|
||||
|
||||
this.context = vm.createContext({}, { name: `workflow:${meta.name}` })
|
||||
|
||||
const globals: Record<string, unknown> = {
|
||||
agent: (prompt: unknown, opts?: unknown) => this.contain(this.agent(prompt, opts)),
|
||||
parallel: (thunks: unknown) => this.contain(this.parallel(thunks)),
|
||||
pipeline: (items: unknown, ...stages: unknown[]) => this.contain(this.pipeline(items, stages)),
|
||||
phase: (title: unknown) => { this.phase(title) },
|
||||
log: (message: unknown) => { this.log(message) },
|
||||
// Cloned once: a script scribbling on args must not mutate the
|
||||
// session's init object (a benign-bug guard; args is plain JSON by the
|
||||
// seam contract and already crossed one structured clone as workerData,
|
||||
// so this clone is total).
|
||||
args: args === undefined ? undefined : structuredClone(args),
|
||||
}
|
||||
for (const [key, value] of Object.entries(globals)) {
|
||||
// Data properties on the contextified global; frozen shape not required —
|
||||
// a script overwriting its own hooks only sabotages itself.
|
||||
;(this.context as Record<string, unknown>)[key] = typeof value === 'function' ? Object.freeze(value) : value
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the run has been cancelled. A METHOD, not an inline property
|
||||
* read: `cancel()` mutates `cancelReason` concurrently (the session's
|
||||
* message handler), and an inline read after an `await` gets narrowed by
|
||||
* control flow into an always-false comparison.
|
||||
*/
|
||||
private isCancelled(): boolean {
|
||||
return this.cancelReason !== undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared hook entry guard: after {@link cancel}, EVERY hook throws
|
||||
* `CANCELLED` at its next call — cancellation is the next HOOK boundary,
|
||||
* not just the next `agent()`, so a script that caught one cancelled
|
||||
* rejection cannot keep emitting progress through `phase`/`log` or enter a
|
||||
* combinator.
|
||||
*/
|
||||
private throwIfCancelled(): void {
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel the run: in-flight children get a cancel RPC (the shared abort
|
||||
* fanout), waiting `agent()` slots reject, and every future hook call
|
||||
* throws `CANCELLED` — the script dies at its next await. A script that
|
||||
* never settles anyway (parked on a promise no hook owns) is the HOST's
|
||||
* problem: its grace timer force-settles the run and terminates the
|
||||
* worker. Idempotent; the first reason wins.
|
||||
* @param reason - human-readable cause, carried on the CANCELLED error and
|
||||
* into child cancel RPCs. Required: every caller (the session's cancel
|
||||
* message, drive()'s settle-reap) has a concrete reason.
|
||||
*/
|
||||
cancel(reason: string): void {
|
||||
if (this.cancelReason !== undefined) return
|
||||
this.cancelReason = reason
|
||||
this.cancelError = new WorkflowError(`workflow run cancelled: ${this.cancelReason}`, 'CANCELLED')
|
||||
this.controller.abort(this.cancelReason)
|
||||
for (const waiter of this.slotWaiters.splice(0)) waiter.reject(this.cancelledError())
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the script to settlement. Resolves — never rejects — with the run's
|
||||
* {@link WorkflowResult}: the materialized return value on `completed`, the
|
||||
* failure message on `error`, and `cancelled` when the script died of
|
||||
* cancellation. After settlement, any stray children a script fired without
|
||||
* awaiting are cancelled (their `agent()` wrappers dispose them via RPC).
|
||||
* @returns the settled outcome — this promise NEVER rejects (the seam's
|
||||
* `result`-never-rejects contract); every failure maps to a variant.
|
||||
*/
|
||||
async drive(): Promise<WorkflowResult> {
|
||||
try {
|
||||
// Cancelled before the body ever ran (an already-aborted start signal,
|
||||
// relayed by the host before its `go`): the script must not execute at
|
||||
// all, let alone report `completed`.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
const scriptPromise = this.compiled.runInContext(this.context, { timeout: this.limits.syncTimeoutMs }) as Promise<unknown>
|
||||
const raw: unknown = await this.contain(Promise.resolve(scriptPromise))
|
||||
// Cancelled while the body ran: a script that settled without touching
|
||||
// another hook (or without any) must still report `cancelled` — the
|
||||
// holder asked for cancellation and `completed` would be a lie.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
const value = raw === undefined ? null : this.materializeResult(raw)
|
||||
return { value, stopReason: 'completed', agentsStarted: this.started }
|
||||
} catch (error: unknown) {
|
||||
// Any failure after cancel() reports `cancelled` with the canonical
|
||||
// reason — the reject path mirrors the resolve path's post-settle check.
|
||||
if (this.isCancelled()) {
|
||||
return { value: null, stopReason: 'cancelled', error: this.cancelledError().message, agentsStarted: this.started }
|
||||
}
|
||||
// renderThrown is total (thrown values of any realm), so this arm
|
||||
// cannot throw — drive() resolving is the `result` never-rejects seam
|
||||
// contract.
|
||||
return { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: this.started }
|
||||
} finally {
|
||||
// Reap strays: a script that fired agent() calls without awaiting them
|
||||
// leaves live children behind after settlement — cancel them all. (The
|
||||
// per-call wrappers dispose each child; the contain() consumer keeps
|
||||
// their rejections from going unhandled.)
|
||||
if (this.cancelReason === undefined) this.cancel('workflow settled')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach a no-op rejection consumer WITHOUT changing what the caller
|
||||
* receives: if the script drops the promise (no await), cancellation cannot
|
||||
* become an unhandled rejection (which would kill the worker thread); if
|
||||
* the script does await it, it still observes the rejection.
|
||||
*/
|
||||
private contain<T>(promise: Promise<T>): Promise<T> {
|
||||
promise.catch(() => { /* consumed: see method contract — a dropped hook promise must not surface an unhandled rejection */ })
|
||||
return promise
|
||||
}
|
||||
|
||||
private cancelledError(): WorkflowError {
|
||||
// cancel() arms cancelError before any caller can observe isCancelled()
|
||||
// === true; the fallback guards the type, not a reachable path.
|
||||
/* v8 ignore next */
|
||||
return this.cancelError ?? new WorkflowError('workflow run cancelled', 'CANCELLED')
|
||||
}
|
||||
|
||||
/** Materialize the script's return value; violations become RESULT_UNSERIALIZABLE. */
|
||||
private materializeResult(raw: unknown): unknown {
|
||||
try {
|
||||
return materializeFromRealm(raw, 'workflow result')
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: materializeFromRealm only throws MaterializeError */
|
||||
if (!(error instanceof MaterializeError)) throw error
|
||||
throw new WorkflowError(
|
||||
`the workflow's return value is not plain JSON data — ${error.message}. Return only JSON-serializable objects/arrays/scalars.`,
|
||||
'RESULT_UNSERIALIZABLE',
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire one concurrency slot (FIFO). Cancellation rejects QUEUED waiters
|
||||
* (see {@link cancel}); the callers guard their own entry and post-acquire
|
||||
* windows, so no cancelled-precheck is duplicated here.
|
||||
*/
|
||||
private acquireSlot(): Promise<void> {
|
||||
if (this.activeSlots < this.limits.maxConcurrentAgents) {
|
||||
this.activeSlots += 1
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
this.slotWaiters.push({
|
||||
resolve: () => {
|
||||
this.activeSlots += 1
|
||||
resolve()
|
||||
},
|
||||
reject,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
private releaseSlot(): void {
|
||||
this.activeSlots -= 1
|
||||
const next = this.slotWaiters.shift()
|
||||
if (next) next.resolve()
|
||||
}
|
||||
|
||||
/** The `agent(prompt, opts)` hook. */
|
||||
private async agent(rawPrompt: unknown, rawOpts: unknown): Promise<unknown> {
|
||||
this.throwIfCancelled()
|
||||
if (typeof rawPrompt !== 'string' || rawPrompt.length === 0) {
|
||||
throw new WorkflowError('agent() requires a non-empty prompt string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const opts = this.readAgentOptions(rawOpts)
|
||||
if (this.started >= this.limits.maxTotalAgents) {
|
||||
throw new WorkflowError(
|
||||
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise maxTotalAgents in the engine config if the scale is intentional`,
|
||||
'AGENT_CAP',
|
||||
)
|
||||
}
|
||||
this.started += 1
|
||||
const seq = this.started
|
||||
const label = opts.label ?? defaultLabel(rawPrompt)
|
||||
const phase = opts.phase ?? this.currentPhase
|
||||
|
||||
await this.acquireSlot()
|
||||
try {
|
||||
// Re-check after the acquire: the await yields at least one microtask
|
||||
// tick even when a slot is free, and a queued waiter resumes a tick
|
||||
// after its release — a cancel() landing in either window must not
|
||||
// reach the host (which would refuse anyway, but the refusal reads as
|
||||
// a start failure rather than the cancellation it is).
|
||||
this.throwIfCancelled()
|
||||
let run: ChildHandle
|
||||
try {
|
||||
run = await this.children.startAgent({
|
||||
prompt: rawPrompt,
|
||||
...opts.schema !== undefined ? { schema: opts.schema } : {},
|
||||
...opts.model !== undefined ? { model: opts.model } : {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
// The host refuses starts once the run is cancelled — a refusal that
|
||||
// races our own cancel state must read as the cancellation it is,
|
||||
// not as a broken seam.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
throw new WorkflowError(`agent() could not start a child: ${renderThrown(error)}`, 'AGENT_START', { cause: error })
|
||||
}
|
||||
// The start round-trip yields to the event loop, so a cancel CAN land
|
||||
// between the host starting the child and this continuation running —
|
||||
// wind the fresh child down instead of leaving it live behind a dead
|
||||
// script.
|
||||
if (this.isCancelled()) {
|
||||
run.cancel(this.cancelReason)
|
||||
await run.dispose()
|
||||
throw this.cancelledError()
|
||||
}
|
||||
const info: WorkflowAgentInfo = { seq, label, ...phase !== undefined ? { phase } : {}, childId: AgentId(run.id) }
|
||||
this.observer.agentStart(info)
|
||||
// Cancellation reaches the child through an explicit cancel RPC per
|
||||
// child (the host also aborts its own per-run signal, but the seam
|
||||
// leaves a provider free to honor either channel, so both are driven).
|
||||
const onAbort = (): void => { run.cancel(this.cancelReason) }
|
||||
this.controller.signal.addEventListener('abort', onAbort, { once: true })
|
||||
try {
|
||||
let result
|
||||
try {
|
||||
result = await run.result
|
||||
} catch (error: unknown) {
|
||||
// A rejected child result is an INFRASTRUCTURE fault relayed by the
|
||||
// host — distinct from a child that failed and resolved. Pair the
|
||||
// lifecycle before propagating, and propagate FATAL: an ordinary
|
||||
// throw would dissolve to a per-item null inside the combinators,
|
||||
// and a broken provider must not read as a failed child.
|
||||
if (this.isCancelled()) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'cancelled' })
|
||||
throw this.cancelledError()
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
throw new WorkflowError(`child agent run failed: ${renderThrown(error)}`, 'AGENT_RESULT', { cause: error })
|
||||
}
|
||||
if (result.stopReason === 'completed') {
|
||||
if (opts.schema !== undefined) {
|
||||
// The provider honored outputSchema (capability-gated at start), so
|
||||
// a completed run without a structured value is a child failure.
|
||||
if (result.structured === undefined) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
return null
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'completed' })
|
||||
return result.structured
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'completed' })
|
||||
return outputText(result.output)
|
||||
}
|
||||
// A cancelled RUN kills the script; a child that failed for its own
|
||||
// reasons resolves null (scripts .filter(Boolean) per the CC contract).
|
||||
if (this.isCancelled()) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'cancelled' })
|
||||
throw this.cancelledError()
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
return null
|
||||
} finally {
|
||||
this.controller.signal.removeEventListener('abort', onAbort)
|
||||
await run.dispose()
|
||||
}
|
||||
} finally {
|
||||
this.releaseSlot()
|
||||
}
|
||||
}
|
||||
|
||||
/** Materialize + validate the `agent()` options bag from the realm. */
|
||||
private readAgentOptions(rawOpts: unknown): { label?: string; phase?: string; model?: string; schema?: StructuredOutputSchema } {
|
||||
if (rawOpts === undefined) return {}
|
||||
let opts: unknown
|
||||
try {
|
||||
opts = materializeFromRealm(rawOpts, 'agent() options')
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: materializeFromRealm only throws MaterializeError */
|
||||
if (!(error instanceof MaterializeError)) throw error
|
||||
throw new WorkflowError(`agent() options must be plain JSON data — ${error.message}`, 'INVALID_ARGUMENT', { cause: error })
|
||||
}
|
||||
if (typeof opts !== 'object' || opts === null || Array.isArray(opts)) {
|
||||
throw new WorkflowError('agent() options must be an object', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const record = opts as Record<string, unknown>
|
||||
for (const key of Object.keys(record)) {
|
||||
if (SUPPORTED_AGENT_OPTIONS.has(key)) continue
|
||||
if (DEFERRED_AGENT_OPTIONS.has(key)) {
|
||||
throw new WorkflowError(`agent() option "${key}" is deferred and not supported by this engine (supported: label, phase, schema, model)`, 'UNSUPPORTED_OPTION')
|
||||
}
|
||||
throw new WorkflowError(`agent() option "${key}" is not recognized (supported: label, phase, schema, model)`, 'UNSUPPORTED_OPTION')
|
||||
}
|
||||
for (const key of ['label', 'phase', 'model'] as const) {
|
||||
if (record[key] !== undefined && typeof record[key] !== 'string') {
|
||||
throw new WorkflowError(`agent() option "${key}" must be a string`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
}
|
||||
let schema: StructuredOutputSchema | undefined
|
||||
if (record.schema !== undefined) {
|
||||
try {
|
||||
assertSupportedOutputSchema(record.schema)
|
||||
schema = record.schema
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: assertSupportedOutputSchema only throws OutputSchemaError */
|
||||
if (!(error instanceof OutputSchemaError)) throw error
|
||||
throw new WorkflowError(`agent() schema is outside the supported subset — ${error.message}`, 'UNSUPPORTED_SCHEMA', { cause: error })
|
||||
}
|
||||
}
|
||||
return {
|
||||
...record.label !== undefined ? { label: record.label as string } : {},
|
||||
...record.phase !== undefined ? { phase: record.phase as string } : {},
|
||||
...record.model !== undefined ? { model: record.model as string } : {},
|
||||
...schema !== undefined ? { schema } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/** The `parallel(thunks)` hook: each thunk caught → `null`; fatal errors propagate. */
|
||||
private async parallel(rawThunks: unknown): Promise<unknown[]> {
|
||||
this.throwIfCancelled()
|
||||
if (!Array.isArray(rawThunks)) {
|
||||
throw new WorkflowError('parallel() requires an array of zero-argument functions', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.assertItemCap(rawThunks.length, 'parallel()')
|
||||
const thunks = rawThunks.map((thunk, index) => {
|
||||
if (typeof thunk !== 'function') {
|
||||
throw new WorkflowError(`parallel() item ${index} is not a function`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
return thunk as () => unknown
|
||||
})
|
||||
return Promise.all(thunks.map(async (thunk) => {
|
||||
try {
|
||||
return await thunk()
|
||||
} catch (error: unknown) {
|
||||
// Hook failures are WorkflowErrors built OUTSIDE the script's realm;
|
||||
// fatality is recognized by `instanceof` against this realm's class —
|
||||
// a script-built object can never pass it, so fatality cannot be
|
||||
// forged (nor accidentally dissolved).
|
||||
if (isFatalWorkflowError(error)) throw error
|
||||
return null
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
/** The `pipeline(items, ...stages)` hook: per-item stage chains, NO cross-stage barrier. */
|
||||
private async pipeline(rawItems: unknown, rawStages: unknown[]): Promise<unknown[]> {
|
||||
this.throwIfCancelled()
|
||||
if (!Array.isArray(rawItems)) {
|
||||
throw new WorkflowError('pipeline() requires an items array', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.assertItemCap(rawItems.length, 'pipeline()')
|
||||
if (rawStages.length === 0) {
|
||||
throw new WorkflowError('pipeline() requires at least one stage function', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const stages = rawStages.map((stage, index) => {
|
||||
if (typeof stage !== 'function') {
|
||||
throw new WorkflowError(`pipeline() stage ${index} is not a function`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
return stage as (previous: unknown, item: unknown, index: number) => unknown
|
||||
})
|
||||
return Promise.all(rawItems.map(async (item: unknown, index) => {
|
||||
let value: unknown = item
|
||||
try {
|
||||
for (const stage of stages) {
|
||||
value = await stage(value, item, index)
|
||||
}
|
||||
return value
|
||||
} catch (error: unknown) {
|
||||
// An ordinary stage throw drops the ITEM to null and skips its
|
||||
// remaining stages; a fatal WorkflowError (see parallel()) kills the
|
||||
// whole script.
|
||||
if (isFatalWorkflowError(error)) throw error
|
||||
return null
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
private assertItemCap(length: number, hook: string): void {
|
||||
if (length > this.limits.maxItemsPerCall) {
|
||||
throw new WorkflowError(
|
||||
`${hook} received ${length} items — over the per-call cap (${this.limits.maxItemsPerCall}); split the work or raise maxItemsPerCall in the engine config`,
|
||||
'ITEM_CAP',
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/** The `phase(title)` hook: sets the current label for subsequent `agent()` calls and notifies observers. */
|
||||
private phase(title: unknown): void {
|
||||
this.throwIfCancelled()
|
||||
if (typeof title !== 'string' || title.length === 0) {
|
||||
throw new WorkflowError('phase() requires a non-empty title string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.currentPhase = title
|
||||
this.observer.phase(title)
|
||||
}
|
||||
|
||||
/** The `log(message)` hook: narration to observers. */
|
||||
private log(message: unknown): void {
|
||||
this.throwIfCancelled()
|
||||
if (typeof message !== 'string') {
|
||||
throw new WorkflowError('log() requires a message string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.observer.log(message)
|
||||
}
|
||||
}
|
||||
210
packages/workflow/workflow-workerthread/src/session.ts
Normal file
210
packages/workflow/workflow-workerthread/src/session.ts
Normal file
@@ -0,0 +1,210 @@
|
||||
/**
|
||||
* The worker-side half of the engine: {@link runWorkerSession} wires one
|
||||
* MessagePort to one {@link WorkflowExecution} — hook progress and child
|
||||
* starts go out as messages, run control and child lifecycle come back in —
|
||||
* and posts the run's terminal result exactly once. Deliberately separated
|
||||
* from the thread bootstrap (./worker.ts): the whole session is drivable
|
||||
* in-process over a `MessageChannel`, which is where its unit coverage lives
|
||||
* (code inside a real Worker is invisible to the main process's coverage).
|
||||
*
|
||||
* Startup handshake: the session posts `ready` and runs the script only
|
||||
* after the host's `go` — without it, a cancellation racing the worker's
|
||||
* boot could arrive AFTER the script's initial synchronous slice already
|
||||
* ran, and a run cancelled before start must not execute the body at all.
|
||||
* A `cancel` arriving instead of `go` still releases the gate: `drive()`
|
||||
* sees the cancelled state and settles without running the body.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/session
|
||||
*/
|
||||
|
||||
import type { MessagePort } from 'node:worker_threads'
|
||||
import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
|
||||
import type { HostToWorkerMessage, WorkerToHostPayloads } from './protocol.ts'
|
||||
import { renderThrown } from './realm.ts'
|
||||
import { WorkflowExecution } from './runtime.ts'
|
||||
import type { ExecutionObserver } from './runtime.ts'
|
||||
import type {
|
||||
ChildHandle,
|
||||
ChildPort,
|
||||
ChildResult,
|
||||
ChildStartRequest,
|
||||
WorkerInit,
|
||||
} from './types.ts'
|
||||
|
||||
/** The book-keeping for one in-flight child RPC (keyed by callId). */
|
||||
interface PendingChild {
|
||||
started: PromiseWithResolvers<string>
|
||||
settled: PromiseWithResolvers<ChildResult>
|
||||
disposed: PromiseWithResolvers<void>
|
||||
}
|
||||
|
||||
/** The typed post half of the port: each tag pairs with ITS payload from the map (a mismatch is a compile error at the call site). */
|
||||
type Post = <T extends WorkerToHostType>(type: T, payload: WorkerToHostPayloads[T]) => void
|
||||
|
||||
/**
|
||||
* The worker-side handle for one started child agent ({@link ChildHandle}):
|
||||
* every member is an RPC to the host keyed by this call's `callId`, resolved
|
||||
* by the session's message handler through the bridge's pending entry.
|
||||
*/
|
||||
class RpcChildHandle implements ChildHandle {
|
||||
readonly result: Promise<ChildResult>
|
||||
|
||||
constructor(
|
||||
private readonly post: Post,
|
||||
private readonly callId: number,
|
||||
private readonly entry: PendingChild,
|
||||
readonly id: string,
|
||||
) {
|
||||
this.result = entry.settled.promise
|
||||
}
|
||||
|
||||
cancel(reason?: string): void {
|
||||
this.post(WorkerToHostType.ChildCancel, { callId: this.callId, reason })
|
||||
}
|
||||
|
||||
dispose(): Promise<void> {
|
||||
this.post(WorkerToHostType.ChildDispose, { callId: this.callId })
|
||||
return this.entry.disposed.promise
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side child-RPC bridge ({@link ChildPort}): allocates callIds,
|
||||
* posts the start/cancel/dispose RPCs, and owns the per-call pending
|
||||
* book-keeping the session's message handler settles via the `onChild*`
|
||||
* entry points.
|
||||
*/
|
||||
class ChildRpcBridge implements ChildPort {
|
||||
private nextCallId = 0
|
||||
private readonly pending = new Map<number, PendingChild>()
|
||||
|
||||
constructor(private readonly post: Post) {}
|
||||
|
||||
async startAgent(request: ChildStartRequest): Promise<ChildHandle> {
|
||||
this.nextCallId += 1
|
||||
const callId = this.nextCallId
|
||||
const entry: PendingChild = {
|
||||
started: Promise.withResolvers<string>(),
|
||||
settled: Promise.withResolvers<ChildResult>(),
|
||||
disposed: Promise.withResolvers<void>(),
|
||||
}
|
||||
// Containment: when the start is refused (or the run torn down) the
|
||||
// settled promise may never gain a consumer — it must not surface as an
|
||||
// unhandled rejection and kill the worker.
|
||||
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after a refused start */ })
|
||||
this.pending.set(callId, entry)
|
||||
this.post(WorkerToHostType.ChildStart, { callId, request })
|
||||
const childId = await entry.started.promise
|
||||
return new RpcChildHandle(this.post, callId, entry, childId)
|
||||
}
|
||||
|
||||
/** The host started the child; releases the `startAgent` await. */
|
||||
onChildStarted(callId: number, childId: string): void {
|
||||
this.pending.get(callId)?.started.resolve(childId)
|
||||
}
|
||||
|
||||
/** The host refused the start; `startAgent` rejects with the rendered cause. */
|
||||
onChildStartError(callId: number, rendered: string): void {
|
||||
this.pending.get(callId)?.started.reject(new Error(rendered))
|
||||
}
|
||||
|
||||
/** The child's terminal result arrived. */
|
||||
onChildSettled(callId: number, result: ChildResult): void {
|
||||
this.pending.get(callId)?.settled.resolve(result)
|
||||
}
|
||||
|
||||
/** The child's `result` rejected host-side (an infrastructure fault, relayed as fatal). */
|
||||
onChildFailed(callId: number, rendered: string): void {
|
||||
this.pending.get(callId)?.settled.reject(new Error(rendered))
|
||||
}
|
||||
|
||||
/** The host acked the dispose; the call's book-keeping is complete. */
|
||||
onChildDisposed(callId: number): void {
|
||||
const entry = this.pending.get(callId)
|
||||
this.pending.delete(callId)
|
||||
entry?.disposed.resolve()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow the nullable `parentPort` the bootstrap reads from
|
||||
* `node:worker_threads`.
|
||||
* @param port - `parentPort` as imported (null on the main thread).
|
||||
* @returns the port, non-null.
|
||||
*/
|
||||
export function requireParentPort(port: MessagePort | null): MessagePort {
|
||||
if (port === null) throw new Error('the workflow worker entry must be loaded inside a worker thread (no parentPort)')
|
||||
return port
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one workflow script to settlement against `port`, posting the terminal
|
||||
* result message exactly once; resolves after that post (stray children may
|
||||
* still be winding down through the port — the host owns their teardown and
|
||||
* ultimately terminates the thread). Never rejects: a constructor failure
|
||||
* (unparseable body — host pre-parse makes this a Node-version-skew signal)
|
||||
* is reported as an `error` result rather than dying without a result.
|
||||
* @param port - the channel to the host (the real `parentPort`, or one side
|
||||
* of an in-process `MessageChannel` in tests).
|
||||
* @param init - the run payload the host provided as `workerData`.
|
||||
*/
|
||||
export async function runWorkerSession(port: MessagePort, init: WorkerInit): Promise<void> {
|
||||
const post: Post = (type, payload) => {
|
||||
port.postMessage({ type, ...payload })
|
||||
}
|
||||
const children = new ChildRpcBridge(post)
|
||||
|
||||
const observer: ExecutionObserver = {
|
||||
phase: (title) => { post(WorkerToHostType.Phase, { title }) },
|
||||
log: (message) => { post(WorkerToHostType.Log, { message }) },
|
||||
agentStart: (info) => { post(WorkerToHostType.AgentStart, { info }) },
|
||||
agentEnd: (info) => { post(WorkerToHostType.AgentEnd, { info }) },
|
||||
}
|
||||
|
||||
let execution: WorkflowExecution
|
||||
try {
|
||||
execution = new WorkflowExecution(init.meta, init.body, init.args, init.limits, observer, children)
|
||||
} catch (error: unknown) {
|
||||
post(WorkerToHostType.Result, { result: { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: 0 } })
|
||||
return
|
||||
}
|
||||
|
||||
const gate = Promise.withResolvers<void>()
|
||||
port.on('message', (message: HostToWorkerMessage) => {
|
||||
switch (message.type) {
|
||||
case HostToWorkerType.Go:
|
||||
gate.resolve()
|
||||
break
|
||||
case HostToWorkerType.Cancel:
|
||||
execution.cancel(message.reason)
|
||||
// A cancel doubles as the gate release: drive() checks the cancelled
|
||||
// state before running the body, so the script never executes.
|
||||
gate.resolve()
|
||||
break
|
||||
case HostToWorkerType.ChildStarted:
|
||||
children.onChildStarted(message.callId, message.childId)
|
||||
break
|
||||
case HostToWorkerType.ChildStartError:
|
||||
children.onChildStartError(message.callId, message.rendered)
|
||||
break
|
||||
case HostToWorkerType.ChildSettled:
|
||||
children.onChildSettled(message.callId, message.result)
|
||||
break
|
||||
case HostToWorkerType.ChildFailed:
|
||||
children.onChildFailed(message.callId, message.rendered)
|
||||
break
|
||||
case HostToWorkerType.ChildDisposed:
|
||||
children.onChildDisposed(message.callId)
|
||||
break
|
||||
/* v8 ignore next 2 -- closed engine-owned union; the arm only makes adding a message type a compile error */
|
||||
default:
|
||||
assertNever(message, 'host-to-worker message')
|
||||
}
|
||||
})
|
||||
|
||||
post(WorkerToHostType.Ready, {})
|
||||
await gate.promise
|
||||
const result = await execution.drive()
|
||||
post(WorkerToHostType.Result, { result })
|
||||
}
|
||||
97
packages/workflow/workflow-workerthread/src/types.ts
Normal file
97
packages/workflow/workflow-workerthread/src/types.ts
Normal file
@@ -0,0 +1,97 @@
|
||||
/**
|
||||
* Non-protocol wire vocabulary for the worker-thread engine: the `workerData` init
|
||||
* payload and the child-port interfaces the worker-side runtime consumes.
|
||||
* The host⇄worker MESSAGE protocol lives in ./protocol.ts; everything here
|
||||
* that a message transports (`ChildStartRequest`, `ChildResult`) is plain
|
||||
* JSON data by construction, so the structured-clone hop never meets a value
|
||||
* it cannot carry. Types only, per the package convention.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/types
|
||||
*/
|
||||
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
|
||||
import type { WorkflowMeta } from '@deepseek-ai/dsh-workflow'
|
||||
|
||||
/**
|
||||
* The per-run limits the worker-side runtime enforces. The host keeps the
|
||||
* knobs only it can act on (`provider`, `disposeGraceMs`).
|
||||
*/
|
||||
export interface WorkerLimits {
|
||||
/** Concurrent `agent()` ceiling (already auto-resolved; ≥ 1). */
|
||||
maxConcurrentAgents: number
|
||||
/** Total `agent()` calls per run (the runaway-loop backstop). */
|
||||
maxTotalAgents: number
|
||||
/** Items accepted by one `parallel()`/`pipeline()` call. */
|
||||
maxItemsPerCall: number
|
||||
/** vm timeout for the script's initial synchronous slice (inside the worker). */
|
||||
syncTimeoutMs: number
|
||||
}
|
||||
|
||||
/** The `workerData` payload one run is initialized with (host → worker, once, at spawn). */
|
||||
export interface WorkerInit {
|
||||
/** The validated meta block (plain data off the start request, validated host-side). */
|
||||
meta: WorkflowMeta
|
||||
/** The plain-JS script body, exactly as the start request carried it. */
|
||||
body: string
|
||||
/** The run's `args` value; the workerData structured clone is the copy that isolates the caller. */
|
||||
args?: unknown
|
||||
/** The worker-enforced limits. */
|
||||
limits: WorkerLimits
|
||||
}
|
||||
|
||||
/** What the worker asks the host to start for one `agent()` call (options already validated worker-side). */
|
||||
export interface ChildStartRequest {
|
||||
/** The child's prompt text. */
|
||||
prompt: string
|
||||
/** The structured-output schema, if the call passed one (already subset-checked). */
|
||||
schema?: StructuredOutputSchema
|
||||
/** The per-child model override, if the call passed one. */
|
||||
model?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The JSON projection of a child's `SubagentResult` crossing the port. The
|
||||
* seam's `stopReason` union is merge-extensible, so it degrades to `string`
|
||||
* on the wire — the runtime only ever branches on `'completed'`.
|
||||
*/
|
||||
export interface ChildResult {
|
||||
/** The child's final assistant output blocks. */
|
||||
output: ContentBlock[]
|
||||
/** The structured value, present iff the request carried a schema AND the provider honored it. */
|
||||
structured?: unknown
|
||||
/** Why the child run ended (`'completed'` is the only value the runtime branches on). */
|
||||
stopReason: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side handle for one started child — the RPC mirror of the
|
||||
* subagent seam's run handle, reduced to what the runtime consumes.
|
||||
*/
|
||||
export interface ChildHandle {
|
||||
/** The child agent's id (minted host-side by the subagent seam). */
|
||||
readonly id: string
|
||||
/**
|
||||
* Resolves with the child's terminal {@link ChildResult}; REJECTS only when
|
||||
* the host reports an infrastructure fault (`child-failed`) — a child that
|
||||
* failed for its own reasons resolves with a non-`completed` stop reason.
|
||||
*/
|
||||
readonly result: Promise<ChildResult>
|
||||
/** Ask the host to cancel the child (fire-and-forget). */
|
||||
cancel(reason?: string): void
|
||||
/** Ask the host to dispose the child; resolves on the host's ack. */
|
||||
dispose(): Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side port the runtime starts child agents through — the seam
|
||||
* that lets the execution core stay ignorant of the thread boundary.
|
||||
*/
|
||||
export interface ChildPort {
|
||||
/**
|
||||
* Start one child agent on the host (the `agent()` hook's start half).
|
||||
* @param request - the prompt and validated options.
|
||||
* @returns the child handle; rejects when the host refuses the start.
|
||||
*/
|
||||
startAgent(request: ChildStartRequest): Promise<ChildHandle>
|
||||
}
|
||||
18
packages/workflow/workflow-workerthread/src/worker.ts
Normal file
18
packages/workflow/workflow-workerthread/src/worker.ts
Normal file
@@ -0,0 +1,18 @@
|
||||
/**
|
||||
* The worker-thread entry the engine spawns: bootstrap ./session.ts on the
|
||||
* real `parentPort`. Deliberately a single statement — every piece of logic
|
||||
* lives in `runWorkerSession`, which the unit suite drives in-process over a
|
||||
* `MessageChannel` (code inside a real Worker is invisible to main-process
|
||||
* coverage); loading this module on the main thread throws via
|
||||
* `requireParentPort`, which is how the suite covers the file itself.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/worker
|
||||
*/
|
||||
|
||||
import { parentPort, workerData } from 'node:worker_threads'
|
||||
import { requireParentPort, runWorkerSession } from './session.ts'
|
||||
import type { WorkerInit } from './types.ts'
|
||||
|
||||
// workerData is `any` at the node:worker_threads boundary; the engine is the
|
||||
// only spawner and always provides a WorkerInit.
|
||||
void runWorkerSession(requireParentPort(parentPort), workerData as WorkerInit)
|
||||
@@ -0,0 +1,57 @@
|
||||
import { existsSync } from 'node:fs'
|
||||
import { rm, writeFile } from 'node:fs/promises'
|
||||
import { join } from 'node:path'
|
||||
import { execFile } from 'node:child_process'
|
||||
import { promisify } from 'node:util'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
const packageRoot = fileURLToPath(new URL('..', import.meta.url))
|
||||
const builtIndex = join(packageRoot, 'lib', 'index.js')
|
||||
const builtWorker = join(packageRoot, 'lib', 'worker.js')
|
||||
const run = promisify(execFile)
|
||||
|
||||
/**
|
||||
* The BUILT-output guard for the worker entry: every other suite runs
|
||||
* unbuilt (src/ + tsx), so nothing else proves that `lib/index.js` resolves
|
||||
* its sibling `lib/worker.js` and that the bundle boots a worker under plain
|
||||
* node (no tsx loader). Keyless — a zero-agent script needs no provider —
|
||||
* and self-skips until `pnpm run build` has produced the bundles.
|
||||
*/
|
||||
describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.js)', () => {
|
||||
it('the built engine spawns its built worker under plain node and completes a run', async () => {
|
||||
// ESM resolves bare specifiers from the IMPORTING FILE's location, so the
|
||||
// driver must live inside the package for its node_modules to apply — a
|
||||
// temp-named file at the package root, removed on the way out.
|
||||
const driver = join(packageRoot, `.built-worker-driver-${process.pid}.mjs`)
|
||||
try {
|
||||
await writeFile(driver, `
|
||||
import { Context } from 'cordis'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import WorkerWorkflowEngine from '@deepseek-ai/dsh-workflow-workerthread'
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
const run = ctx.workflows.start({
|
||||
script: 'return 6 * 7',
|
||||
meta: { name: 'built-smoke', description: 'built worker smoke' },
|
||||
// A zero-agent script never touches the provider, so a bare id suffices.
|
||||
parent: { id: 'built-smoke-parent', options: {} },
|
||||
})
|
||||
const result = await run.result
|
||||
await run.dispose()
|
||||
if (result.stopReason !== 'completed' || result.value !== 42) {
|
||||
console.error('unexpected result: ' + JSON.stringify(result))
|
||||
process.exit(1)
|
||||
}
|
||||
console.log('built-worker-smoke-ok')
|
||||
`, 'utf8')
|
||||
// Plain node — no tsx loader anywhere; the bundle must stand on its own.
|
||||
const { stdout } = await run(process.execPath, [driver], { cwd: packageRoot, timeout: 60_000 })
|
||||
expect(stdout).toContain('built-worker-smoke-ok')
|
||||
} finally {
|
||||
await rm(driver, { force: true })
|
||||
}
|
||||
}, 120_000)
|
||||
})
|
||||
@@ -0,0 +1,91 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import * as Invariants from '@deepseek-ai/dsh-invariants'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import * as spawn from '@deepseek-ai/dsh-subagent-spawn'
|
||||
import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-inprocess'
|
||||
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
import WorkerWorkflowEngine from '../src/index.ts'
|
||||
|
||||
type Script = ConstructorParameters<typeof MockAdapter>[0]
|
||||
|
||||
/**
|
||||
* The whole in-process stack, keyless, with the script in a REAL worker
|
||||
* thread: the engine drives the REAL spawn backend (with its
|
||||
* structured runtime) on a real agent loop; the scripted mock MODEL is the
|
||||
* only mocked boundary. This is the guard the unit suites structurally
|
||||
* cannot give — the MessageChannel suite fakes the host, and the host suite
|
||||
* stubs the subagent seam.
|
||||
*/
|
||||
async function setup(script: Script) {
|
||||
const ctx = new Context()
|
||||
const adapter = new MockAdapter(script)
|
||||
await ctx.plugin(LlmService)
|
||||
await ctx.plugin(SessionStore)
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(AgentRegistry)
|
||||
await ctx.plugin(Invariants)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(spawn, { providerName: 'spawn' })
|
||||
await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const parent = ctx.agentLoop.create(AgentId('parent'), { model: 'mock' })
|
||||
return { ctx, parent, adapter }
|
||||
}
|
||||
|
||||
describe('dsh-workflow-workerthread over the real in-process stack', () => {
|
||||
it('runs a two-stage workflow: a plain child, then a schema child through the structured runtime', async () => {
|
||||
const { ctx, parent } = await setup([
|
||||
textResponse('the file list is a.ts'),
|
||||
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { verdict: 'real', confidence: 0.9 }),
|
||||
])
|
||||
const childIds: string[] = []
|
||||
ctx.on('workflow/agent-start', (_info, agent) => { childIds.push(agent.childId) })
|
||||
const run = ctx.workflows.start({
|
||||
meta: { name: 'integration', description: 'plain + structured children' },
|
||||
script: `phase('Read')
|
||||
const prose = await agent('read the repo')
|
||||
phase('Judge')
|
||||
const judged = await agent('judge: ' + prose, {
|
||||
schema: { type: 'object', properties: { verdict: { type: 'string', enum: ['real', 'bogus'] }, confidence: { type: 'number' } }, required: ['verdict'] },
|
||||
})
|
||||
return { prose, verdict: judged.verdict, confidence: judged.confidence }`,
|
||||
parent,
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ prose: 'the file list is a.ts', verdict: 'real', confidence: 0.9 })
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
await run.dispose()
|
||||
// Both children were disposed to quiescence — no live child agents remain.
|
||||
expect(childIds.length).toBe(2)
|
||||
for (const childId of childIds) {
|
||||
expect(ctx.agents.get(AgentId(childId))).toBeUndefined()
|
||||
}
|
||||
})
|
||||
|
||||
it('a child that fails against its schema (nudges exhausted) reaches the script as null', async () => {
|
||||
const { ctx, parent } = await setup([
|
||||
textResponse('prose only'),
|
||||
textResponse('still prose after the nudge'),
|
||||
])
|
||||
const run = ctx.workflows.start({
|
||||
meta: { name: 'null-path', description: 'schema failure maps to null' },
|
||||
script: `const judged = await agent('judge it', { schema: { type: 'object', properties: { v: { type: 'string' } } } })
|
||||
return { got: judged === null ? 'null' : 'value' }`,
|
||||
parent,
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ got: 'null' })
|
||||
await run.dispose()
|
||||
})
|
||||
})
|
||||
88
packages/workflow/workflow-workerthread/tests/meta.spec.ts
Normal file
88
packages/workflow/workflow-workerthread/tests/meta.spec.ts
Normal file
@@ -0,0 +1,88 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import { validateMeta } from '../src/meta.ts'
|
||||
|
||||
/** Assert a META_INVALID throw whose message matches every given fragment. */
|
||||
function expectInvalid(value: unknown, ...fragments: string[]): void {
|
||||
let thrown: unknown
|
||||
try {
|
||||
validateMeta(value)
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(WorkflowError)
|
||||
expect((thrown as WorkflowError).code).toBe('META_INVALID')
|
||||
for (const fragment of fragments) {
|
||||
expect((thrown as WorkflowError).message).toContain(fragment)
|
||||
}
|
||||
}
|
||||
|
||||
describe('validateMeta', () => {
|
||||
it('accepts a minimal meta and returns a normalized copy (no aliasing of the input)', () => {
|
||||
const input = { name: 'audit', description: 'audit the repo' }
|
||||
const meta = validateMeta(input)
|
||||
expect(meta).toEqual({ name: 'audit', description: 'audit the repo' })
|
||||
expect(meta).not.toBe(input)
|
||||
input.name = 'mutated'
|
||||
expect(meta.name).toBe('audit')
|
||||
})
|
||||
|
||||
it('accepts the full shape and rebuilds phases entry by entry', () => {
|
||||
const meta = validateMeta({
|
||||
name: 'migrate',
|
||||
description: 'migrate call sites',
|
||||
whenToUse: 'large mechanical sweeps',
|
||||
phases: [
|
||||
{ title: 'Discover' },
|
||||
{ title: 'Transform', detail: 'one agent per file', model: 'deepseek-v4-pro' },
|
||||
],
|
||||
})
|
||||
expect(meta).toEqual({
|
||||
name: 'migrate',
|
||||
description: 'migrate call sites',
|
||||
whenToUse: 'large mechanical sweeps',
|
||||
phases: [
|
||||
{ title: 'Discover' },
|
||||
{ title: 'Transform', detail: 'one agent per file', model: 'deepseek-v4-pro' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects non-object values loud', () => {
|
||||
expectInvalid(undefined, 'meta must be an object')
|
||||
expectInvalid('a string', 'meta must be an object')
|
||||
expectInvalid(null, 'meta must be an object')
|
||||
expectInvalid([{ name: 'x', description: 'd' }], 'meta must be an object')
|
||||
})
|
||||
|
||||
it('rejects unknown fields by name (accepted-then-ignored is banned)', () => {
|
||||
expectInvalid({ name: 'x', description: 'd', color: 'red' }, 'meta.color is not a recognized field')
|
||||
})
|
||||
|
||||
it('rejects missing or mistyped name/description/whenToUse', () => {
|
||||
expectInvalid({ description: 'd' }, 'meta.name must be a non-empty string')
|
||||
expectInvalid({ name: '', description: 'd' }, 'meta.name must be a non-empty string')
|
||||
expectInvalid({ name: 'x' }, 'meta.description must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 42 }, 'meta.description must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 'd', whenToUse: 3 }, 'meta.whenToUse must be a string')
|
||||
})
|
||||
|
||||
it('rejects malformed phases, entry by entry', () => {
|
||||
expectInvalid({ name: 'x', description: 'd', phases: 'Scan' }, 'meta.phases must be an array')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: ['Scan'] }, 'meta.phases[0] must be an object')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: '' }] }, 'meta.phases[0].title must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', order: 1 }] }, 'meta.phases[0].order is not a recognized field')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', detail: 9 }] }, 'meta.phases[0].detail must be a string')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', model: 9 }] }, 'meta.phases[0].model must be a string')
|
||||
})
|
||||
|
||||
it('names EVERY violation in one throw, not just the first', () => {
|
||||
expectInvalid(
|
||||
{ description: 7, extra: true, phases: [{ title: 'Scan' }, 'bad'] },
|
||||
'meta.extra is not a recognized field',
|
||||
'meta.name must be a non-empty string',
|
||||
'meta.description must be a non-empty string',
|
||||
'meta.phases[1] must be an object',
|
||||
)
|
||||
})
|
||||
})
|
||||
145
packages/workflow/workflow-workerthread/tests/realm.spec.ts
Normal file
145
packages/workflow/workflow-workerthread/tests/realm.spec.ts
Normal file
@@ -0,0 +1,145 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import * as vm from 'node:vm'
|
||||
import { materializeFromRealm, MaterializeError, renderThrown } from '../src/realm.ts'
|
||||
|
||||
/** Evaluate an expression inside a fresh vm realm and hand back the raw realm value. */
|
||||
function inRealm(expression: string): unknown {
|
||||
return vm.runInNewContext(`(${expression})`)
|
||||
}
|
||||
|
||||
/** The MaterializeError message for a value that must be rejected (throws if accepted). */
|
||||
function rejection(value: unknown): string {
|
||||
try {
|
||||
materializeFromRealm(value)
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof MaterializeError) return error.message
|
||||
throw error
|
||||
}
|
||||
throw new Error('expected the value to be rejected')
|
||||
}
|
||||
|
||||
describe('materializeFromRealm', () => {
|
||||
it('copies realm objects/arrays/scalars into host plain data', () => {
|
||||
const value = inRealm("{ a: 1, b: 'x', c: true, d: null, list: [1, [2, { deep: 'y' }]] }")
|
||||
const out = materializeFromRealm(value) as Record<string, unknown>
|
||||
expect(out).toEqual({ a: 1, b: 'x', c: true, d: null, list: [1, [2, { deep: 'y' }]] })
|
||||
// The copy is HOST data: prototypes are the host intrinsics.
|
||||
expect(Object.getPrototypeOf(out)).toBe(Object.prototype)
|
||||
expect(Array.isArray(out.list)).toBe(true)
|
||||
// And it round-trips through JSON byte-identically (the whole point).
|
||||
expect(JSON.parse(JSON.stringify(out))).toEqual(out)
|
||||
})
|
||||
|
||||
it('accepts undefined ONLY at the root (a valueless script return)', () => {
|
||||
expect(materializeFromRealm(undefined)).toBeUndefined()
|
||||
expect(rejection(inRealm('{ a: undefined }'))).toContain('value.a')
|
||||
})
|
||||
|
||||
it('invokes getters ordinarily — the getter RESULT is what crosses (trust premise)', () => {
|
||||
const counter = inRealm(`
|
||||
(() => {
|
||||
globalThis.reads = 0
|
||||
return { get x() { globalThis.reads += 1; return globalThis.reads } }
|
||||
})()
|
||||
`)
|
||||
expect(materializeFromRealm(counter)).toEqual({ x: 1 })
|
||||
})
|
||||
|
||||
it('a getter that THROWS surfaces as a MaterializeError carrying the rendered failure', () => {
|
||||
const hostile = inRealm("{ get x() { throw new Error('read failed') } }")
|
||||
const message = rejection(hostile)
|
||||
expect(message).toContain('reading the value threw')
|
||||
expect(message).toContain('read failed')
|
||||
})
|
||||
|
||||
it('a "__proto__" key becomes an OWN data property of the copy, never a prototype mutation', () => {
|
||||
const value: unknown = vm.runInNewContext('JSON.parse(\'{"__proto__": {"polluted": 1}, "ok": 2}\')')
|
||||
const out = materializeFromRealm(value) as Record<string, unknown>
|
||||
expect(Object.getPrototypeOf(out)).toBe(Object.prototype)
|
||||
expect(Object.prototype.hasOwnProperty.call(out, '__proto__')).toBe(true)
|
||||
expect(out.ok).toBe(2)
|
||||
// The host Object.prototype was NOT touched.
|
||||
expect(({} as Record<string, unknown>).polluted).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects functions, symbols (keys and values), and bigints with path-qualified messages', () => {
|
||||
expect(rejection(inRealm('{ fn: () => 1 }'))).toContain('value.fn')
|
||||
expect(rejection(inRealm("{ [Symbol('k')]: 1 }"))).toContain('symbol-keyed')
|
||||
expect(rejection(inRealm("{ s: Symbol('v') }"))).toContain('value.s')
|
||||
expect(rejection(inRealm('{ big: 1n }'))).toContain('value.big')
|
||||
expect(rejection(inRealm("[Symbol('x')]"))).toContain('value[0]')
|
||||
const taggedArray = inRealm("(() => { const a = [1]; a[Symbol('t')] = 1; return a })()")
|
||||
expect(rejection(taggedArray)).toContain('symbol-keyed')
|
||||
})
|
||||
|
||||
it('rejects non-finite numbers and undefined values inside containers', () => {
|
||||
expect(rejection(inRealm('{ n: NaN }'))).toContain('non-finite')
|
||||
expect(rejection(inRealm('[Infinity]'))).toContain('non-finite')
|
||||
})
|
||||
|
||||
it('rejects exotic prototypes (Date, Map, class instances) but accepts null-prototype data', () => {
|
||||
expect(rejection(inRealm('{ d: new Date(0) }'))).toContain('exotic prototype')
|
||||
expect(rejection(inRealm('new Map()'))).toContain('exotic prototype')
|
||||
expect(rejection(inRealm('(() => { class C { constructor() { this.x = 1 } } return new C() })()')))
|
||||
.toContain('exotic prototype')
|
||||
expect(materializeFromRealm(inRealm('Object.assign(Object.create(null), { a: 1 })'))).toEqual({ a: 1 })
|
||||
})
|
||||
|
||||
it('rejects cycles and accepts the same object reused as a sibling (a DAG)', () => {
|
||||
expect(rejection(inRealm('(() => { const o = {}; o.self = o; return o })()'))).toContain('circular')
|
||||
const dag = inRealm('(() => { const leaf = { v: 1 }; return { a: leaf, b: leaf } })()')
|
||||
expect(materializeFromRealm(dag)).toEqual({ a: { v: 1 }, b: { v: 1 } })
|
||||
})
|
||||
|
||||
it('rejects sparse arrays and non-index array properties; an array getter element materializes its value', () => {
|
||||
expect(rejection(inRealm('[1, , 3]'))).toContain('sparse')
|
||||
expect(rejection(inRealm('(() => { const a = [1]; a.total = 3; return a })()')))
|
||||
.toContain('non-index')
|
||||
expect(materializeFromRealm(inRealm('(() => { const a = [1]; Object.defineProperty(a, 0, { get: () => 7, enumerable: true }); return a })()')))
|
||||
.toEqual([7])
|
||||
})
|
||||
|
||||
it('skips non-enumerable own properties (matching JSON.stringify exactly)', () => {
|
||||
const value = inRealm(`(() => {
|
||||
const o = { visible: 1 }
|
||||
Object.defineProperty(o, 'hidden', { value: () => 1, enumerable: false })
|
||||
return o
|
||||
})()`)
|
||||
expect(materializeFromRealm(value)).toEqual({ visible: 1 })
|
||||
})
|
||||
|
||||
it('works on plain host values too (the boundary is realm-agnostic)', () => {
|
||||
expect(materializeFromRealm({ a: [1, 'x'] })).toEqual({ a: [1, 'x'] })
|
||||
expect(materializeFromRealm('str')).toBe('str')
|
||||
expect(materializeFromRealm(3)).toBe(3)
|
||||
expect(materializeFromRealm(false)).toBe(false)
|
||||
expect(materializeFromRealm(null)).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('renderThrown', () => {
|
||||
it('prefers the stack, for host and realm errors alike', () => {
|
||||
const host = renderThrown(new Error('host failure'))
|
||||
expect(host).toContain('host failure')
|
||||
expect(host).toContain('at ') // a real stack, not just the message
|
||||
const realmError: unknown = vm.runInNewContext('(() => { try { throw new Error("realm failure") } catch (e) { return e } })()')
|
||||
expect(renderThrown(realmError)).toContain('realm failure')
|
||||
})
|
||||
|
||||
it('falls back from stack to message to String()', () => {
|
||||
expect(renderThrown({ stack: 'custom data stack' })).toBe('custom data stack')
|
||||
const stackless = new Error('stackless failure')
|
||||
delete stackless.stack
|
||||
expect(renderThrown(stackless)).toBe('stackless failure')
|
||||
expect(renderThrown({ code: 42 })).toBe('[object Object]')
|
||||
expect(renderThrown('plain')).toBe('plain')
|
||||
expect(renderThrown(42)).toBe('42')
|
||||
expect(renderThrown(undefined)).toBe('undefined')
|
||||
expect(renderThrown(null)).toBe('null')
|
||||
})
|
||||
|
||||
it('is total: a value whose accessors/toString throw renders as a fixed label', () => {
|
||||
expect(renderThrown({ get stack() { throw new Error('nope') } })).toBe('[unrenderable thrown value]')
|
||||
expect(renderThrown({ [Symbol.toPrimitive]() { throw new Error('nope') } })).toBe('[unrenderable thrown value]')
|
||||
})
|
||||
})
|
||||
504
packages/workflow/workflow-workerthread/tests/session.spec.ts
Normal file
504
packages/workflow/workflow-workerthread/tests/session.spec.ts
Normal file
@@ -0,0 +1,504 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { MessageChannel } from 'node:worker_threads'
|
||||
import type { MessagePort } from 'node:worker_threads'
|
||||
import { HostToWorkerType, WorkerToHostType } from '../src/protocol.ts'
|
||||
import type { HostToWorkerMessage, WorkerToHostMessage } from '../src/protocol.ts'
|
||||
import { requireParentPort, runWorkerSession } from '../src/session.ts'
|
||||
import type { ChildResult, WorkerInit } from '../src/types.ts'
|
||||
|
||||
/** Default limits for in-process sessions (concurrency pinned; auto is machine-derived). */
|
||||
function limits(overrides?: Partial<WorkerInit['limits']>): WorkerInit['limits'] {
|
||||
return { maxConcurrentAgents: 8, maxTotalAgents: 1000, maxItemsPerCall: 4096, syncTimeoutMs: 5000, ...overrides }
|
||||
}
|
||||
|
||||
/** Wrap a body in the minimal valid meta header (the session receives it pre-extracted). */
|
||||
function init(body: string, args?: unknown, limitOverrides?: Partial<WorkerInit['limits']>): WorkerInit {
|
||||
return {
|
||||
meta: { name: 'test-flow', description: 'a test workflow' },
|
||||
body,
|
||||
...args !== undefined ? { args } : {},
|
||||
limits: limits(limitOverrides),
|
||||
}
|
||||
}
|
||||
|
||||
/** One scripted host over the other end of a MessageChannel. */
|
||||
interface FakeHost {
|
||||
port: MessagePort
|
||||
messages: WorkerToHostMessage[]
|
||||
/** Messages of one type, as they arrive. */
|
||||
ofType<T extends WorkerToHostMessage['type']>(type: T): Extract<WorkerToHostMessage, { type: T }>[]
|
||||
send(message: HostToWorkerMessage): void
|
||||
/** Resolves with the terminal result message. */
|
||||
result(): Promise<Extract<WorkerToHostMessage, { type: 'result' }>['result']>
|
||||
close(): void
|
||||
}
|
||||
|
||||
interface FakeHostOptions {
|
||||
/** Auto-respond to child-start: reply started + settled per child index. Omit a reply to leave the child pending. */
|
||||
reply?: (request: { prompt: string; schema?: unknown; model?: string }, index: number) => ChildResult | undefined
|
||||
/** Reject the start instead (child-start-error) when returning a string. */
|
||||
refuse?: (index: number) => string | undefined
|
||||
/** Auto-send `go` on `ready` (default true). */
|
||||
go?: boolean
|
||||
/** Manual mode: do NOT auto-answer child-start at all (the test scripts the replies). */
|
||||
manual?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Drive runWorkerSession IN-PROCESS over a MessageChannel: this is where the
|
||||
* worker-side files earn their coverage — code inside a real Worker is
|
||||
* invisible to main-process coverage. The fake host mirrors the real host's
|
||||
* protocol discipline (one started/start-error per start; settled/disposed
|
||||
* follow).
|
||||
*/
|
||||
function fakeHost(options?: FakeHostOptions): FakeHost {
|
||||
const channel = new MessageChannel()
|
||||
const messages: WorkerToHostMessage[] = []
|
||||
const resultGate = Promise.withResolvers<Extract<WorkerToHostMessage, { type: 'result' }>['result']>()
|
||||
let childIndex = 0
|
||||
channel.port1.on('message', (message: WorkerToHostMessage) => {
|
||||
messages.push(message)
|
||||
switch (message.type) {
|
||||
case WorkerToHostType.Ready:
|
||||
if (options?.go !== false) channel.port1.postMessage({ type: HostToWorkerType.Go } satisfies HostToWorkerMessage)
|
||||
break
|
||||
case WorkerToHostType.ChildStart: {
|
||||
if (options?.manual) break
|
||||
const index = childIndex
|
||||
childIndex += 1
|
||||
const refusal = options?.refuse?.(index)
|
||||
if (refusal !== undefined) {
|
||||
channel.port1.postMessage(
|
||||
{ type: HostToWorkerType.ChildStartError, callId: message.callId, rendered: refusal } satisfies HostToWorkerMessage,
|
||||
)
|
||||
break
|
||||
}
|
||||
channel.port1.postMessage({ type: HostToWorkerType.ChildStarted, callId: message.callId, childId: `child-${index}` } satisfies HostToWorkerMessage)
|
||||
const reply = options?.reply?.(message.request, index)
|
||||
if (reply !== undefined) {
|
||||
channel.port1.postMessage(
|
||||
{ type: HostToWorkerType.ChildSettled, callId: message.callId, result: reply } satisfies HostToWorkerMessage,
|
||||
)
|
||||
}
|
||||
break
|
||||
}
|
||||
case WorkerToHostType.ChildDispose:
|
||||
channel.port1.postMessage({ type: HostToWorkerType.ChildDisposed, callId: message.callId } satisfies HostToWorkerMessage)
|
||||
break
|
||||
case WorkerToHostType.Result:
|
||||
resultGate.resolve(message.result)
|
||||
break
|
||||
default:
|
||||
break
|
||||
}
|
||||
})
|
||||
return {
|
||||
port: channel.port2,
|
||||
messages,
|
||||
ofType: type => messages.filter((message): message is never => message.type === type),
|
||||
send: (message) => { channel.port1.postMessage(message) },
|
||||
result: () => resultGate.promise,
|
||||
close: () => { channel.port1.close() },
|
||||
}
|
||||
}
|
||||
|
||||
/** A completed text child result. */
|
||||
function text(reply: string): ChildResult {
|
||||
return { output: [{ type: 'text', text: reply }], stopReason: 'completed' }
|
||||
}
|
||||
|
||||
describe('runWorkerSession over an in-process MessageChannel', () => {
|
||||
it('runs a script end to end: ready/go handshake, phases, log, agents, result', async () => {
|
||||
const host = fakeHost({ reply: (_request, index) => text(`answer-${index}`) })
|
||||
const session = runWorkerSession(host.port, init(`
|
||||
phase('Scan')
|
||||
log('starting with ' + args.files.length + ' files')
|
||||
const answers = await pipeline(args.files, (prev, item) => agent('read ' + item))
|
||||
return { answers }
|
||||
`, { files: ['a.ts', 'b.ts'] }))
|
||||
const result = await host.result()
|
||||
await session
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
expect(result.value).toEqual({ answers: ['answer-0', 'answer-1'] })
|
||||
expect(host.messages[0]!.type).toBe('ready')
|
||||
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['Scan'])
|
||||
expect(host.ofType(WorkerToHostType.Log).map(m => m.message)).toEqual(['starting with 2 files'])
|
||||
expect(host.ofType(WorkerToHostType.AgentStart).map(m => m.info.childId)).toEqual(['child-0', 'child-1'])
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd).every(m => m.info.outcome === 'completed')).toBe(true)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('agent({schema}) forwards the schema on the start request and returns the structured value', async () => {
|
||||
const host = fakeHost({ reply: () => ({ output: [], structured: { files: ['x.ts'] }, stopReason: 'completed' }) })
|
||||
void runWorkerSession(host.port, init(`
|
||||
const found = await agent('list files', { schema: { type: 'object', properties: { files: { type: 'array', items: { type: 'string' } } } }, model: 'deepseek-v4-pro' })
|
||||
return { first: found.files[0] }
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual({ first: 'x.ts' })
|
||||
const start = host.ofType(WorkerToHostType.ChildStart)[0]!
|
||||
expect(start.request.schema).toEqual({ type: 'object', properties: { files: { type: 'array', items: { type: 'string' } } } })
|
||||
expect(start.request.model).toBe('deepseek-v4-pro')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a schema child completing WITHOUT a structured value resolves null with a failed outcome', async () => {
|
||||
const host = fakeHost({ reply: () => text('prose, no structure') })
|
||||
void runWorkerSession(host.port, init("return await agent('p', { schema: { type: 'object' } })"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toBeNull()
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('failed')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child settling non-completed resolves null (scripts filter), never throwing into the script', async () => {
|
||||
const host = fakeHost({ reply: (_request, index) => index === 0 ? { output: [], stopReason: 'error' } : text('ok') })
|
||||
void runWorkerSession(host.port, init("return await parallel([() => agent('one'), () => agent('two')])"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual([null, 'ok'])
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd).map(m => m.info.outcome)).toEqual(expect.arrayContaining(['failed', 'completed']))
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a start refusal (child-start-error) is a fatal AGENT_START that kills the script through a combinator', async () => {
|
||||
const host = fakeHost({ refuse: () => 'no provider here' })
|
||||
void runWorkerSession(host.port, init("return await pipeline([1], () => agent('p'))"))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('agent() could not start a child')
|
||||
expect(result.error).toContain('no provider here')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child-failed message (infrastructure rejection) is fatal AGENT_RESULT with the paired failed outcome', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
try { await agent('p'); return 'unreachable' } catch (e) { return { name: e.name, code: e.code, fatal: e.fatal } }
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId, rendered: 'backend exploded' })
|
||||
const result = await host.result()
|
||||
expect(result.value).toMatchObject({ name: 'WorkflowError', code: 'AGENT_RESULT', fatal: true })
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('failed')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancel before go: the body never runs at all and the result is cancelled (a second cancel is a no-op)', async () => {
|
||||
const host = fakeHost({ go: false })
|
||||
const session = runWorkerSession(host.port, init("log('ran')\nreturn 123"))
|
||||
await vi.waitFor(() => { expect(host.messages.some(m => m.type === WorkerToHostType.Ready)).toBe(true) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'aborted before start' })
|
||||
// Idempotence: the first reason wins; a duplicate cancel changes nothing.
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'a later reason that must lose' })
|
||||
const result = await host.result()
|
||||
await session
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('aborted before start')
|
||||
expect(result.error).not.toContain('must lose')
|
||||
expect(result.value).toBeNull()
|
||||
expect(host.ofType(WorkerToHostType.Log)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a script with no return value resolves value: null', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init("await agent('p')"))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBeNull()
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancel mid-run: in-flight children get cancel RPCs, hooks throw at entry, the run reports cancelled', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
phase('before')
|
||||
try { await agent('x') } catch (e) {}
|
||||
try { phase('after') } catch (e) {}
|
||||
try { log('after') } catch (e) {}
|
||||
try { await parallel([() => 'ran']) } catch (e) {}
|
||||
try { await pipeline(['item'], p => p) } catch (e) {}
|
||||
return 'survived by catching'
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'stop everything' })
|
||||
// The real host settles the aborted child; mirror it.
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId, result: { output: [], stopReason: 'aborted' } })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('stop everything')
|
||||
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
|
||||
// No post-cancel narration left the runtime (the hooks threw at entry).
|
||||
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['before'])
|
||||
expect(host.ofType(WorkerToHostType.Log)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancellation between a queued waiter and its slot: the waiter rejects without a child-start', async () => {
|
||||
const host = fakeHost({ go: true })
|
||||
void runWorkerSession(host.port, init(
|
||||
"return await parallel([() => agent('a'), () => agent('b')])",
|
||||
undefined,
|
||||
{ maxConcurrentAgents: 1 },
|
||||
))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'raced' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
// Only the first agent ever reached the host.
|
||||
expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a stray (never-awaited) agent is reaped after settlement: cancel + dispose RPCs flow, no unhandled rejection', async () => {
|
||||
const unhandled: unknown[] = []
|
||||
const onUnhandled = (reason: unknown): void => { unhandled.push(reason) }
|
||||
process.on('unhandledRejection', onUnhandled)
|
||||
try {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
agent('stray, never awaited')
|
||||
return 'done without awaiting'
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId, result: { output: [], stopReason: 'aborted' } })
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId) })
|
||||
await new Promise(resolve => setTimeout(resolve, 20))
|
||||
expect(unhandled).toEqual([])
|
||||
host.close()
|
||||
} finally {
|
||||
process.off('unhandledRejection', onUnhandled)
|
||||
}
|
||||
})
|
||||
|
||||
it('an unparseable body settles an error result instead of dying without one (host pre-parse skew guard)', async () => {
|
||||
const host = fakeHost()
|
||||
await runWorkerSession(host.port, init('return ((('))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('does not parse')
|
||||
expect(result.agentsStarted).toBe(0)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a synchronous spin in the initial slice dies by the in-worker vm timeout', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init('while (true) {}', undefined, { syncTimeoutMs: 50 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error?.toLowerCase()).toContain('timed out')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a non-JSON return value fails loud as RESULT_UNSERIALIZABLE', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init('return { when: new Date(0) }'))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('not plain JSON data')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('tolerates replies for unknown callIds (a teardown race): nothing crashes, the run completes', async () => {
|
||||
const host = fakeHost({ reply: () => text('fine') })
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId: 999, childId: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildStartError, callId: 999, rendered: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId: 999, result: text('ghost') })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId: 999, rendered: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildDisposed, callId: 999 })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBe('fine')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('caps and malformed hook arguments reject loud (the runtime runs unchanged inside the session)', async () => {
|
||||
const cases: [string, string][] = [
|
||||
['return await agent(42)', 'non-empty prompt string'],
|
||||
["return await agent('')", 'non-empty prompt string'],
|
||||
["return await agent('p', 'opts')", 'options must be an object'],
|
||||
["return await agent('p', { label: 3 })", '"label" must be a string'],
|
||||
["return await agent('p', { get label() { throw new Error('read failed') } })", 'options must be plain JSON data'],
|
||||
["return await agent('p', { bogus: true })", '"bogus" is not recognized'],
|
||||
["return await agent('p', { effort: 'high' })", '"effort" is deferred'],
|
||||
["return await agent('p', { schema: { type: 'object', oneOf: [] } })", 'outside the supported subset'],
|
||||
['return await parallel([() => 1, () => 2, () => 3])', 'over the per-call cap (2)'],
|
||||
['return await pipeline([1, 2, 3], (x) => x)', 'maxItemsPerCall'],
|
||||
["return await parallel('no')", 'parallel() requires an array'],
|
||||
['return await parallel([3])', 'item 0 is not a function'],
|
||||
["return await pipeline('no', () => 1)", 'pipeline() requires an items array'],
|
||||
['return await pipeline([1])', 'at least one stage'],
|
||||
["return await pipeline([1], 'x')", 'stage 0 is not a function'],
|
||||
["phase('')", 'phase() requires a non-empty title string'],
|
||||
['log(3)', 'log() requires a message string'],
|
||||
]
|
||||
for (const [body, expected] of cases) {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init(body, undefined, { maxItemsPerCall: 2 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain(expected)
|
||||
host.close()
|
||||
}
|
||||
})
|
||||
|
||||
it('combinator semantics: thunk/stage throws null the item; a forged fatal-shaped object stays null; real fatals propagate', async () => {
|
||||
const host = fakeHost({ reply: () => text('fine') })
|
||||
void runWorkerSession(host.port, init(`
|
||||
const viaParallel = await parallel([
|
||||
() => { throw new Error('boom') },
|
||||
() => agent('fine'),
|
||||
() => 'plain value',
|
||||
() => { throw { name: 'WorkflowError', fatal: true, message: 'forged fatal' } },
|
||||
])
|
||||
const viaPipeline = await pipeline([10, 20],
|
||||
(prev, item, index) => { if (item === 10) throw new Error('ordinary failure'); return 'kept-' + item + '-' + index },
|
||||
)
|
||||
return { viaParallel, viaPipeline }
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({
|
||||
viaParallel: [null, 'fine', 'plain value', null],
|
||||
viaPipeline: [null, 'kept-20-1'],
|
||||
})
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('trips the total-agent cap with a message naming the config knob', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init("await agent('1'); await agent('2'); await agent('3')", undefined, { maxTotalAgents: 2 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('total agent cap (2)')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('queued agents proceed through the concurrency semaphore in FIFO order', async () => {
|
||||
const host = fakeHost({ reply: request => text(`ok:${request.prompt}`) })
|
||||
void runWorkerSession(host.port, init(
|
||||
"return await parallel([1, 2, 3].map((n) => () => agent('job ' + n)))",
|
||||
undefined,
|
||||
{ maxConcurrentAgents: 1 },
|
||||
))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual(['ok:job 1', 'ok:job 2', 'ok:job 3'])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('labels default from the prompt first line, truncated; explicit label/phase options win', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init(`
|
||||
phase('Find')
|
||||
await agent('a prompt that is quite long and will surely get truncated down to a display label\\n'
|
||||
+ 'with a second line the label must not include')
|
||||
await agent('short', { label: 'named', phase: 'Custom' })
|
||||
return null
|
||||
`))
|
||||
await host.result()
|
||||
const starts = host.ofType(WorkerToHostType.AgentStart).map(m => m.info)
|
||||
expect(starts[0]).toMatchObject({ seq: 1, phase: 'Find' })
|
||||
expect(starts[0]!.label.length).toBeLessThanOrEqual(48)
|
||||
expect(starts[0]!.label).not.toContain('second line')
|
||||
expect(starts[1]).toMatchObject({ seq: 2, label: 'named', phase: 'Custom' })
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('non-text output blocks are filtered out of the text result', async () => {
|
||||
const host = fakeHost({
|
||||
reply: () => ({
|
||||
output: [
|
||||
{ type: 'text', text: 'first ' },
|
||||
{ type: 'tool_call', id: 'c1', name: 'x', arguments: {} } as never,
|
||||
{ type: 'text', text: 'second' },
|
||||
],
|
||||
stopReason: 'completed',
|
||||
}),
|
||||
})
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toBe('first second')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a cancel landing DURING the start round-trip winds the fresh child down (cancel + dispose) and dies cancelled', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
// Cancel FIRST, then the (stale) started reply: the worker processes them
|
||||
// in order, so the agent() continuation resumes already-cancelled — the
|
||||
// window the real host cannot produce (it refuses starts once cancelled)
|
||||
// but a teardown race can.
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'raced the start' })
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
await vi.waitFor(() => {
|
||||
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
|
||||
expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId)
|
||||
})
|
||||
// The child never became an agent-start: it was wound down pre-lifecycle.
|
||||
expect(host.ofType(WorkerToHostType.AgentStart)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a start refusal arriving after a cancel reads as the cancellation, not a broken seam', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init(`
|
||||
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code } }
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'stopping' })
|
||||
host.send({ type: HostToWorkerType.ChildStartError, callId, rendered: 'workflow run cancelled: stopping' })
|
||||
const result = await host.result()
|
||||
// The run reports cancelled (the script died of CANCELLED, not AGENT_START).
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child result rejection while cancelled pairs a cancelled agent-end, and the run reports cancelled', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init("return await agent('doomed')"))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.AgentStart).length).toBe(1) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'user aborted' })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId, rendered: 'backend crashed on abort' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
|
||||
host.close()
|
||||
})
|
||||
|
||||
})
|
||||
|
||||
describe('the worker bootstrap', () => {
|
||||
it('requireParentPort narrows a real port and throws on the main thread', () => {
|
||||
const channel = new MessageChannel()
|
||||
expect(requireParentPort(channel.port1)).toBe(channel.port1)
|
||||
channel.port1.close()
|
||||
expect(() => requireParentPort(null)).toThrow(/inside a worker thread/)
|
||||
})
|
||||
|
||||
it('the entry module itself throws when loaded on the main thread (no parentPort)', async () => {
|
||||
// This import EXECUTES ../src/worker.ts on the main thread, which is what
|
||||
// covers the bootstrap file: requireParentPort throws before
|
||||
// runWorkerSession is reached.
|
||||
await expect(import('../src/worker.ts')).rejects.toThrow(/inside a worker thread/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,102 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import LlmService from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import * as Spawn from '@deepseek-ai/dsh-subagent-spawn'
|
||||
import WorkerWorkflowEngine from '../src/index.ts'
|
||||
|
||||
/**
|
||||
* With-key e2e: a REAL script in a REAL worker thread
|
||||
* drives REAL spawn children against the live DeepSeek API — one plain child
|
||||
* and one schema'd child through the real structured-output runtime — and
|
||||
* the run's value, events, and child sessions are asserted from the outside
|
||||
* (never the script's self-report alone). Key-gated (self-skips without
|
||||
* DEEPSEEK_API_KEY).
|
||||
*/
|
||||
|
||||
let ctx: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
})
|
||||
|
||||
async function harness(): Promise<Context> {
|
||||
const built = new Context()
|
||||
await built.plugin(LlmService)
|
||||
await built.plugin(SessionStore)
|
||||
await built.plugin(SystemPrompt)
|
||||
await built.plugin(ToolRegistry)
|
||||
await built.plugin(AgentRegistry)
|
||||
await built.plugin(AgentLoop, { agents: [] })
|
||||
await built.plugin(LlmDeepSeek, { models: ['deepseek-v4-flash'] })
|
||||
await built.plugin(SubagentService)
|
||||
await built.plugin(Spawn, { providerName: 'spawn' })
|
||||
await built.plugin(WorkerWorkflowEngine, { provider: 'spawn' })
|
||||
return built
|
||||
}
|
||||
|
||||
const META = {
|
||||
name: 'e2e-worker-arithmetic',
|
||||
description: 'two real children through a worker thread: one prose, one structured',
|
||||
phases: [{ title: 'Ask' }, { title: 'Judge' }],
|
||||
}
|
||||
const SCRIPT = `phase('Ask')
|
||||
log('asking the prose child')
|
||||
const prose = await agent('Reply with exactly one short sentence: what is 2 + 2?')
|
||||
phase('Judge')
|
||||
const judged = await agent(
|
||||
'Here is an answer to the question "what is 2+2": ' + prose
|
||||
+ ' — report whether it contains the number 4 and your confidence between 0 and 1.',
|
||||
{ schema: { type: 'object', properties: { containsFour: { type: 'boolean' }, confidence: { type: 'number' } }, required: ['containsFour'] } },
|
||||
)
|
||||
return { prose, containsFour: judged === null ? null : judged.containsFour }`
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('worker workflow engine with-key e2e', () => {
|
||||
it('runs a two-phase script in a worker thread over real children, one through the structured runtime', async () => {
|
||||
ctx = await harness()
|
||||
const parentHandle = ctx.agents.create({
|
||||
agentId: AgentId('wf-worker-e2e-parent'),
|
||||
sessionId: 'wf-worker-e2e-session' as never,
|
||||
agentOptions: { model: 'deepseek-v4-flash' },
|
||||
})
|
||||
|
||||
const events: string[] = []
|
||||
const childIds: string[] = []
|
||||
for (const name of ['workflow/start', 'workflow/phase', 'workflow/log', 'workflow/agent-start', 'workflow/agent-end', 'workflow/end'] as const) {
|
||||
ctx.on(name, (...payload: unknown[]) => {
|
||||
events.push(name)
|
||||
if (name === 'workflow/agent-start') childIds.push((payload[1] as { childId: string }).childId)
|
||||
})
|
||||
}
|
||||
|
||||
const run = ctx.workflows.start({ script: SCRIPT, meta: META, parent: parentHandle.agent })
|
||||
const result = await run.result
|
||||
await run.dispose()
|
||||
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
const value = result.value as { prose: string; containsFour: boolean | null }
|
||||
// World checks: the prose child really answered (a real completion), and
|
||||
// the structured child judged it against the REAL schema-forced tool.
|
||||
expect(value.prose.length).toBeGreaterThan(0)
|
||||
expect(value.containsFour).toBe(true)
|
||||
|
||||
expect(events[0]).toBe('workflow/start')
|
||||
expect(events.at(-1)).toBe('workflow/end')
|
||||
expect(events.filter(name => name === 'workflow/phase').length).toBe(2)
|
||||
expect(events.filter(name => name === 'workflow/agent-start').length).toBe(2)
|
||||
expect(childIds.length).toBe(2)
|
||||
// The children were disposed to quiescence after collection.
|
||||
for (const childId of childIds) {
|
||||
expect(ctx.agents.get(AgentId(childId))).toBeUndefined()
|
||||
}
|
||||
await parentHandle.dispose()
|
||||
}, 240_000)
|
||||
})
|
||||
@@ -0,0 +1,904 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import { AgentId } from '@deepseek-ai/dsh-agent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentCapabilities, SubagentProvider, SubagentResult, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
|
||||
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
|
||||
import * as workerEngineModule from '../src/index.ts'
|
||||
import WorkerWorkflowEngine, { type Config } from '../src/index.ts'
|
||||
|
||||
/** A minimal parent stand-in: the engine only threads it through to the provider. */
|
||||
function fakeParent(): Agent {
|
||||
return { id: AgentId('workflow-parent'), options: {} } as unknown as Agent
|
||||
}
|
||||
|
||||
/** The vm-context escape hatch, spelled once: real Worker tests use it to make the WORKER misbehave. */
|
||||
const ESCAPE = "globalThis.constructor.constructor('return process')()"
|
||||
|
||||
/** One controllable child run: the test (or auto mode) settles it. */
|
||||
interface ControlledRun {
|
||||
request: SubagentStartRequest
|
||||
settle(result: SubagentResult): void
|
||||
cancelled: string | undefined
|
||||
disposed: boolean
|
||||
disposeCalls: number
|
||||
}
|
||||
|
||||
/**
|
||||
* A scripted in-test provider over the REAL SubagentService registry: `auto`
|
||||
* settles each run via the reply function on a microtask; `manual` piles runs
|
||||
* up in `runs` for the test to settle. A run aborts (settles `aborted`) when
|
||||
* the request signal fires, like the real in-process backends.
|
||||
*/
|
||||
class StubProvider implements SubagentProvider {
|
||||
readonly capabilities: SubagentCapabilities = { outputSchema: true, depthLimit: true, toolFilter: true }
|
||||
readonly inheritsParentContext = false
|
||||
readonly runs: ControlledRun[] = []
|
||||
|
||||
constructor(
|
||||
readonly name: string,
|
||||
private readonly reply?: (request: SubagentStartRequest, index: number) => SubagentResult,
|
||||
private readonly disposeDelayMs = 0,
|
||||
) {}
|
||||
|
||||
start(request: SubagentStartRequest): SubagentRun {
|
||||
let settle!: (result: SubagentResult) => void
|
||||
const result = new Promise<SubagentResult>((resolve) => { settle = resolve })
|
||||
const controlled: ControlledRun = { request, settle, cancelled: undefined, disposed: false, disposeCalls: 0 }
|
||||
this.runs.push(controlled)
|
||||
const index = this.runs.length - 1
|
||||
request.signal?.addEventListener('abort', () => { settle({ output: [], stopReason: 'aborted' }) }, { once: true })
|
||||
if (this.reply) {
|
||||
const reply = this.reply
|
||||
queueMicrotask(() => { settle(reply(request, index)) })
|
||||
}
|
||||
return {
|
||||
id: AgentId(`stub-child-${index}`),
|
||||
result,
|
||||
cancel: (reason?: string) => {
|
||||
controlled.cancelled = reason ?? 'cancelled'
|
||||
settle({ output: [], stopReason: 'aborted' })
|
||||
},
|
||||
dispose: () => {
|
||||
controlled.disposeCalls += 1
|
||||
if (this.disposeDelayMs === 0) {
|
||||
controlled.disposed = true
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise<void>((resolve) => {
|
||||
setTimeout(() => {
|
||||
controlled.disposed = true
|
||||
resolve()
|
||||
}, this.disposeDelayMs)
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Text-reply helper for auto providers. */
|
||||
function text(reply: string): SubagentResult {
|
||||
return { output: [{ type: 'text', text: reply }], stopReason: 'completed' }
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
config?: Config
|
||||
reply?: (request: SubagentStartRequest, index: number) => SubagentResult
|
||||
manual?: boolean
|
||||
disposeDelayMs?: number
|
||||
}
|
||||
|
||||
async function setup(options?: SetupOptions) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const provider = new StubProvider(
|
||||
'stub',
|
||||
options?.manual ? undefined : options?.reply ?? (() => text('stub reply')),
|
||||
options?.disposeDelayMs ?? 0,
|
||||
)
|
||||
ctx.subagents.registerProvider(provider)
|
||||
// A fixed concurrency ceiling: the auto-resolved default is machine-derived
|
||||
// (cores - 2, floored at 1), so tests that expect N children in flight
|
||||
// would wedge on small CI runners.
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'stub', maxConcurrentAgents: 8, ...options?.config })
|
||||
return { ctx, provider, parent: fakeParent() }
|
||||
}
|
||||
|
||||
/** The standard test meta plus a body, spread into a start request. */
|
||||
function scripted(body: string, metaExtra?: Partial<WorkflowMeta>): { script: string; meta: WorkflowMeta } {
|
||||
return { script: body, meta: { name: 'test-flow', description: 'a test workflow', ...metaExtra } }
|
||||
}
|
||||
|
||||
/** Start + await one run, disposing on the way out. */
|
||||
async function run(ctx: Context, parent: Agent, source: { script: string; meta: WorkflowMeta }, args?: unknown): Promise<WorkflowResult> {
|
||||
const handle = ctx.workflows.start({ ...source, parent, ...args !== undefined ? { args } : {} })
|
||||
try {
|
||||
return await handle.result
|
||||
} finally {
|
||||
await handle.dispose()
|
||||
}
|
||||
}
|
||||
|
||||
describe('dsh-workflow-workerthread', () => {
|
||||
describe('script execution over a real worker thread', () => {
|
||||
it('runs a script end-to-end: agent() text results, phases, log, args, return value, events', async () => {
|
||||
const { ctx, parent, provider } = await setup({ reply: (_request, index) => text(`answer-${index}`) })
|
||||
const events: [string, unknown[]][] = []
|
||||
for (const name of ['workflow/start', 'workflow/phase', 'workflow/log', 'workflow/agent-start', 'workflow/agent-end', 'workflow/end'] as const) {
|
||||
ctx.on(name, (...payload: unknown[]) => { events.push([name, payload]) })
|
||||
}
|
||||
const result = await run(ctx, parent, scripted(`
|
||||
phase('Scan')
|
||||
log('starting with ' + args.files.length + ' files')
|
||||
const answers = await pipeline(args.files, (prev, item) => agent('read ' + item))
|
||||
phase('Report')
|
||||
return { answers, count: args.files.length }
|
||||
`, { phases: [{ title: 'Scan' }, { title: 'Report' }] }), { files: ['a.ts', 'b.ts'] })
|
||||
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
expect(result.value).toEqual({ answers: ['answer-0', 'answer-1'], count: 2 })
|
||||
expect(provider.runs.every(r => r.disposed)).toBe(true)
|
||||
|
||||
const names = events.map(([name]) => name)
|
||||
expect(names[0]).toBe('workflow/start')
|
||||
expect(names).toContain('workflow/phase')
|
||||
expect(names).toContain('workflow/log')
|
||||
expect(names.at(-1)).toBe('workflow/end')
|
||||
const info = events[0]![1][0] as WorkflowRunInfo
|
||||
expect(info.meta.name).toBe('test-flow')
|
||||
const end = events.at(-1)![1][1] as Record<string, unknown>
|
||||
expect(end).toEqual({ stopReason: 'completed', agentsStarted: 2 })
|
||||
expect('value' in end).toBe(false)
|
||||
})
|
||||
|
||||
it('agent({schema, model}) forwards outputSchema and agentOptions to the provider across the thread', async () => {
|
||||
const { ctx, parent, provider } = await setup({
|
||||
reply: () => ({ output: [], structured: { files: ['x.ts', 'y.ts'] }, stopReason: 'completed' }),
|
||||
})
|
||||
const result = await run(ctx, parent, scripted(`
|
||||
const found = await agent('list files', { model: 'deepseek-v4-pro', schema: { type: 'object', properties: { files: { type: 'array', items: { type: 'string' } } }, required: ['files'] } })
|
||||
return { first: found.files[0], count: found.files.length }
|
||||
`))
|
||||
expect(result.value).toEqual({ first: 'x.ts', count: 2 })
|
||||
expect(provider.runs[0]!.request.outputSchema).toEqual({
|
||||
type: 'object',
|
||||
properties: { files: { type: 'array', items: { type: 'string' } } },
|
||||
required: ['files'],
|
||||
})
|
||||
expect(provider.runs[0]!.request.agentOptions).toEqual({ model: 'deepseek-v4-pro' })
|
||||
expect(provider.runs[0]!.request.parent).toBeDefined()
|
||||
})
|
||||
|
||||
it('a fatal hook error inside the worker kills the script and reports the error', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
const result = await run(ctx, parent, scripted("return await parallel([() => agent('x', { isolation: 'worktree' })])"))
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('"isolation" is deferred')
|
||||
})
|
||||
|
||||
it('a provider start failure crosses back as a fatal AGENT_START error', async () => {
|
||||
const { ctx, parent } = await setup({ config: { provider: 'nonexistent' } })
|
||||
const result = await run(ctx, parent, scripted("return await pipeline([1], () => agent('p'))"))
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('agent() could not start a child')
|
||||
})
|
||||
|
||||
it('a child result REJECTION crosses back as a fatal AGENT_RESULT error (a broken provider is not a failed child)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const provider: SubagentProvider = {
|
||||
name: 'rejecting',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => ({
|
||||
id: AgentId('reject-child'),
|
||||
result: Promise.reject(new Error('backend exploded')),
|
||||
cancel: () => { /* nothing in flight */ },
|
||||
dispose: () => Promise.resolve(),
|
||||
}),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'rejecting', maxConcurrentAgents: 2 })
|
||||
const result = await run(ctx, fakeParent(), scripted(`
|
||||
try { await agent('p'); return 'unreachable' } catch (e) { return { name: e.name, code: e.code, fatal: e.fatal, message: e.message } }
|
||||
`))
|
||||
expect(result.value).toMatchObject({ name: 'WorkflowError', code: 'AGENT_RESULT', fatal: true })
|
||||
expect((result.value as { message: string }).message).toContain('backend exploded')
|
||||
})
|
||||
|
||||
it('a child whose dispose() rejects cannot wedge the script (the host acks anyway)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const provider: SubagentProvider = {
|
||||
name: 'bad-dispose',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => ({
|
||||
id: AgentId('bad-dispose-child'),
|
||||
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
|
||||
cancel: () => { /* settled already */ },
|
||||
dispose: () => Promise.reject(new Error('dispose exploded')),
|
||||
}),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'bad-dispose', maxConcurrentAgents: 2 })
|
||||
const result = await run(ctx, fakeParent(), scripted("return await agent('p')"))
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBe('fine')
|
||||
})
|
||||
|
||||
it('a child dispose() rejecting an UNRENDERABLE value still acks — the containment warn is total', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const provider: SubagentProvider = {
|
||||
name: 'coercion-trap-dispose',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => ({
|
||||
id: AgentId('trap-child'),
|
||||
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
|
||||
cancel: () => { /* settled already */ },
|
||||
// The rejection VALUE's own coercion throws: a warn built with bare
|
||||
// String(error) would itself throw, skipping the ChildDisposed ack
|
||||
// and wedging the script's finally until the grace/terminate path.
|
||||
// eslint-disable-next-line @typescript-eslint/prefer-promise-reject-errors -- the non-Error rejection IS the scenario under test
|
||||
dispose: () => Promise.reject({ toString: () => { throw new Error('coercion trap') } }),
|
||||
}),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'coercion-trap-dispose', maxConcurrentAgents: 2 })
|
||||
const result = await run(ctx, fakeParent(), scripted("return await agent('p')"))
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBe('fine')
|
||||
})
|
||||
|
||||
it('the worker spawns with an EMPTY environment: an escaped script finds no ambient credentials', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
// A canary in the HARNESS process's env: with an inherited environment
|
||||
// the escape below would read it back (exactly how DEEPSEEK_API_KEY
|
||||
// would leak); env: {} in the spawn options is what keeps it out.
|
||||
process.env.WORKFLOW_ENV_CANARY = 'leak me'
|
||||
try {
|
||||
const result = await run(ctx, parent, scripted(`
|
||||
const proc = ${ESCAPE}
|
||||
return { canary: proc.env.WORKFLOW_ENV_CANARY ?? null, keys: Object.keys(proc.env).length }
|
||||
`))
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ canary: null, keys: 0 })
|
||||
} finally {
|
||||
delete process.env.WORKFLOW_ENV_CANARY
|
||||
}
|
||||
})
|
||||
|
||||
it('the unbuilt worker forwards exactly TSX_TSCONFIG_PATH through the scrub: the paths-map pin survives, secrets do not', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
// The ACP snapshot harness runs the parent with its cwd OUTSIDE the
|
||||
// repo and pins the repo tsconfig through this variable; the worker
|
||||
// must inherit the pin (or its dsh-* imports silently resolve to
|
||||
// unbuilt lib/ bundles) while every other variable stays scrubbed.
|
||||
const tsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url))
|
||||
process.env.TSX_TSCONFIG_PATH = tsconfig
|
||||
process.env.WORKFLOW_ENV_CANARY = 'leak me'
|
||||
try {
|
||||
const result = await run(ctx, parent, scripted(`
|
||||
const proc = ${ESCAPE}
|
||||
return { keys: Object.keys(proc.env), tsconfig: proc.env.TSX_TSCONFIG_PATH }
|
||||
`))
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ keys: ['TSX_TSCONFIG_PATH'], tsconfig })
|
||||
} finally {
|
||||
delete process.env.TSX_TSCONFIG_PATH
|
||||
delete process.env.WORKFLOW_ENV_CANARY
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('lifecycle: parse errors, cancellation, termination, disposal', () => {
|
||||
it('start() throws synchronously for invalid meta data or an unparseable body (host-side pre-checks)', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
// Meta is DATA — shape violations reject loud, every one named.
|
||||
expect(() => ctx.workflows.start({ script: 'return 1', meta: { name: '', description: 'd' }, parent })).toThrow(/meta\.name must be a non-empty string/)
|
||||
expect(() => ctx.workflows.start({ script: 'return 1', meta: { name: 'x', description: 'd', extra: 1 } as unknown as WorkflowMeta, parent })).toThrow(/META_INVALID|not a recognized field/)
|
||||
expect(() => ctx.workflows.start({ ...scripted('return ((('), parent })).toThrow(/does not parse/)
|
||||
// The likeliest authoring slip — a Claude Code-style meta header in the
|
||||
// body — gets a pointed message, not a bare SyntaxError.
|
||||
expect(() => ctx.workflows.start({ ...scripted("export const meta = { name: 'x', description: 'd' }\nreturn 1"), parent })).toThrow(/meta rides the `meta` request field/)
|
||||
})
|
||||
|
||||
it('cancel() aborts in-flight children (signal AND cancel RPC) and settles the run cancelled', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const ends: unknown[] = []
|
||||
ctx.on('workflow/agent-end', (_info, agent) => { ends.push(agent) })
|
||||
const runEnds: WorkflowResultInfo[] = []
|
||||
ctx.on('workflow/end', (_info, result) => { runEnds.push(result) })
|
||||
const handle = ctx.workflows.start({ ...scripted("return await agent('long job')"), parent })
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(1) })
|
||||
handle.cancel('user stopped it')
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('user stopped it')
|
||||
await handle.dispose()
|
||||
expect(provider.runs[0]!.disposed).toBe(true)
|
||||
expect(ends).toEqual([expect.objectContaining({ seq: 1, outcome: 'cancelled' })])
|
||||
// workflow/end is an observer's only death signal: it fires for a
|
||||
// cancelled run too, mirroring the settled outcome data.
|
||||
expect(runEnds).toEqual([{ stopReason: 'cancelled', error: result.error, agentsStarted: result.agentsStarted }])
|
||||
})
|
||||
|
||||
it('an already-aborted request signal cancels before the body ever runs (the go handshake holds it)', async () => {
|
||||
const { ctx, parent, provider } = await setup()
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
const logs: string[] = []
|
||||
ctx.on('workflow/log', (_info, message) => { logs.push(message) })
|
||||
const handle = ctx.workflows.start({ ...scripted("log('ran')\nreturn 123"), parent, signal: controller.signal })
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.value).toBeNull()
|
||||
expect(logs).toEqual([])
|
||||
expect(provider.runs.length).toBe(0)
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it('cancel() right after start() cancels before the body runs; the signal aborting mid-run cancels like cancel()', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const first = ctx.workflows.start({ ...scripted("return await agent('never')"), parent })
|
||||
// No-reason cancel: the canonical default reason must ride the result.
|
||||
first.cancel()
|
||||
const firstResult = await first.result
|
||||
expect(firstResult.stopReason).toBe('cancelled')
|
||||
expect(firstResult.error).toContain('workflow cancelled')
|
||||
expect(provider.runs.length).toBe(0)
|
||||
await first.dispose()
|
||||
|
||||
const controller = new AbortController()
|
||||
const second = ctx.workflows.start({ ...scripted("return await agent('job')"), parent, signal: controller.signal })
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(1) })
|
||||
controller.abort()
|
||||
expect((await second.result).stopReason).toBe('cancelled')
|
||||
await second.dispose()
|
||||
})
|
||||
|
||||
it('a child-start racing the host cancel is refused: no child starts after cancellation', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
// Cancel from INSIDE the log listener: the worker has already posted
|
||||
// its child-start (queued right behind the log message), so the host
|
||||
// processes it with cancelReason set — the refusal arm no real-world
|
||||
// timing can hit reliably. (The closure runs only after `handle` below
|
||||
// is initialized — the listener fires on the worker's first message.)
|
||||
ctx.on('workflow/log', () => { handle.cancel('cancelled from the log listener') })
|
||||
const handle = ctx.workflows.start({ ...scripted("log('mark')\nreturn await agent('late')"), parent })
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(provider.runs.length).toBe(0)
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it('post-cancel narration is suppressed host-side, and completion racing a cancel reports cancelled', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
const narration: string[] = []
|
||||
ctx.on('workflow/log', (_info, message) => { narration.push(message) })
|
||||
ctx.on('workflow/phase', (_info, title) => { narration.push(`phase:${title}`) })
|
||||
const handle = ctx.workflows.start({
|
||||
// The sync spin keeps the worker's loop busy so the cancel message
|
||||
// cannot be processed before the script settles `completed` — the
|
||||
// worker posts a completed result that must LOSE to the in-flight
|
||||
// host cancellation. The trailing narration exercises host-side
|
||||
// suppression: posted pre-cancel-processing worker-side, arriving
|
||||
// post-cancel host-side.
|
||||
...scripted(`
|
||||
log('started')
|
||||
const end = Date.now() + 1000
|
||||
while (Date.now() < end) {}
|
||||
phase('late phase')
|
||||
log('late log')
|
||||
return 'done'
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(narration).toContain('started') })
|
||||
handle.cancel('raced the completion')
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('raced the completion')
|
||||
expect(narration).toEqual(['started'])
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('cancel() force-settles a script parked on a promise no hook owns, and TERMINATES its worker', async () => {
|
||||
const { ctx, parent } = await setup({ config: { provider: 'stub', disposeGraceMs: 50 } })
|
||||
const runEnds: WorkflowResultInfo[] = []
|
||||
ctx.on('workflow/end', (_info, result) => { runEnds.push(result) })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted("await new Promise(() => {})\nreturn 'unreachable'"),
|
||||
parent,
|
||||
})
|
||||
handle.cancel('user aborted')
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('user aborted')
|
||||
// The grace force-settle fires workflow/end exactly like an ordinary
|
||||
// settlement — a terminated script's death still reaches observers.
|
||||
expect(runEnds).toEqual([{ stopReason: 'cancelled', error: result.error, agentsStarted: 0 }])
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it('dispose() on a stuck script returns within the grace instead of hanging (result settles cancelled)', async () => {
|
||||
const { ctx, parent } = await setup({ config: { provider: 'stub', disposeGraceMs: 50 } })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted("await new Promise(() => {})\nreturn 'unreachable'"),
|
||||
parent,
|
||||
})
|
||||
const before = Date.now()
|
||||
await handle.dispose()
|
||||
expect(Date.now() - before).toBeLessThan(2000)
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
})
|
||||
|
||||
it('dispose() is idempotent and settles cleanly after a completed run', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
const handle = ctx.workflows.start({ ...scripted('return 1'), parent })
|
||||
await handle.result
|
||||
await handle.dispose()
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it('a settled run arms NO grace timer: disposing a completed run must not pin it for disposeGraceMs', async () => {
|
||||
// A distinctive grace so the spy can tell the cancel-path grace timer
|
||||
// apart from every other timeout in flight.
|
||||
const GRACE = 44_444
|
||||
const { ctx, parent } = await setup({ config: { provider: 'stub', disposeGraceMs: GRACE } })
|
||||
const handle = ctx.workflows.start({ ...scripted('return 1'), parent })
|
||||
await handle.result
|
||||
const spy = vi.spyOn(globalThis, 'setTimeout')
|
||||
try {
|
||||
await handle.dispose()
|
||||
// dispose()'s own bounded-wait sleep is the ONLY grace-sized timer
|
||||
// allowed here; before the settled guard, cancel() armed a second one
|
||||
// that nothing would ever clear (the run was already settled), keeping
|
||||
// the WorkerRun/Worker closure alive until the grace expired.
|
||||
const graceTimers = spy.mock.calls.filter(call => call[1] === GRACE)
|
||||
expect(graceTimers.length).toBe(1)
|
||||
} finally {
|
||||
spy.mockRestore()
|
||||
}
|
||||
})
|
||||
|
||||
it('strays: children fired without await are aborted once the script settles, and dispose() waits for their disposal', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true, disposeDelayMs: 40 })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted(`
|
||||
agent('stray')
|
||||
return 'done without awaiting'
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(1) })
|
||||
await handle.dispose()
|
||||
// Not a waitFor: by the time dispose() returns, the slow child disposal
|
||||
// must already be complete (host-side registry quiescence).
|
||||
expect(provider.runs[0]!.disposed).toBe(true)
|
||||
})
|
||||
|
||||
it('the settle-reap fires the request signal too: a provider honoring ONLY the signal winds its stray down promptly', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const aborted: string[] = []
|
||||
const provider: SubagentProvider = {
|
||||
name: 'signal-only',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: (request) => {
|
||||
let settle!: (result: SubagentResult) => void
|
||||
const result = new Promise<SubagentResult>((resolve) => { settle = resolve })
|
||||
request.signal?.addEventListener('abort', () => {
|
||||
aborted.push(String(request.signal?.reason))
|
||||
settle({ output: [], stopReason: 'aborted' })
|
||||
}, { once: true })
|
||||
return {
|
||||
id: AgentId('signal-only-child'),
|
||||
result,
|
||||
// The seam leaves a provider free to honor EITHER cancel channel;
|
||||
// this one deliberately ignores run.cancel() — only the request
|
||||
// signal can wind it down.
|
||||
cancel: () => { /* signal-only by design */ },
|
||||
dispose: () => Promise.resolve(),
|
||||
}
|
||||
},
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'signal-only', maxConcurrentAgents: 2 })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted(`
|
||||
agent('stray, never awaited')
|
||||
return 'done'
|
||||
`),
|
||||
parent: fakeParent(),
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
// BEFORE dispose(): the settlement itself must have aborted the signal —
|
||||
// without it this child would stay live until dispose's terminate.
|
||||
await vi.waitFor(() => { expect(aborted).toEqual(['workflow settled']) })
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it("cancel() drives each child's explicit cancel() host-side: a wedged worker cannot delay it", async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
let starts = 0
|
||||
const cancelled: string[] = []
|
||||
const provider: SubagentProvider = {
|
||||
name: 'cancel-only',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => {
|
||||
starts += 1
|
||||
return {
|
||||
id: AgentId('cancel-only-child'),
|
||||
result: new Promise(() => { /* only cancel() ends this child */ }),
|
||||
// Deliberately ignores the request signal — the seam leaves a
|
||||
// provider free to honor ONLY the explicit cancel() channel.
|
||||
cancel: (reason?: string) => { cancelled.push(reason ?? 'cancelled') },
|
||||
dispose: () => Promise.resolve(),
|
||||
}
|
||||
},
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
// A deliberately huge grace: if only the grace/terminate reap could
|
||||
// reach this child, the assertion below would time out first.
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'cancel-only', maxConcurrentAgents: 2, disposeGraceMs: 30_000 })
|
||||
const handle = ctx.workflows.start({
|
||||
// The stray child's start RPC reaches the host, then the script wedges
|
||||
// its own worker in a synchronous spin: the worker cannot process the
|
||||
// Cancel message, so it can relay NO ChildCancel RPC — only the host's
|
||||
// own children loop can deliver the explicit cancel in time. The
|
||||
// microtask yields let the agent() continuation POST its child-start
|
||||
// before the spin seizes the worker's loop (the posted message needs
|
||||
// no further worker-loop turns to reach the host).
|
||||
...scripted(`
|
||||
agent('wedged child')
|
||||
for (let i = 0; i < 20; i++) await null
|
||||
const end = Date.now() + 1500
|
||||
while (Date.now() < end) {}
|
||||
return 'raced'
|
||||
`),
|
||||
parent: fakeParent(),
|
||||
})
|
||||
await vi.waitFor(() => { expect(starts).toBe(1) })
|
||||
handle.cancel('stop now')
|
||||
await vi.waitFor(() => { expect(cancelled).toEqual(['stop now']) }, { timeout: 800 })
|
||||
// The wedged worker's own completion loses to the in-flight cancel.
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('dispose() on a wedged worker host-drives child disposal inside the grace: it returns with the children DISPOSED, not with their teardown still in flight', async () => {
|
||||
const { ctx, parent, provider } = await setup({
|
||||
manual: true,
|
||||
disposeDelayMs: 40,
|
||||
config: { provider: 'stub', maxConcurrentAgents: 8, disposeGraceMs: 400 },
|
||||
})
|
||||
const handle = ctx.workflows.start({
|
||||
// Same shape as the wedged-cancel test above: the child's start RPC
|
||||
// reaches the host, then the script seizes its worker's loop, so the
|
||||
// worker can relay NO dispose RPC — the host's own dispose() drive is
|
||||
// the only thing that can start (and finish) this child's disposal
|
||||
// before the grace runs out.
|
||||
...scripted(`
|
||||
agent('wedged child')
|
||||
for (let i = 0; i < 20; i++) await null
|
||||
const end = Date.now() + 1500
|
||||
while (Date.now() < end) {}
|
||||
return 'raced'
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(1) })
|
||||
const before = Date.now()
|
||||
await handle.dispose()
|
||||
// Bounded by the grace (plus the terminate), never by the 1.5s spin.
|
||||
expect(Date.now() - before).toBeLessThan(1200)
|
||||
// Not a waitFor: dispose() resolving IS the quiescence claim — the slow
|
||||
// child disposal must be complete, not merely started (before the
|
||||
// host-driven drive, disposal only STARTED at the post-terminate reap,
|
||||
// so dispose() returned with it still in flight).
|
||||
expect(provider.runs[0]!.disposed).toBe(true)
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
}, 15_000)
|
||||
|
||||
it('a live child disposed by the dispose() drive is disposed ONCE, and the worker\'s late dispose RPC still gets its ack (the script settles, not the grace)', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted(`
|
||||
await agent('long child')
|
||||
return 'unreachable'
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(1) })
|
||||
const handleDispose = handle.dispose()
|
||||
const result = await handle.result
|
||||
// The script itself settled (the wrapper's own dispose RPC found the
|
||||
// child already reaped host-side and was acked) — a missing ack would
|
||||
// wedge the wrapper's finally until the 5s default grace force-settle.
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('workflow disposed')
|
||||
await handleDispose
|
||||
expect(provider.runs[0]!.disposed).toBe(true)
|
||||
// The memo: the host drive and the worker's RPC share one disposal.
|
||||
expect(provider.runs[0]!.disposeCalls).toBe(1)
|
||||
})
|
||||
|
||||
it('the grace force-settle pairs every stranded start: a host-synthesized cancelled agent-end lands before workflow/end', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true, config: { provider: 'stub', maxConcurrentAgents: 8, disposeGraceMs: 300 } })
|
||||
const ends: { seq: number; outcome: string }[] = []
|
||||
const order: string[] = []
|
||||
ctx.on('workflow/agent-start', (_info, agent) => { order.push(`start:${agent.seq}`) })
|
||||
ctx.on('workflow/agent-end', (_info, agent) => {
|
||||
ends.push({ seq: agent.seq, outcome: agent.outcome })
|
||||
order.push(`end:${agent.seq}`)
|
||||
})
|
||||
ctx.on('workflow/end', () => { order.push('run-end') })
|
||||
const handle = ctx.workflows.start({
|
||||
// 'slow' starts and its agent-start crosses to observers (the awaited
|
||||
// 'fast' call keeps the worker loop turning), then the script seizes
|
||||
// the loop: the wedged worker can never author slow's agent-end —
|
||||
// only the host's ledger can close the pair.
|
||||
...scripted(`
|
||||
const p = agent('slow')
|
||||
await agent('fast')
|
||||
const end = Date.now() + 1500
|
||||
while (Date.now() < end) {}
|
||||
return 'raced'
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(order.filter(entry => entry.startsWith('start:')).length).toBe(2) })
|
||||
const fast = provider.runs.find(run => (run.request.prompt[0] as { text?: string }).text === 'fast')!
|
||||
fast.settle(text('fast done'))
|
||||
handle.cancel('stop now')
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
// fast's end is the worker's own report; slow's is host-synthesized at
|
||||
// the force-settle — exactly one end per started seq, no third event.
|
||||
expect(ends).toEqual([
|
||||
{ seq: 2, outcome: 'completed' },
|
||||
{ seq: 1, outcome: 'cancelled' },
|
||||
])
|
||||
// Both ends reached observers BEFORE workflow/end: a progress consumer
|
||||
// can finalize its state at run-end without dangling agents.
|
||||
expect(order.indexOf('run-end')).toBe(order.length - 1)
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('graceful cancellation keeps pairing worker-authored: exactly one agent-end per start, nothing synthesized on top', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const ends: { seq: number; outcome: string }[] = []
|
||||
const order: string[] = []
|
||||
ctx.on('workflow/agent-end', (_info, agent) => {
|
||||
ends.push({ seq: agent.seq, outcome: agent.outcome })
|
||||
order.push(`end:${agent.seq}`)
|
||||
})
|
||||
ctx.on('workflow/end', () => { order.push('run-end') })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted("await parallel([() => agent('a'), () => agent('b')])\nreturn 'unreachable'"),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(provider.runs.length).toBe(2) })
|
||||
handle.cancel('user stop')
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
// The live worker reported both pairs itself; the ledger must not add
|
||||
// a synthesized duplicate on any path that settles inside the grace.
|
||||
expect(ends.map(end => end.outcome)).toEqual(['cancelled', 'cancelled'])
|
||||
expect(new Set(ends.map(end => end.seq)).size).toBe(2)
|
||||
expect(order.indexOf('run-end')).toBe(order.length - 1)
|
||||
await handle.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
describe('worker death', () => {
|
||||
it('a worker that exits before settling reports an error result and reaps its children', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
// The child's dispose() REJECTS on top of the worker death: the reap
|
||||
// must contain it (warn, not crash) while still emptying the registry.
|
||||
const cancelled: string[] = []
|
||||
const provider: SubagentProvider = {
|
||||
name: 'doomed',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => ({
|
||||
id: AgentId('doomed-child'),
|
||||
result: new Promise(() => { /* never settles; the reap is the teardown */ }),
|
||||
cancel: (reason?: string) => { cancelled.push(reason ?? 'cancelled') },
|
||||
dispose: () => Promise.reject(new Error('dispose exploded during reap')),
|
||||
}),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'doomed', maxConcurrentAgents: 2 })
|
||||
const runEnds: WorkflowResultInfo[] = []
|
||||
ctx.on('workflow/end', (_info, result) => { runEnds.push(result) })
|
||||
const handle = ctx.workflows.start({
|
||||
// The stray child's start RPC reaches the host, then the script kills
|
||||
// its own worker through the documented vm escape — the host must
|
||||
// settle `error` with the exit diagnostics and wind the child down.
|
||||
...scripted(`
|
||||
agent('doomed')
|
||||
const proc = ${ESCAPE}
|
||||
const st = globalThis.constructor.constructor('return setTimeout')()
|
||||
await new Promise(resolve => st(resolve, 200))
|
||||
proc.exit(7)
|
||||
`),
|
||||
parent: fakeParent(),
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('exit code 7')
|
||||
expect(result.agentsStarted).toBe(1)
|
||||
// A worker death is a stop reason like any other: workflow/end fires
|
||||
// with the error outcome — for a bus observer it is the only obituary.
|
||||
expect(runEnds).toEqual([{ stopReason: 'error', error: result.error, agentsStarted: 1 }])
|
||||
await vi.waitFor(() => { expect(cancelled.length).toBe(1) })
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('an uncaught exception inside the worker surfaces as an error result and reaps the in-flight child', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted(`
|
||||
agent('in flight when the worker dies')
|
||||
const proc = ${ESCAPE}
|
||||
const st = globalThis.constructor.constructor('return setTimeout')()
|
||||
await new Promise(resolve => st(resolve, 200))
|
||||
proc.nextTick(() => { throw new Error('worker blew up') })
|
||||
await new Promise(() => {})
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('worker blew up')
|
||||
// The reap wound the stray child down (cancel + a CLEAN dispose).
|
||||
await vi.waitFor(() => {
|
||||
expect(provider.runs.length).toBe(1)
|
||||
expect(provider.runs[0]!.disposed).toBe(true)
|
||||
})
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('a worker death pairs every stranded start: the synthesized cancelled agent-end precedes the error workflow/end', async () => {
|
||||
const { ctx, parent, provider } = await setup({ manual: true })
|
||||
const ends: { seq: number; outcome: string }[] = []
|
||||
const order: string[] = []
|
||||
ctx.on('workflow/agent-start', (_info, agent) => { order.push(`start:${agent.seq}`) })
|
||||
ctx.on('workflow/agent-end', (_info, agent) => {
|
||||
ends.push({ seq: agent.seq, outcome: agent.outcome })
|
||||
order.push(`end:${agent.seq}`)
|
||||
})
|
||||
ctx.on('workflow/end', () => { order.push('run-end') })
|
||||
const handle = ctx.workflows.start({
|
||||
// Same choreography as the force-settle pairing test, but the worker
|
||||
// DIES (the documented vm escape) instead of being terminated: the
|
||||
// exit path must close slow's pair from the ledger too. The escaped
|
||||
// setTimeout lets the already-posted messages flush before the kill.
|
||||
...scripted(`
|
||||
const p = agent('slow')
|
||||
await agent('fast')
|
||||
const proc = ${ESCAPE}
|
||||
const st = globalThis.constructor.constructor('return setTimeout')()
|
||||
await new Promise(resolve => st(resolve, 150))
|
||||
proc.exit(7)
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
await vi.waitFor(() => { expect(order.filter(entry => entry.startsWith('start:')).length).toBe(2) })
|
||||
const fast = provider.runs.find(run => (run.request.prompt[0] as { text?: string }).text === 'fast')!
|
||||
fast.settle(text('fast done'))
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('exit code 7')
|
||||
expect(ends).toEqual([
|
||||
{ seq: 2, outcome: 'completed' },
|
||||
{ seq: 1, outcome: 'cancelled' },
|
||||
])
|
||||
expect(order.indexOf('run-end')).toBe(order.length - 1)
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('a dispose ack racing the worker death is dropped, not crashed (post after exit)', async () => {
|
||||
// Slow child disposal: the ack resolves only AFTER the worker died, so
|
||||
// it has nowhere to go and must be dropped silently (the workerGone
|
||||
// guard in post()).
|
||||
const { ctx, parent, provider } = await setup({ disposeDelayMs: 300 })
|
||||
const handle = ctx.workflows.start({
|
||||
// The STRAY child settles instantly, so its wrapper starts the slow
|
||||
// host-side disposal concurrently while the script goes on to kill
|
||||
// its own worker — the ack then resolves into a dead thread.
|
||||
...scripted(`
|
||||
agent('stray, never awaited')
|
||||
const proc = ${ESCAPE}
|
||||
const st = globalThis.constructor.constructor('return setTimeout')()
|
||||
await new Promise(resolve => st(resolve, 150))
|
||||
proc.exit(5)
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('exit code 5')
|
||||
await vi.waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) })
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
|
||||
it('a worker death AFTER a cancel reports cancelled, not error', async () => {
|
||||
const { ctx, parent } = await setup({ config: { provider: 'stub', disposeGraceMs: 60_000 } })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted(`
|
||||
const proc = ${ESCAPE}
|
||||
const st = globalThis.constructor.constructor('return setTimeout')()
|
||||
log('armed')
|
||||
await new Promise(resolve => st(resolve, 400))
|
||||
proc.exit(3)
|
||||
`),
|
||||
parent,
|
||||
})
|
||||
const logs: string[] = []
|
||||
ctx.on('workflow/log', (_info, message) => { logs.push(message) })
|
||||
await vi.waitFor(() => { expect(logs).toContain('armed') })
|
||||
handle.cancel('stop it')
|
||||
// The grace is deliberately huge: only the worker's own death (exit 3,
|
||||
// unreachable by the cancel — the script ignores hooks) settles this.
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('stop it')
|
||||
await handle.dispose()
|
||||
}, 15_000)
|
||||
})
|
||||
|
||||
describe('service surface', () => {
|
||||
it('run ids are unique per start; the run handle and event payloads hold SEPARATE meta clones', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
let eventMeta: WorkflowRunInfo | undefined
|
||||
ctx.on('workflow/start', (info) => { eventMeta = info })
|
||||
const first = ctx.workflows.start({ ...scripted('return 1'), parent })
|
||||
const second = ctx.workflows.start({ ...scripted('return 2'), parent })
|
||||
expect(first.id).not.toBe(second.id)
|
||||
eventMeta!.meta.name = 'corrupted'
|
||||
expect(second.meta.name).toBe('test-flow')
|
||||
await Promise.all([first.result, second.result])
|
||||
await first.dispose()
|
||||
await second.dispose()
|
||||
})
|
||||
|
||||
it('unregisters ctx.workflows when the engine fiber is disposed (HMR safety), and default config runs (auto concurrency)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
const fiber = await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
expect(ctx.get('workflows')).toBeDefined()
|
||||
// A zero-agent run through the DEFAULT config exercises the auto
|
||||
// concurrency resolution (cores - 2, capped) in start().
|
||||
const result = await run(ctx, fakeParent(), scripted('return 6 * 7'))
|
||||
expect(result.value).toBe(42)
|
||||
await fiber.dispose()
|
||||
expect(ctx.get('workflows')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('has the class-plugin export shape (default = the engine service class)', () => {
|
||||
expect(workerEngineModule.default).toBe(WorkerWorkflowEngine)
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped: unknown = loader.unwrapExports(workerEngineModule)
|
||||
expect(unwrapped).toBe(WorkerWorkflowEngine)
|
||||
})
|
||||
})
|
||||
})
|
||||
39
packages/workflow/workflow-workerthread/tsconfig.json
Normal file
39
packages/workflow/workflow-workerthread/tsconfig.json
Normal file
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../util/brand"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../subagent/subagent"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
},
|
||||
{
|
||||
"path": "../workflow"
|
||||
}
|
||||
]
|
||||
}
|
||||
32
packages/workflow/workflow-workerthread/tsdown.config.ts
Normal file
32
packages/workflow/workflow-workerthread/tsdown.config.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/**
|
||||
* The engine ships two runtime entries: the engine service (index) and the
|
||||
* worker-thread entry (worker) the engine spawns via `new Worker`. The
|
||||
* entries are JS emitted by tsc under lib/types and are bundled as two
|
||||
* single-entry passes so shared modules (realm, runtime, session) are inlined
|
||||
* into each instead of split into a hash-named chunk (the worker entry must
|
||||
* be a self-contained file the Worker constructor can load by path).
|
||||
*/
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
29
packages/workflow/workflow/README.md
Normal file
29
packages/workflow/workflow/README.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# @deepseek-ai/dsh-workflow
|
||||
|
||||
The **workflow seam** (`ctx.workflows`): an abstract service defining WHAT a workflow engine does — execute a model-written orchestration script that fans out subagents — without saying HOW. The bash-shaped third of the [workflow family](../README.md): implementations subclass `WorkflowService` and register as the `workflows` service (one per context); [`dsh-workflow-workerthread`](../workflow-workerthread/README.md) (one worker thread per run) is the implementation, and [`dsh-tool-workflow`](../tool-workflow/README.md) is the model-facing consumer.
|
||||
|
||||
## Service: `WorkflowService` (abstract)
|
||||
|
||||
`start(request: WorkflowStartRequest): WorkflowRun` — parse and execute a script. Throws synchronously (`SCRIPT_PARSE`/`META_INVALID`) for a script that cannot begin; once a run is returned, its `result` NEVER rejects — every failure resolves with `stopReason: 'error'` (or `'cancelled'`) — and once the run is cancelled, `result` settles within the implementation's bounded grace even if the script itself never settles (a consumer awaiting `result` must never be wedged past a cancellation). `dispose()` must reach quiescence within a bounded grace (cancel → wait for the script to settle and its children to finish disposing → abandon), never hanging its caller. Runs are HOLDER-owned: the engine does not track its live runs, so disposing the engine's fiber mid-run leaves each run to its holder's teardown.
|
||||
|
||||
The protected `emitWorkflowEvent` helper dispatches the `workflow/*` events with PER-LISTENER containment and PER-LISTENER payload snapshots (a throwing subscriber is logged, never propagated, and cannot starve later listeners; each subscriber gets its own clone of the payload, so mutating it corrupts neither the engine nor other listeners) — the same containment guarantee as the subagent seam's lifecycle emits.
|
||||
|
||||
## Vocabulary
|
||||
|
||||
- `WorkflowStartRequest` — `{ script, args?, parent: Agent, signal? }`. `parent` is REQUIRED: every child the script spawns is attributed to it. `args` must be plain host-realm JSON data.
|
||||
- `WorkflowMeta` / `WorkflowPhase` — the workflow's identity block, carried as plain JSON data on the start request (Claude Code meta vocabulary: required `name`/`description`, optional `whenToUse`/`phases`) and shape-validated by the engine.
|
||||
- `WorkflowRun` — `{ id, meta, result, cancel(reason?), dispose() }`; the consumer awaits `result` and MUST `dispose` on every path.
|
||||
- `WorkflowResult` — `{ value, stopReason: 'completed'|'cancelled'|'error', error?, agentsStarted }`; `value` is the script's materialized return (plain JSON data; `null` for no return).
|
||||
- `WorkflowError` — `HarnessError` with a `WorkflowErrorCode` and a `fatal` flag driving the combinator discipline: a fatal error (bad hook arguments, unsupported options/schemas, tripped caps, seam start failures, cancellation) always propagates through `parallel()`/`pipeline()` instead of dissolving into a per-item `null`. `isFatalWorkflowError(error)` is the catch-site predicate.
|
||||
|
||||
## Events
|
||||
|
||||
All observe-only emits carrying DATA SNAPSHOTS (`WorkflowRunInfo` = id + meta) — never the live `WorkflowRun`, so a listener cannot gain `cancel`/`dispose`; control stays with the `start()` caller:
|
||||
|
||||
- `workflow/start`(info) / `workflow/end`(info, resultInfo) — run lifecycle; `resultInfo` deliberately omits the value.
|
||||
- `workflow/phase`(info, title) / `workflow/log`(info, message) — script narration.
|
||||
- `workflow/agent-start`(info, agent) / `workflow/agent-end`(info, agent + outcome) — one pair per `agent()` call that STARTED a child run (a call rejected at validation or caps, refused at start, or cancelled while queued for a slot emits no pair), correlated by `seq`.
|
||||
|
||||
## Non-goals (this cut)
|
||||
|
||||
Background collection, journaling/resume, saved workflows, nested `workflow()`, token budgets — see the [RFC's deferred section](../../../docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md).
|
||||
37
packages/workflow/workflow/package.json
Normal file
37
packages/workflow/workflow/package.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-workflow",
|
||||
"description": "Workflow capability seam: ctx.workflows service, run vocabulary, and workflow/* events",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-brand": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.6"
|
||||
}
|
||||
}
|
||||
269
packages/workflow/workflow/src/index.ts
Normal file
269
packages/workflow/workflow/src/index.ts
Normal file
@@ -0,0 +1,269 @@
|
||||
/**
|
||||
* The workflow capability seam (`ctx.workflows`): an abstract service defining
|
||||
* WHAT a workflow engine does — execute a model-written orchestration script
|
||||
* that fans out subagents — without saying HOW. Implementations subclass
|
||||
* {@link WorkflowService} and register as the `workflows` service (one
|
||||
* implementation per context, cordis' standard duplicate-service behavior);
|
||||
* the implementation is `@deepseek-ai/dsh-workflow-workerthread`, which runs each
|
||||
* script in its own worker thread. Hardened engines (an isolated-vm or
|
||||
* separate-process sandbox) swap in without touching the model-facing tool
|
||||
* that consumes them (`@deepseek-ai/dsh-tool-workflow`).
|
||||
*
|
||||
* The `workflow/*` lifecycle events are OBSERVE-ONLY data snapshots: they
|
||||
* carry {@link WorkflowRunInfo} (id + meta), never the live {@link WorkflowRun}
|
||||
* — a listener must not gain `cancel`/`dispose`; control stays with the
|
||||
* `start()` caller holding the run. Every emit is per-listener contained (a
|
||||
* throwing subscriber is logged, never propagated) and every listener gets its
|
||||
* own payload clone (mutating it corrupts nothing), so one bad observer can
|
||||
* neither strand a live run, starve later listeners, nor poison another
|
||||
* listener's view.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow
|
||||
*/
|
||||
|
||||
import { Context, Service } from 'cordis'
|
||||
import { HarnessError } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
WorkflowAgentEndInfo,
|
||||
WorkflowAgentInfo,
|
||||
WorkflowResultInfo,
|
||||
WorkflowRun,
|
||||
WorkflowRunInfo,
|
||||
WorkflowStartRequest,
|
||||
} from './types.ts'
|
||||
|
||||
export { WorkflowRunId } from './types.ts'
|
||||
export type {
|
||||
WorkflowAgentEndInfo,
|
||||
WorkflowAgentInfo,
|
||||
WorkflowAgentOutcome,
|
||||
WorkflowMeta,
|
||||
WorkflowPhase,
|
||||
WorkflowResult,
|
||||
WorkflowResultInfo,
|
||||
WorkflowRun,
|
||||
WorkflowRunInfo,
|
||||
WorkflowStartRequest,
|
||||
WorkflowStopReason,
|
||||
} from './types.ts'
|
||||
|
||||
declare module 'cordis' {
|
||||
interface Context {
|
||||
workflows: WorkflowService
|
||||
}
|
||||
|
||||
interface Events {
|
||||
/**
|
||||
* A workflow run started — the script's meta block validated, the body
|
||||
* about to execute. Paired with {@link Events['workflow/end']}.
|
||||
* @param info - the run's identity snapshot (id + meta).
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/start'(info: WorkflowRunInfo): void
|
||||
/**
|
||||
* The script entered a phase (a `phase(title)` call) — progress grouping
|
||||
* for observers; no execution semantics.
|
||||
* @param info - the run's identity snapshot.
|
||||
* @param title - the phase title, verbatim.
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/phase'(info: WorkflowRunInfo, title: string): void
|
||||
/**
|
||||
* The script emitted a narration line (a `log(message)` call).
|
||||
* @param info - the run's identity snapshot.
|
||||
* @param message - the logged message, verbatim.
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/log'(info: WorkflowRunInfo, message: string): void
|
||||
/**
|
||||
* One `agent()` call started a child run. Paired with
|
||||
* {@link Events['workflow/agent-end']} by `agent.seq`.
|
||||
* @param info - the run's identity snapshot.
|
||||
* @param agent - the call's sequence number, label, phase, and child id.
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/agent-start'(info: WorkflowRunInfo, agent: WorkflowAgentInfo): void
|
||||
/**
|
||||
* One `agent()` call settled (clean result, child failure, or run
|
||||
* cancellation). Paired with {@link Events['workflow/agent-start']} by
|
||||
* `agent.seq`, exactly once per started call on every stop path — on an
|
||||
* engine termination path (a worker killed past its grace) the end is
|
||||
* engine-synthesized with outcome `'cancelled'`.
|
||||
* @param info - the run's identity snapshot.
|
||||
* @param agent - the call identity plus its outcome.
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/agent-end'(info: WorkflowRunInfo, agent: WorkflowAgentEndInfo): void
|
||||
/**
|
||||
* A workflow run settled (any stop reason). Fired when
|
||||
* {@link WorkflowRun.result} resolves. Paired with
|
||||
* {@link Events['workflow/start']}.
|
||||
* @param info - the run's identity snapshot.
|
||||
* @param result - the outcome data (stop reason, error, agent count) —
|
||||
* deliberately WITHOUT the result value (see {@link WorkflowResultInfo}).
|
||||
* @mode emit
|
||||
*/
|
||||
'workflow/end'(info: WorkflowRunInfo, result: WorkflowResultInfo): void
|
||||
}
|
||||
}
|
||||
|
||||
/** The full set of `workflow/*` event names {@link WorkflowService.emitWorkflowEvent} dispatches. */
|
||||
export type WorkflowEventName =
|
||||
| 'workflow/start'
|
||||
| 'workflow/phase'
|
||||
| 'workflow/log'
|
||||
| 'workflow/agent-start'
|
||||
| 'workflow/agent-end'
|
||||
| 'workflow/end'
|
||||
|
||||
/**
|
||||
* The workflow-seam error codes. Every one of these is FATAL when it reaches
|
||||
* a script (see {@link WorkflowError.fatal}): the combinators re-throw it
|
||||
* instead of dissolving it into an ordinary per-item `null`.
|
||||
*
|
||||
* - `SCRIPT_PARSE` — the script (or its meta statement) does not parse.
|
||||
* - `META_INVALID` — the meta block evaluated but fails the shape contract.
|
||||
* - `INVALID_ARGUMENT` — a hook was called with malformed arguments.
|
||||
* - `UNSUPPORTED_OPTION` — an `agent()` option this engine does not support
|
||||
* (deferred: `effort`/`isolation`/`agentType`) or does not know.
|
||||
* - `UNSUPPORTED_SCHEMA` — an `agent()` schema outside the structured-output
|
||||
* subset (see dsh-tools).
|
||||
* - `AGENT_CAP` / `ITEM_CAP` — the run/agent caps tripped.
|
||||
* - `AGENT_START` — the subagent seam refused to start a child.
|
||||
* - `AGENT_RESULT` — a child's `result` REJECTED: an infrastructure fault at
|
||||
* the subagent seam, distinct from a child that failed and resolved (which
|
||||
* is the per-item `null`, never an error).
|
||||
* - `RESULT_UNSERIALIZABLE` — a value crossing the script/host value boundary
|
||||
* is not plain JSON data.
|
||||
* - `CANCELLED` — the run was cancelled; pending and future hooks reject
|
||||
* with this (the script-kill mechanism).
|
||||
*/
|
||||
export type WorkflowErrorCode =
|
||||
| 'SCRIPT_PARSE'
|
||||
| 'META_INVALID'
|
||||
| 'INVALID_ARGUMENT'
|
||||
| 'UNSUPPORTED_OPTION'
|
||||
| 'UNSUPPORTED_SCHEMA'
|
||||
| 'AGENT_CAP'
|
||||
| 'ITEM_CAP'
|
||||
| 'AGENT_START'
|
||||
| 'AGENT_RESULT'
|
||||
| 'RESULT_UNSERIALIZABLE'
|
||||
| 'CANCELLED'
|
||||
|
||||
/**
|
||||
* Typed error for workflow-seam failures. Extends {@link HarnessError}, so the
|
||||
* `code` is machine-routable taxonomy. `fatal` drives the combinator
|
||||
* discipline: `parallel()`/`pipeline()` re-throw a fatal error (a typo'd
|
||||
* option or a tripped cap must kill the script loudly), and reserve the
|
||||
* per-item `null` for child-run failures and ordinary in-stage script errors.
|
||||
* Every {@link WorkflowErrorCode} is fatal in this cut; the flag exists so the
|
||||
* distinction is explicit at every catch site rather than implied.
|
||||
*/
|
||||
export class WorkflowError extends HarnessError {
|
||||
/** Whether combinators must propagate this error instead of nulling the item. */
|
||||
readonly fatal: boolean
|
||||
|
||||
constructor(message: string, code: WorkflowErrorCode, options?: ErrorOptions & { fatal?: boolean }) {
|
||||
super(message, code, options)
|
||||
this.name = 'WorkflowError'
|
||||
this.fatal = options?.fatal ?? true
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether combinators must re-throw `error` instead of mapping the item to `null`.
|
||||
* @param error - any thrown value; fatality is host `instanceof` (unforgeable from a script realm).
|
||||
* @returns true iff `error` is a {@link WorkflowError} whose `fatal` flag is set.
|
||||
*/
|
||||
export function isFatalWorkflowError(error: unknown): boolean {
|
||||
return error instanceof WorkflowError && error.fatal
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract workflow execution service. Subclass, implement {@link start}, and
|
||||
* load the subclass as a plugin — it registers as `ctx.workflows` (one
|
||||
* implementation per context; loading a second throws, cordis' standard
|
||||
* duplicate-service behavior).
|
||||
*
|
||||
* Semantics every implementation must honor:
|
||||
* - {@link start} throws synchronously for a request that cannot begin (an
|
||||
* unparseable script, an invalid meta block). Once it returns a
|
||||
* {@link WorkflowRun}, `result` NEVER rejects — every failure resolves with
|
||||
* `stopReason: 'error'` (or `'cancelled'`) — and once the run is cancelled,
|
||||
* `result` SETTLES within the implementation's bounded grace even if the
|
||||
* script itself never settles (a consumer awaiting `result` must never be
|
||||
* wedged past a cancellation).
|
||||
* - The `workflow/*` events fire through {@link emitWorkflowEvent} (data
|
||||
* snapshots, per-listener containment); `workflow/end` fires exactly once
|
||||
* per started run, after `result` is settled or as it settles.
|
||||
* - `dispose()` reaches quiescence within a bounded grace: it cancels, waits
|
||||
* for the script to settle AND its started children to finish disposing,
|
||||
* and abandons whatever is left rather than hanging its caller (the engine
|
||||
* documents what abandonment leaves behind).
|
||||
* - Runs are HOLDER-OWNED: the engine hands control (`cancel`/`dispose`) to
|
||||
* the `start()` caller and does not track its live runs — disposing the
|
||||
* engine's own fiber mid-run deliberately leaves those runs to their
|
||||
* holders' teardown, so an engine reload cannot yank a run out from under
|
||||
* the consumer awaiting it.
|
||||
*/
|
||||
export abstract class WorkflowService extends Service {
|
||||
constructor(ctx: Context) {
|
||||
super(ctx, 'workflows')
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse and execute a workflow script.
|
||||
* @param request - the script, its `args`, the parent agent, and an
|
||||
* optional cancel signal.
|
||||
* @returns the live run; its `result` resolves when the script settles.
|
||||
*/
|
||||
abstract start(request: WorkflowStartRequest): WorkflowRun
|
||||
|
||||
/**
|
||||
* Emit one `workflow/*` lifecycle event with PER-LISTENER containment and
|
||||
* PER-LISTENER payload snapshots: each subscriber is dispatched individually
|
||||
* with its OWN structural clone of the payload (the payloads are plain JSON
|
||||
* data by the seam contract), so a listener mutating what it received can
|
||||
* corrupt neither the engine's live state nor any other listener's or later
|
||||
* event's view; a thrown listener is logged (never propagated — the logging
|
||||
* itself is total, even for a thrown value whose own string coercion
|
||||
* throws), so one bad subscriber can neither fail the engine mid-run,
|
||||
* surface as an unhandled rejection on a detached settle hook, nor starve
|
||||
* the listeners registered after it (cordis `emit` halts on the first throw
|
||||
* — same guarantee as the subagent seam's lifecycle emits).
|
||||
* @param name - the `workflow/*` event to dispatch.
|
||||
* @param args - the event's payload, matching its declared signature.
|
||||
*/
|
||||
protected emitWorkflowEvent(name: WorkflowEventName, ...args: unknown[]): void {
|
||||
for (const callback of this.ctx.events.dispatch('emit', [name, ...args])) {
|
||||
try {
|
||||
// The declared workflow/* signatures are all void-returning emits; the
|
||||
// dispatch callback applies the payload tuple.
|
||||
;(callback as (...payload: unknown[]) => void)(...structuredClone(args))
|
||||
} catch (error: unknown) {
|
||||
this.ctx.logger.warn(`workflow: ${name} listener threw: ${renderListenerError(error)}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Total renderer for a listener-thrown value: the containment catch must never
|
||||
* itself throw, and `String(error)` does when the value's own `toString` /
|
||||
* `Symbol.toPrimitive` throws. Local rather than an engine package's renderer
|
||||
* — the seam sits below every engine and cannot import one.
|
||||
* @param error - any thrown value.
|
||||
* @returns `String(error)`, or a fixed label when even coercion throws.
|
||||
*/
|
||||
function renderListenerError(error: unknown): string {
|
||||
try {
|
||||
return String(error)
|
||||
} catch {
|
||||
// Only a throwing toString/Symbol.toPrimitive lands here; the fixed label
|
||||
// keeps the containment guarantee total.
|
||||
return '[unrenderable thrown value]'
|
||||
}
|
||||
}
|
||||
|
||||
export default WorkflowService
|
||||
173
packages/workflow/workflow/src/types.ts
Normal file
173
packages/workflow/workflow/src/types.ts
Normal file
@@ -0,0 +1,173 @@
|
||||
/**
|
||||
* Workflow seam vocabulary: the request/run/result types a workflow engine
|
||||
* consumes and produces, plus the payload shapes of the `workflow/*` events.
|
||||
* Types only (plus the id-brand factory), per the package convention.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-workflow/types
|
||||
*/
|
||||
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
import type { Agent, AgentId } from '@deepseek-ai/dsh-agent'
|
||||
|
||||
/** Identifies one workflow run. */
|
||||
export type WorkflowRunId = Branded<'WorkflowRunId'>
|
||||
|
||||
/**
|
||||
* Brand a string as a {@link WorkflowRunId}.
|
||||
* @param id - the raw id string (the engine mints UUIDs; tests may pass fixtures).
|
||||
* @returns the same string, branded.
|
||||
*/
|
||||
export function WorkflowRunId(id: string): WorkflowRunId {
|
||||
return id as WorkflowRunId
|
||||
}
|
||||
|
||||
/**
|
||||
* One phase declared in a script's `meta.phases` (progress vocabulary only —
|
||||
* phases group agents in observers/UIs; they impose no execution structure).
|
||||
*/
|
||||
export interface WorkflowPhase {
|
||||
/** The phase title; `phase()` calls match against it by exact string. */
|
||||
title: string
|
||||
/** Optional one-line description of what the phase does. */
|
||||
detail?: string
|
||||
/** Optional model override this phase is expected to use (informational). */
|
||||
model?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The script's identity block, provided as plain JSON data alongside the
|
||||
* script body (the model-facing tool carries it as its `meta` parameter) and
|
||||
* validated by the engine before the body runs. `name`/`description` are
|
||||
* required; the rest is optional annotation. The field vocabulary matches the
|
||||
* Claude Code dynamic-workflows meta block.
|
||||
*/
|
||||
export interface WorkflowMeta {
|
||||
/** Short kebab-case workflow name (display + persistence key). */
|
||||
name: string
|
||||
/** One-line description of what the workflow does. */
|
||||
description: string
|
||||
/** Optional guidance on when this workflow applies (shown in listings). */
|
||||
whenToUse?: string
|
||||
/** Optional phase declarations matched by `phase()` calls. */
|
||||
phases?: WorkflowPhase[]
|
||||
}
|
||||
|
||||
/**
|
||||
* What a caller asks for when starting a workflow run. `meta` and `args` are
|
||||
* plain JSON DATA by the seam contract (the tool builds both from the model's
|
||||
* schema-validated call; the engine validates `meta`'s shape and rejects loud
|
||||
* before anything runs) — an engine never evaluates script text to obtain
|
||||
* them. `parent` is REQUIRED — every `agent()` the script spawns is
|
||||
* attributed to it (cwd, lineage, depth flow through the subagent seam).
|
||||
*/
|
||||
export interface WorkflowStartRequest {
|
||||
/** The plain-JS script body (top-level await allowed; ends with `return <json-value>`). */
|
||||
script: string
|
||||
/** The workflow's identity block, as plain JSON data (shape-validated by the engine). */
|
||||
meta: WorkflowMeta
|
||||
/** Optional input exposed verbatim to the script as the `args` global. */
|
||||
args?: unknown
|
||||
/** The agent on whose behalf the run executes (parent of every child). */
|
||||
parent: Agent
|
||||
/** Cancels the run when aborted (the tool's `exec.signal`). */
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/**
|
||||
* Why a run settled. CLOSED union (engine-owned, consumers may exhaust):
|
||||
* `completed` = the script ran to its final `return`; `cancelled` = the run
|
||||
* was cancelled (caller `cancel()`/signal); `error` = the script threw, a
|
||||
* fatal `WorkflowError` propagated, or the result failed materialization.
|
||||
*/
|
||||
export type WorkflowStopReason = 'completed' | 'cancelled' | 'error'
|
||||
|
||||
/**
|
||||
* The outcome of one run, resolved by {@link WorkflowRun.result}. `value` is
|
||||
* the script's materialized return value (plain host-realm JSON data; `null`
|
||||
* when the script returned `undefined`) — meaningful only for `completed`.
|
||||
* A non-`completed` reason carries the failure in `error`; the consumer maps
|
||||
* it to an `isError` tool result rather than reporting partial output.
|
||||
*/
|
||||
export interface WorkflowResult {
|
||||
/** The script's return value (host JSON data; `null` for no return). */
|
||||
value: unknown
|
||||
/** Why the run settled. */
|
||||
stopReason: WorkflowStopReason
|
||||
/** The failure message (present iff `stopReason` is not `completed`). */
|
||||
error?: string
|
||||
/**
|
||||
* How many `agent()` calls the run accepted over its whole lifetime. On a
|
||||
* graceful settlement this is the script-side count (calls still queued for
|
||||
* a concurrency slot included); on a termination path (grace force-settle,
|
||||
* worker death) it degrades to the host-observed count — calls queued
|
||||
* inside a terminated script are unknowable then.
|
||||
*/
|
||||
agentsStarted: number
|
||||
}
|
||||
|
||||
/**
|
||||
* The handle the consumer holds while a script executes. The consumer awaits
|
||||
* `result`, may `cancel` mid-flight, and MUST `dispose` on every path.
|
||||
* `result` does NOT reject — a script failure resolves with `stopReason:
|
||||
* 'error'` — and once the run is cancelled it SETTLES within the engine's
|
||||
* bounded grace even if the script itself never settles (the engine
|
||||
* force-settles `cancelled`; what becomes of the script is engine-documented
|
||||
* — the worker-thread engine terminates its worker), so a consumer awaiting
|
||||
* `result` is never wedged past a cancellation. `dispose()` = cancel + that
|
||||
* bounded settle + child quiescence; it never hangs on a stuck script and is
|
||||
* safe to call on every path (idempotent).
|
||||
*/
|
||||
export interface WorkflowRun {
|
||||
readonly id: WorkflowRunId
|
||||
/** The validated meta block (available before the body runs). */
|
||||
readonly meta: WorkflowMeta
|
||||
readonly result: Promise<WorkflowResult>
|
||||
/** Cancel the run: children abort, pending hooks reject, the script dies at its next await (or is force-settled at the grace). */
|
||||
cancel(reason?: string): void
|
||||
/** Cancel + bounded-grace settle; safe to call on every path (idempotent). */
|
||||
dispose(): Promise<void>
|
||||
}
|
||||
|
||||
/** Identifying detail for a run, carried by every `workflow/*` event (a data snapshot, never the live run). */
|
||||
export interface WorkflowRunInfo {
|
||||
/** The run's id. */
|
||||
id: WorkflowRunId
|
||||
/** The run's validated meta block. */
|
||||
meta: WorkflowMeta
|
||||
}
|
||||
|
||||
/** One `agent()` call's identity within a run (the `workflow/agent-start` payload). */
|
||||
export interface WorkflowAgentInfo {
|
||||
/** 1-based sequence number of this `agent()` call within the run. */
|
||||
seq: number
|
||||
/** The display label (the `label` option, or a prompt snippet). */
|
||||
label: string
|
||||
/** The phase this agent belongs to (the `phase` option, else the current `phase()` title). */
|
||||
phase?: string
|
||||
/** The child agent's id on the subagent seam. */
|
||||
childId: AgentId
|
||||
}
|
||||
|
||||
/** How one `agent()` call settled: clean result, child failure (script sees `null`), or run cancellation. */
|
||||
export type WorkflowAgentOutcome = 'completed' | 'failed' | 'cancelled'
|
||||
|
||||
/** One `agent()` call's settlement (the `workflow/agent-end` payload). */
|
||||
export interface WorkflowAgentEndInfo extends WorkflowAgentInfo {
|
||||
/** How the call settled. */
|
||||
outcome: WorkflowAgentOutcome
|
||||
}
|
||||
|
||||
/**
|
||||
* A settled run's outcome as event data (the `workflow/end` payload): the
|
||||
* {@link WorkflowResult} minus `value` (a listener observing outcomes must not
|
||||
* receive a mutable alias of the caller's result value; a consumer that needs
|
||||
* the value holds the run and awaits `result`).
|
||||
*/
|
||||
export interface WorkflowResultInfo {
|
||||
/** Why the run settled. */
|
||||
stopReason: WorkflowStopReason
|
||||
/** The failure message (present iff `stopReason` is not `completed`). */
|
||||
error?: string
|
||||
/** How many `agent()` calls the run accepted (see {@link WorkflowResult.agentsStarted}). */
|
||||
agentsStarted: number
|
||||
}
|
||||
124
packages/workflow/workflow/tests/workflow.spec.ts
Normal file
124
packages/workflow/workflow/tests/workflow.spec.ts
Normal file
@@ -0,0 +1,124 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import WorkflowServiceDefault, {
|
||||
isFatalWorkflowError,
|
||||
WorkflowError,
|
||||
WorkflowRunId,
|
||||
WorkflowService,
|
||||
} from '../src/index.ts'
|
||||
import type { WorkflowRun, WorkflowRunInfo, WorkflowStartRequest } from '../src/index.ts'
|
||||
|
||||
/** A minimal concrete subclass exposing the protected emit helper for tests. */
|
||||
class StubEngine extends WorkflowService {
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
void request
|
||||
throw new Error('not under test')
|
||||
}
|
||||
|
||||
emit(name: Parameters<WorkflowService['emitWorkflowEvent']>[0], ...args: unknown[]): void {
|
||||
this.emitWorkflowEvent(name, ...args)
|
||||
}
|
||||
}
|
||||
|
||||
const INFO: WorkflowRunInfo = { id: WorkflowRunId('run-1'), meta: { name: 'w', description: 'd' } }
|
||||
|
||||
describe('dsh-workflow (interface)', () => {
|
||||
it('WorkflowRunId brands a string (identity at runtime)', () => {
|
||||
expect(WorkflowRunId('abc')).toBe('abc')
|
||||
})
|
||||
|
||||
it('WorkflowError carries code + fatal (default true) and reads as a HarnessError', () => {
|
||||
const error = new WorkflowError('cap hit', 'AGENT_CAP')
|
||||
expect(error.code).toBe('AGENT_CAP')
|
||||
expect(error.fatal).toBe(true)
|
||||
expect(error.name).toBe('WorkflowError')
|
||||
const soft = new WorkflowError('advisory', 'ITEM_CAP', { fatal: false })
|
||||
expect(soft.fatal).toBe(false)
|
||||
})
|
||||
|
||||
it('isFatalWorkflowError: true only for a fatal WorkflowError', () => {
|
||||
expect(isFatalWorkflowError(new WorkflowError('x', 'CANCELLED'))).toBe(true)
|
||||
expect(isFatalWorkflowError(new WorkflowError('x', 'CANCELLED', { fatal: false }))).toBe(false)
|
||||
expect(isFatalWorkflowError(new Error('plain'))).toBe(false)
|
||||
expect(isFatalWorkflowError('string')).toBe(false)
|
||||
})
|
||||
|
||||
it('registers as ctx.workflows and unregisters when its fiber is disposed (HMR safety)', async () => {
|
||||
const ctx = new Context()
|
||||
const fiber = await ctx.plugin(StubEngine)
|
||||
expect(ctx.get('workflows')).toBeInstanceOf(StubEngine)
|
||||
await fiber.dispose()
|
||||
expect(ctx.get('workflows')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('emitWorkflowEvent dispatches to every listener with the payload tuple', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(StubEngine)
|
||||
const seen: unknown[][] = []
|
||||
ctx.on('workflow/log', (info, message) => { seen.push([info, message]) })
|
||||
ctx.on('workflow/agent-start', (info, agent) => { seen.push([info, agent]) })
|
||||
const engine = ctx.workflows as StubEngine
|
||||
engine.emit('workflow/log', INFO, 'hello')
|
||||
engine.emit('workflow/agent-start', INFO, { seq: 1, label: 'l', childId: 'c' })
|
||||
expect(seen).toEqual([
|
||||
[INFO, 'hello'],
|
||||
[INFO, { seq: 1, label: 'l', childId: 'c' }],
|
||||
])
|
||||
})
|
||||
|
||||
it('gives each listener its OWN payload snapshot: mutation corrupts neither peers nor the caller', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(StubEngine)
|
||||
const seen: string[] = []
|
||||
ctx.on('workflow/agent-start', (info, agent) => {
|
||||
agent.label = 'HACKED'
|
||||
info.meta.name = 'HACKED'
|
||||
seen.push('mutator')
|
||||
})
|
||||
ctx.on('workflow/agent-start', (info, agent) => {
|
||||
seen.push(`${info.meta.name}/${agent.label}`)
|
||||
})
|
||||
const engine = ctx.workflows as StubEngine
|
||||
const info: WorkflowRunInfo = { id: WorkflowRunId('run-2'), meta: { name: 'w', description: 'd' } }
|
||||
const payload = { seq: 1, label: 'original', childId: 'c' }
|
||||
engine.emit('workflow/agent-start', info, payload)
|
||||
expect(seen).toEqual(['mutator', 'w/original'])
|
||||
// The caller's own objects are pristine too — no listener ever saw them.
|
||||
expect(info.meta.name).toBe('w')
|
||||
expect(payload.label).toBe('original')
|
||||
})
|
||||
|
||||
it('contains a throwing listener PER LISTENER: later listeners still run, nothing propagates', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(StubEngine)
|
||||
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => ctx.logger)
|
||||
const reached: string[] = []
|
||||
ctx.on('workflow/phase', () => { throw new Error('bad listener') })
|
||||
ctx.on('workflow/phase', (_info, title) => { reached.push(title) })
|
||||
const engine = ctx.workflows as StubEngine
|
||||
expect(() => { engine.emit('workflow/phase', INFO, 'Scan') }).not.toThrow()
|
||||
expect(reached).toEqual(['Scan'])
|
||||
expect(warn).toHaveBeenCalledOnce()
|
||||
expect(String(warn.mock.calls[0]![0])).toContain('workflow/phase listener threw')
|
||||
})
|
||||
|
||||
it('containment is total: a listener throwing a value whose coercion throws neither propagates nor starves later listeners', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(StubEngine)
|
||||
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => ctx.logger)
|
||||
const reached: string[] = []
|
||||
ctx.on('workflow/phase', () => {
|
||||
throw { toString: () => { throw new Error('coercion trap') } }
|
||||
})
|
||||
ctx.on('workflow/phase', (_info, title) => { reached.push(title) })
|
||||
const engine = ctx.workflows as StubEngine
|
||||
expect(() => { engine.emit('workflow/phase', INFO, 'Scan') }).not.toThrow()
|
||||
expect(reached).toEqual(['Scan'])
|
||||
expect(warn).toHaveBeenCalledOnce()
|
||||
expect(String(warn.mock.calls[0]![0])).toContain('[unrenderable thrown value]')
|
||||
})
|
||||
|
||||
it('has the expected export surface (default = the abstract service class)', () => {
|
||||
expect(WorkflowServiceDefault).toBe(WorkflowService)
|
||||
})
|
||||
})
|
||||
27
packages/workflow/workflow/tsconfig.json
Normal file
27
packages/workflow/workflow/tsconfig.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../util/brand"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user