Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge PR #224 updates into prose cleanup

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-12 23:36:49 +08:00
parent 75838e10b5 2ca806a1ec
commit 74ada5777c
165 changed files with 11693 additions and 6395 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
packages/workflow/workflow-workerthread/README.md
View File

@@ -1,53 +1,80 @@
# @deepseek-ai/dsh-workflow-workerthread
The [`WorkflowService`](../workflow/README.md) implementation, on **`node:worker_threads`**: each run gets its OWN worker thread (one run = one worker, no pooling — a run is heavyweight, so the ~tens-of-ms thread spin-up is noise), the script executes in a vm context INSIDE that worker with the workflow hooks injected, and every `agent()` call bridges back over the message port to [`ctx.subagents`](../../subagent/README.md) on the host. Child agents are I/O-bound LLM loops and stay on the host event loop; the thread isolates the SCRIPT, the only part that can spin synchronously.
This package implements `WorkflowService` with one Node worker thread per run. The worker executes the orchestration script; child agents remain on the host and are reached through `ctx.subagents` over a typed host/worker protocol.
## Trust premise: what the thread buys (and what it does not)
The split has one primary purpose: a synchronous script loop cannot block the harness event loop, and a script that ignores cancellation can be terminated with its worker. It is not a security sandbox.
Workflow scripts are **model-written** — the same trust level as the model's existing bash access — so this engine defends against **buggy** scripts, never hostile ones. A worker thread is NOT a security boundary: the vm context inside it is escapable by construction (`node:vm` shares object machinery with its surrounding realm, so a script can reach the `Function` constructor via `globalThis.constructor.constructor` and from it `process` and every Node builtin), and an escapee holds the same process privileges as the host — Node's permission model is process-wide. The absent globals are API surface that keeps honest scripts portable, not walls. What the thread concretely buys:
## Trust and isolation boundary
- **The host never blocks**: `start()` returns without running any script code on the host; a synchronous spin anywhere in the script occupies the worker's loop, not the harness's.
- **Termination is real**: a script that outlives its post-cancel grace is `worker.terminate()`d — nothing of it survives `dispose()`, where an in-process engine could only abandon the spin on its own loop.
- **No ambient credentials**: the worker spawns with an EMPTY environment (`env: {}` plus hermetic `execArgv`, the same stance as `dsh-code-runtime-worker`; the unbuilt dev shape forwards exactly one loader variable, `TSX_TSCONFIG_PATH` — path plumbing, not a secret), so an escapee reading `process.env` finds no harness secrets — ambient-channel hardening only; the process-wide privileges above (fs and the rest) remain, so a genuine sandbox is still the engine swap.
- **Serialization by construction**: everything crossing the thread is structured-clone data, and plain JSON before that — the `materializeFromRealm` walk rejects loud what JSON cannot carry, which is also what makes every postMessage hop total.
Workflow scripts are model-written and have the same trust premise as the model's existing bash access. `node:vm` inside a worker is an API-shaping mechanism, not a security boundary: an escaped script can recover Node capabilities with the host process's privileges.
What the seam guarantees regardless, because benign scripts hit these constantly: `result` never rejects, a dropped hook promise never becomes an unhandled rejection, values JSON cannot carry are rejected **loud** instead of silently mangled, and hook misuse is fatal instead of dissolving into a per-item `null`. Genuine sandboxing (containing what an escaped script may touch) remains an isolated-vm/separate-process engine swap behind the seam, still deferred.
The worker still provides useful containment:
## The script contract it executes
- Script CPU work and synchronous spins stay off the host event loop.
- `worker.terminate()` gives disposal a real final stop.
- The worker starts with an empty environment, except unbuilt loader plumbing, so ambient credentials do not cross through `process.env`.
- Host/worker messages use structured-clone data, with plain-JSON validation at the script boundary.
- **Meta as DATA** (`validateMeta`, host-side): the workflow's identity arrives on the start request as plain JSON (the tool carries it as its schema-validated `meta` parameter — never as script text) and is shape-validated loud, every violation named (`name`/`description` required; unknown fields rejected). The engine deliberately evaluates NO script text to obtain meta: an evaluated meta literal could smuggle getters that run on the host outside any vm timeout — the exact spin the worker thread exists to isolate. A body that still opens with a Claude Code-style `export const meta` statement is rejected with a pointed `SCRIPT_PARSE` message.
- **Hooks**: `agent(prompt, {label, phase, schema, model})` (schema = the [structured-output subset](../../core/tools/README.md), forwarded as `outputSchema`; result = validated object, or final text without a schema; a failed child resolves `null`), `parallel(thunks)`, `pipeline(items, ...stages)` with NO cross-stage barrier and `(prev, item, index)` stage callbacks, `phase(title)`, `log(message)`, and the `args` global. Anything else — `effort`/`isolation`/`agentType`, unknown options, malformed arguments, schemas outside the subset — throws a FATAL `WorkflowError` that `parallel`/`pipeline` re-throw rather than nulling (see the seam README's failure discipline).
- **No ambient APIs**: no timers, filesystem, or Node APIs are injected into the context (absence is API surface, not containment — see the trust premise).
A genuinely untrusted-script sandbox would require a different engine behind the same workflow seam.
## How a run executes
## Script contract
`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (`src/worker.ts` unbuilt via an explicit tsx `execArgv`; the sibling `lib/worker.js` bundle when built) with the meta, body, `args`, and worker-side limits as `workerData`.
The workflow's `meta` is host-provided data, not evaluated script text. The engine validates its required `name` and `description`, rejects unknown fields, and parse-checks the body before returning a run.
Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**. `agent()` sends `child-start`, and the host starts the child on `ctx.subagents` with parent attribution, the shared per-run abort signal, and `outputSchema`/`model` pass-through.
Inside the worker, the script receives `args` and these hooks:
The host observes `run.result` immediately but buffers its snapshotted wire projection until `run.started` fulfills. It then replies `child-started` with the child id before forwarding settlement, so `workflow/agent-start` always names a ready child and precedes its end. A readiness rejection replies `child-start-error`, emits no workflow agent pair, and makes the host dispose the attempt because the worker never received a handle; the worker classifies it as fatal `AGENT_START` unless cancellation already owns the run. If readiness fulfills, an infrastructure result rejection crosses as `child-failed`/`AGENT_RESULT` regardless of whether that rejection settled before or after readiness. Child disposal acknowledgements complete the RPC.
- `agent(prompt, { label, phase, schema, model })` starts one host-side subagent. With a schema it returns the structured value; otherwise it returns final text. An ordinary failed child yields `null`.
- `parallel(thunks)` runs thunks under the configured concurrency limit.
- `pipeline(items, ...stages)` passes `(previous, item, index)` without a cross-stage barrier.
- `phase(title)` and `log(message)` emit observer narration.
Observer narration (`phase`/`log`/`agent-start`/`agent-end`) crosses as messages and re-emits as the seam's `workflow/*` events. A **ready→go handshake** gates the body: a cancellation racing worker boot arrives before `go`, so a run cancelled before start never executes the body at all.
Unknown options, malformed arguments, unsupported schemas, tripped caps, provider-start failures, and infrastructure result failures are fatal workflow errors. No timers, filesystem API, or Node globals are intentionally injected, though the trust caveat above still applies.
## The value boundary
## Run sequence
Values LEAVING the script (hook options/schemas, the script's return) are materialized by `materializeFromRealm`: a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested `undefined`), copying into plain containers via `defineProperty` so a `"__proto__"` key becomes a data property, never a prototype mutation. Getters are read ordinarily — the RESULT is what crosses; a read that throws fails loud; both run in the WORKER, never on the host. Values ENTERING the realm (`args`, `agent()` results, hook promises and their failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; `args` is cloned once at start so a script scribbling on it cannot mutate the caller's object. One script-visible consequence: an error thrown by a hook is built OUTSIDE the script's vm context, so `e instanceof Error` inside the script is `false` — branch on `e.name`/`e.code` instead (the combinators recognize fatality by `instanceof` against their own realm's class, which a script-built object can never pass, so fatal-vs-null cannot be forged or dissolved).
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
## Cancellation, death, disposal
For each `agent()` call:
Per-run limits: a concurrency semaphore (`maxConcurrentAgents`), a total-`agent()` cap (`maxTotalAgents`), and a per-call item cap (`maxItemsPerCall`), all config. `cancel()` posts the cancel to the worker (its hooks start throwing `CANCELLED`; the script dies at its next await) and cancels every host-side child NOW on **both seam channels** — the shared request signal aborts AND each registered child's explicit `cancel()` is called host-side, because the seam leaves a provider free to honor either channel and a worker wedged in a synchronous spin could not relay its own per-child cancel RPCs (those later land as idempotent no-ops). The grace then arms: a run still unsettled `disposeGraceMs` later force-settles `cancelled` and the worker is **terminated**. A cancellation that lands before the body runs (the ready→go handshake) reports `cancelled` without executing anything; a worker `result` racing an in-flight host cancellation reports `cancelled` too (first-wins settlement — the seam-visible result had not settled when cancellation was requested); post-cancel `phase`/`log` narration is suppressed host-side, while cancelled children still deliver their paired `agent-end`.
1. The worker sends `child-start` with a plain-data prompt and options.
2. The host calls the configured provider through async `SubagentService.start`, passing the workflow's parent and one canonical per-run abort signal.
3. If start rejects, the host sends `child-start-error`; provider startup has already reached quiescence and no child lifecycle event is emitted.
4. If start fulfills while the workflow still admits work, the host records the run, observes `result`, then sends `child-started`. Even an already-settled result is forwarded afterward, preserving start-before-result order.
5. The worker emits paired `workflow/agent-start` and `workflow/agent-end` narration and requests child disposal after collection.
A worker that dies unexpectedly (an OOM, a script reaching `process.exit` through the documented vm escape) settles the run `stopReason: 'error'` with the exit diagnostics — or `'cancelled'` when a cancel was in flight — and the host-side child registry is what winds every surviving child down. `dispose()` = cancel + immediate host-driven disposal of every registered child (a wedged worker can relay no dispose RPC, so child teardown overlaps the grace instead of starting after it; the worker's own dispose RPCs join the same per-child disposal) + bounded wait (result, then child-registry quiescence, capped by the grace) + unconditional `worker.terminate()`: the thread never outlives its run. Once a run settles, stray children a script fired without awaiting are cancelled too, and `dispose()` waits for their disposal (bounded by the grace) before returning. `agent-start`/`agent-end` pairing is host-guaranteed the same way: forwarded starts live in a ledger, worker-reported ends pair them on the graceful paths, and the termination paths (grace force-settle, worker death) synthesize the missing ends (outcome `cancelled`) before the run settles — a start still in flight across the force-settle can surface after `workflow/end`, immediately paired the same way.
Provider starts are tracked separately from published children. If cancellation, worker death, or normal workflow settlement closes admission while a start is pending, the shared signal aborts it. A provider that nevertheless fulfills after closure is disposed by the host and never announced to the worker.
**Engine-specific limitations**: worker startup is paid per run; on a termination path `agentsStarted` reports the HOST-observed count (accepted `child-start`s — calls still queued worker-side for a concurrency slot are unknowable then); and a returned promise or thenable resolves per JavaScript semantics BEFORE materialization — that is what makes an un-awaited `return agent('x')` work — with the value-boundary guard applying to the resolution.
## Value boundary
Values leaving the script pass through `materializeFromRealm`, which accepts plain, lossless JSON data and rejects exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, and nested `undefined`. The walk runs in the worker, and defines object keys as data properties so `__proto__` cannot mutate a prototype.
Child results are projected and snapshotted before crossing from the host to the worker. This is a real process-like serialization boundary; it is deliberately different from trusted same-process workflow and subagent event payloads, which are borrowed immutable values.
## Cancellation and disposal
`WorkflowRun.cancel()` records the first reason, tells the worker to cancel, aborts the one signal shared by every pending and published child, and arms the `disposeGraceMs` timer. Worker hooks then throw `CANCELLED` at their next await. If the run remains unsettled at the deadline, the host resolves it as cancelled, pairs stranded child lifecycle events, and terminates the worker.
The subagent seam has one cancellation channel: the request signal. There is no separate child-cancel RPC. Published child teardown uses `run.dispose()`; pending provider starts remain provider-owned until their promise rejects or fulfills.
Normal settlement also aborts pending starts and begins disposing any published fire-and-forget children before the result becomes externally settled. The host's quiescence condition includes both pending starts and published child disposals, so cleanup does not forget an async startup transaction.
`dispose()` is idempotent. It cancels the run, starts host-driven disposal immediately, waits for result plus child quiescence up to the same grace, terminates the worker unconditionally, and performs a final survivor sweep. Per-child disposal is memoized so worker RPC, host cancellation, death cleanup, and public disposal all join one operation.
## Outcome and event guarantees
Terminal outcome is first-wins at host claim points. An accepted external cancellation overrides a later non-cancelled worker result; a result or worker death that claims first cannot be rewritten by reentrant cleanup callbacks.
Worker error, message failure, or premature exit closes message admission before cleanup, then resolves `error` unless cancellation already owns the run. Late queued messages cannot create children or narrate after that logical boundary.
The host keeps a ledger of forwarded child starts. A graceful worker supplies their ends; death or force termination synthesizes any missing end as cancelled. Every forwarded `workflow/agent-start` is therefore paired exactly once, although cleanup after an already-arrived workflow result may complete afterward.
## Config
| Key | Default | Meaning |
|---|---|---|
| `provider` | `spawn` | The `ctx.subagents` provider children run on (host-side). |
| `maxConcurrentAgents` | `0` (auto) | Concurrent `agent()` ceiling; `0` resolves to `min(16, max(1, cores - 2))`. |
| `maxTotalAgents` | `1000` | Total `agent()` calls one run may start (runaway-loop backstop). |
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()`/`pipeline()` call. |
| `syncTimeoutMs` | `5000` | vm timeout for the script's initial synchronous slice (in the worker). |
| `disposeGraceMs` | `5000` | How long a cancelled run may stay unsettled before force-settle + terminate; also bounds `dispose()`. |
| `provider` | `spawn` | Host-side subagent provider used by `agent()`. |
| `maxConcurrentAgents` | `0` | Concurrent `agent()` ceiling; `0` resolves from available CPU parallelism. |
| `maxTotalAgents` | `1000` | Total `agent()` calls in one run. |
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()` or `pipeline()` call. |
| `syncTimeoutMs` | `5000` | VM timeout for the script's initial synchronous slice. |
| `disposeGraceMs` | `5000` | Bound before force-settlement/termination and for public disposal. |

1
packages/workflow/workflow-workerthread/package.json
View File

@@ -30,6 +30,7 @@
"@deepseek-ai/dsh-agent": "^0.0.1",
"@deepseek-ai/dsh-brand": "^0.0.1",
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-subagent": "^0.0.1",
"@deepseek-ai/dsh-tools": "^0.0.1",
"@deepseek-ai/dsh-workflow": "^0.0.1",

467
packages/workflow/workflow-workerthread/src/host.ts
View File

@@ -1,16 +1,53 @@
/**
* The host half of one worker-engine run: spawn the Worker, bridge its child RPC onto
* `ctx.subagents`, fan its observer messages into the engine's events, and own cancellation,
* the settle-within-grace guarantee, and child cleanup.
* The host half of one worker-engine run: spawn the Worker, bridge its child
* RPC onto the holder-bound subagent service, fan its observer messages into
* the engine's events, and own cancellation, the settle-within-grace
* guarantee, and child cleanup. The worker's lifetime IS the run's lifetime:
* `dispose()` always ends with `worker.terminate()`, so no thread outlives its
* run.
*
* The run's `result` promise settles exactly once, from whichever of these
* lands first: receipt of the worker's `result` message, an unexpected worker
* death (`error`/`messageerror`/premature `exit` → `stopReason: 'error'`, or
* `'cancelled'` when a cancel was in flight), or the post-cancel grace timer (a
* script that never settles is force-settled `cancelled` and its worker
* terminated — the real kill an in-process engine could not perform). At
* `result` receipt the host snapshots whether caller/signal/dispose
* cancellation is already in flight: an earlier cancellation overrides a
* non-cancelled report; otherwise the report wins before settlement-only child
* cleanup invokes arbitrary provider callbacks. Worker death uses the same
* boundary: it claims `error` (or a previously requested `cancelled`) before
* reaping children, so cleanup callbacks cannot rewrite the outcome. That
* first signal also closes inbound message admission: Node may emit `error`,
* then deliver queued messages, then emit `exit`, but those late messages may
* neither create work nor narrate after settlement. If Result or grace already
* owns the outcome, death preserves it while still cleaning resources; the
* eventual exit performs a final disposal-only sweep without repeating child
* cancellation.
*
* Provider starts and published children are tracked separately. Every start
* receives one shared per-run abort signal; the provider owns partial setup
* until its promise fulfills. If admission closes while a start is pending,
* the signal aborts it; a late fulfillment is disposed without publication to
* the worker. Ready runs enter a callId registry whose memoized disposal is
* shared by graceful worker RPC, public disposal, normal-settlement reap, and
* worker-death cleanup. Quiescence requires both pending starts and published
* children to drain. Lifecycle pairing is host-guaranteed independently:
* every forwarded `agent-start` enters a ledger, and a dead or terminated
* worker's missing `agent-end` is synthesized exactly once as cancelled. On a
* termination path `agentsStarted` reports the host-observed child-start count;
* calls still queued worker-side for a concurrency slot are unknowable.
*
* @module @deepseek-ai/dsh-workflow-workerthread/host
*/
import { fileURLToPath } from 'node:url'
import { Worker } from 'node:worker_threads'
import type { WorkerOptions } from 'node:worker_threads'
import type { Context } from 'cordis'
import type { Agent } from '@deepseek-ai/dsh-agent'
import { assertNever } from '@deepseek-ai/dsh-llm'
import { snapshotJsonValue } from '@deepseek-ai/dsh-session'
import type SubagentService from '@deepseek-ai/dsh-subagent'
import type { SubagentRun } from '@deepseek-ai/dsh-subagent'
import type { WorkflowAgentEndInfo, WorkflowAgentInfo, WorkflowMeta, WorkflowResult, WorkflowRun, WorkflowRunId } from '@deepseek-ai/dsh-workflow'
import { renderThrown } from './realm.ts'
@@ -19,9 +56,36 @@ import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
import type { HostToWorkerPayloads, WorkerToHostMessage } from './protocol.ts'
import type { ChildResult, ChildStartRequest, WorkerInit } from './types.ts'
/** One published child and its shared quiescent-disposal transaction. */
interface ChildRecord {
readonly run: SubagentRun
disposal?: Promise<void>
}
/**
* Resolve the worker entry and spawn options for the current runtime shape.
* Unbuilt (tsx demos, vitest — `import.meta.url` points into `src/`), the
* entry is a JavaScript data-URL bootstrap. That bootstrap runs INSIDE the
* user worker, registers tsx's ESM AND CommonJS transforms there, and only
* then imports the TypeScript sibling. The whole mixed-module source graph
* therefore receives TypeScript transformation and the tsconfig paths map in
* the worker's own module-loader realm. A worker inherits no
* transform pipeline from vitest (vite transforms in-process), and a parent
* `--import tsx` registration is not a contract that user workers share on
* every supported Node line. Built (`lib/index.js`), the entry is the sibling
* bundle the package tsdown config emits and no loader is needed (`execArgv`
* pinned empty in both shapes — hermetic, like the environment).
*
* Both shapes spawn with an EMPTY environment (`env: {}`): the documented vm
* escape reaches `process`, and the harness's ambient credentials
* (`DEEPSEEK_API_KEY` et al.) must not ride along — the same stance as
* `dsh-code-runtime-worker`, stronger than the scrubbed env the
* defensive-patterns rule requires for spawned commands (a shell needs PATH;
* this worker needs nothing). Sole exception: the unbuilt shape forwards
* `TSX_TSCONFIG_PATH` when the parent carries it (loader plumbing the paths
* map depends on outside the repo cwd, not a secret). This closes the
* AMBIENT channel only — an escapee still holds process-wide privileges
* like fs access (the README's trust premise stands).
* @param init - the run payload, passed as `workerData`.
* @returns the entry URL and the Worker options to spawn it with.
*/
@@ -30,14 +94,31 @@ function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOpti
if (!import.meta.url.endsWith('.ts')) {
return { entry: new URL('./worker.js', import.meta.url), options: { workerData: init, env: {}, execArgv: [] } }
}
// Lazy tsx resolution: only the unbuilt shape needs it, so the built bundle never requires
// tsx to be installed.
// Resolve tsx lazily: only the unbuilt shape executes this arm, so a built
// consumer never needs the dev-only loader installed. A JavaScript entry is
// essential — it can install tsx's ESM and CommonJS hooks from INSIDE the
// user worker before any TypeScript enters Node's native strip-only parser.
// Both hooks are load-bearing because the source graph crosses both module
// shapes on supported Node lines. TSX_TSCONFIG_PATH is
// the one variable forwarded through the scrub: a parent running outside
// the repo cwd (the ACP snapshot harness is the real case) pins the paths
// map through it. Loader plumbing, not a secret.
const workerEntry = new URL('./worker.ts', import.meta.url)
const tsxEsmApiEntry = import.meta.resolve('tsx/esm/api')
const tsxCjsApiEntry = import.meta.resolve('tsx/cjs/api')
const bootstrap = [
`import { register as registerEsm } from ${JSON.stringify(tsxEsmApiEntry)}`,
`import { register as registerCjs } from ${JSON.stringify(tsxCjsApiEntry)}`,
'registerCjs()',
'registerEsm()',
`await import(${JSON.stringify(workerEntry.href)})`,
].join('\n')
return {
entry: new URL('./worker.ts', import.meta.url),
entry: new URL(`data:text/javascript,${encodeURIComponent(bootstrap)}`),
options: {
workerData: init,
env: process.env.TSX_TSCONFIG_PATH === undefined ? {} : { TSX_TSCONFIG_PATH: process.env.TSX_TSCONFIG_PATH },
execArgv: ['--import', fileURLToPath(import.meta.resolve('tsx'))],
execArgv: [],
},
}
}
@@ -45,15 +126,21 @@ function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOpti
/**
* One live worker-engine run — the seam's {@link WorkflowRun}, returned by
* `start()` directly. Owns the Worker, the child registry, and the result
* settlement; `result` never rejects. `meta` is this handle's OWN clone
* (event payloads carry separate clones), so a consumer mutating it corrupts
* nothing.
* settlement; `result` never rejects. `meta` is trusted same-process data
* borrowed as immutable by the handle and lifecycle events. The holder-bound
* SubagentService handle is captured before the
* engine returns this run, so unloading the engine removes only the ability to
* start another workflow; this run can still start and clean up its children.
*/
export class WorkerRun implements WorkflowRun {
/** Settles exactly once with the run's outcome; never rejects. */
readonly result: Promise<WorkflowResult>
private settleResolve!: (result: WorkflowResult) => void
private settled = false
/** A Result/death/grace outcome atomically won before teardown callbacks. */
private terminalClaimed = false
/** The first death signal closes worker-message admission and owns failure-time cleanup. */
private workerDeathObserved = false
private cancelReason: string | undefined
private graceTimer: NodeJS.Timeout | undefined
private readonly worker: Worker
@@ -61,19 +148,23 @@ export class WorkerRun implements WorkflowRun {
private workerGone = false
/** Accepted `child-start` messages — the terminate-path `agentsStarted` (see module doc). */
private hostStarted = 0
/** Live children by callId; an entry leaves ONLY after its dispose settles (quiescence = empty). */
private readonly children = new Map<number, SubagentRun>()
/** In-flight child disposals by callId — the memo that gives every path (worker RPC, dispose(), reap) ONE shared disposal per child. */
private readonly childDisposals = new Map<number, Promise<void>>()
/** Published children by callId; an entry leaves only after disposal settles. */
private readonly children = new Map<number, ChildRecord>()
/** Provider starts that have not yet fulfilled or rejected. */
private readonly pendingStarts = new Set<Promise<void>>()
/** Started-but-not-ended agents by seq — the pairing ledger the HOST guarantees (see {@link endAgent}). */
private readonly liveAgents = new Map<number, WorkflowAgentInfo>()
private readonly quiescenceWaiters: (() => void)[] = []
/** The per-run abort fanout every child start request carries. */
private readonly controller = new AbortController()
/** External start signal and the exact callback installed on it, retained only until first settle/teardown. */
private inputSignal: AbortSignal | undefined
private inputSignalAbort: (() => void) | undefined
private disposed: Promise<void> | undefined
constructor(
private readonly ctx: Context,
private readonly subagents: SubagentService,
readonly id: WorkflowRunId,
readonly meta: WorkflowMeta,
private readonly parent: Agent,
@@ -90,34 +181,53 @@ export class WorkerRun implements WorkflowRun {
const { entry, options } = resolveWorkerSpawn(init)
this.worker = new Worker(entry, options)
this.worker.on('message', (message: WorkerToHostMessage) => { this.onMessage(message) })
this.worker.on('error', (error) => { this.onWorkerDeath(`workflow worker failed: ${renderThrown(error)}`) })
this.worker.on('error', (error) => { this.onWorkerDeath(`workflow worker failed: ${renderThrown(error)}`, false) })
/* v8 ignore next -- messageerror: not constructible from the engine's own protocol (every payload is JSON data) */
this.worker.on('messageerror', (error) => { this.onWorkerDeath(`workflow worker message failed to deserialize: ${renderThrown(error)}`) })
this.worker.on('messageerror', (error) => { this.onWorkerDeath(`workflow worker message failed to deserialize: ${renderThrown(error)}`, false) })
this.worker.on('exit', (code) => {
this.workerGone = true
this.onWorkerDeath(`workflow worker exited before the run settled (exit code ${code})`)
this.onWorkerDeath(`workflow worker exited before the run settled (exit code ${code})`, true)
})
if (signal?.aborted) {
this.cancel('workflow start signal already aborted')
} else {
signal?.addEventListener('abort', () => { this.cancel('workflow signal aborted') }, { once: true })
} else if (signal !== undefined) {
const onAbort = (): void => {
this.detachInputSignal()
this.cancel('workflow signal aborted')
}
this.inputSignal = signal
this.inputSignalAbort = onAbort
signal.addEventListener('abort', onAbort, { once: true })
}
}
/**
* Cancel the worker and host-owned children, then arm forced settlement.
* Cancel the run: the worker is told (its hooks start throwing and the
* script dies at its next await), the required signal shared by every child
* start is aborted, and the grace timer
* arms: a run still unsettled `disposeGraceMs` later force-settles
* `cancelled` and its worker is TERMINATED. Idempotent; the first reason
* wins.
* @param reason - human-readable cause (default `'workflow cancelled'`).
*/
cancel(reason?: string): void {
// Do not arm a grace timer after settlement.
if (this.settled || this.cancelReason !== undefined) return
// A settled run has nothing left to cancel, and a terminal source claimed
// before its cleanup callbacks must exclude cancellation reentered by one
// of those callbacks. Without the settled guard the
// ordinary consumer path (await result, then dispose -> cancel) would arm
// a grace timer nothing ever clears, pinning the run and its Worker
// closure until the grace expires - a bounded leak per completed run.
if (this.settled || this.terminalClaimed || this.cancelReason !== undefined) return
this.cancelReason = reason ?? 'workflow cancelled'
this.post(HostToWorkerType.Cancel, { reason: this.cancelReason })
this.controller.abort(this.cancelReason)
// Host-side cancellation still reaches children when the worker is wedged.
for (const run of this.children.values()) run.cancel(this.cancelReason)
this.abortChildren(this.cancelReason)
this.graceTimer = setTimeout(() => {
// Pair stranded child starts before terminal workflow events.
// Cancellation already owns the race through cancelReason; close the
// terminal boundary explicitly before observer teardown callbacks.
this.terminalClaimed = true
// The worker may no longer speak (it is about to be terminated): pair
// every stranded start before the run settles, so ends precede
// workflow/end.
this.endStrandedAgents()
this.settleResult(this.cancelledResult(this.hostStarted))
void this.worker.terminate()
@@ -127,14 +237,36 @@ export class WorkerRun implements WorkflowRun {
}
/**
* Cancel + bounded settle + termination.
*
* Cancel + bounded settle + termination. Host-drives every registered
* child's disposal IMMEDIATELY — a wedged worker can relay no dispose RPC,
* and deferring child teardown to the post-terminate reap would spend the
* whole grace waiting for a quiescence that cannot start, then return with
* the disposals still in flight — so child disposal overlaps the same
* grace the worker gets to settle (the worker's own dispose RPCs join the
* shared per-child disposal). Waits (at most the grace) for the result and
* child quiescence, then terminates the worker unconditionally — the
* thread never outlives its run — and reaps whatever children remain
* (their disposal is contained, not awaited past the grace, the same
* abandonment the seam documents for a slow-disposing child). Idempotent;
* safe on every path.
* @returns resolves when the run's resources are released or abandoned.
*/
dispose(): Promise<void> {
this.disposed ??= (async () => {
if (this.disposed !== undefined) return this.disposed
// Claim the public transaction BEFORE its body invokes child/provider
// disposal. A raw provider callback can reenter handle.dispose(); it must
// join this promise rather than start a second traversal.
const claimed = Promise.withResolvers<undefined>()
this.disposed = claimed.promise
void (async () => {
this.detachInputSignal()
this.cancel('workflow disposed')
for (const [callId, run] of [...this.children]) void this.disposeChild(callId, run)
// cancel() deliberately becomes a no-op after terminal settlement, but
// disposal still owns every registered child. Reap independently so an
// already-settled workflow cannot wait on child quiescence before it has
// started the surviving children's disposals. On an unsettled run this
// joins the cancel path through the per-call cancellation/disposal gates.
this.reapChildren('workflow disposed')
await Promise.race([
(async () => {
await this.result
@@ -144,13 +276,17 @@ export class WorkerRun implements WorkflowRun {
])
await this.worker.terminate()
this.reapChildren('workflow disposed')
})()
})().then(
() => { claimed.resolve(undefined) },
/* v8 ignore next -- result/quiescence never reject and Worker.terminate is the only external promise */
(error: unknown) => { claimed.reject(error) },
)
return this.disposed
}
/** Post one message to the worker (payload looked up from the tag's map entry), tolerating a thread that is already gone. */
private post<T extends HostToWorkerType>(type: T, payload: HostToWorkerPayloads[T]): void {
if (this.workerGone) return
if (this.workerGone || this.workerDeathObserved) return
try {
this.worker.postMessage({ type, ...payload })
} catch (error: unknown) {
@@ -163,6 +299,11 @@ export class WorkerRun implements WorkflowRun {
}
private onMessage(message: WorkerToHostMessage): void {
// Node may emit `error`, then deliver an already-queued `message`, then
// emit `exit`. The first death signal is the host's logical delivery
// barrier: nothing arriving afterward may create a child, narrate after
// workflow/end, or compete with the chosen outcome.
if (this.workerDeathObserved) return
switch (message.type) {
case WorkerToHostType.Ready:
this.post(HostToWorkerType.Go, {})
@@ -192,9 +333,6 @@ export class WorkerRun implements WorkflowRun {
case WorkerToHostType.ChildStart:
this.onChildStart(message.callId, message.request)
break
case WorkerToHostType.ChildCancel:
this.children.get(message.callId)?.cancel(message.reason)
break
case WorkerToHostType.ChildDispose:
this.onChildDispose(message.callId)
break
@@ -207,18 +345,44 @@ export class WorkerRun implements WorkflowRun {
}
}
private onChildStart(callId: number, request: ChildStartRequest): void {
/** Why a ready provider result may no longer be admitted to the worker. */
private childAdmissionFailure(): { reason: string; rendered: string } | undefined {
if (this.cancelReason !== undefined) {
// The worker's start raced our cancel: refuse — a child must never
// start on an already-aborted signal (a provider subscribing only to
// future abort events would never observe it).
this.post(HostToWorkerType.ChildStartError, { callId, rendered: `workflow run cancelled: ${this.cancelReason}` })
return { reason: this.cancelReason, rendered: `workflow run cancelled: ${this.cancelReason}` }
}
if (this.workerDeathObserved) {
return { reason: 'workflow worker gone', rendered: 'workflow worker is no longer available' }
}
if (this.terminalClaimed) {
return { reason: 'workflow settled', rendered: 'workflow run already settled' }
}
return undefined
}
private onChildStart(callId: number, request: ChildStartRequest): void {
const initialFailure = this.childAdmissionFailure()
if (initialFailure !== undefined) {
// Refuse after a terminal boundary: a child must never start on an
// already-aborted signal (a provider subscribing only to future abort
// events would never observe it).
this.post(HostToWorkerType.ChildStartError, { callId, rendered: initialFailure.rendered })
return
}
this.hostStarted += 1
const task = this.startChild(callId, request)
this.pendingStarts.add(task)
void task.then(
() => { this.finishPendingStart(task) },
/* v8 ignore next -- startChild contains provider and cleanup failures */
() => { this.finishPendingStart(task) },
)
}
/** Await one provider-owned startup transaction and publish only while admitted. */
private async startChild(callId: number, request: ChildStartRequest): Promise<void> {
let run: SubagentRun
try {
run = this.ctx.subagents.start(this.provider, {
run = await this.subagents.start(this.provider, {
prompt: [{ type: 'text', text: request.prompt }],
parent: this.parent,
signal: this.controller.signal,
@@ -226,21 +390,38 @@ export class WorkerRun implements WorkflowRun {
...request.model !== undefined ? { agentOptions: { model: request.model } } : {},
})
} catch (error: unknown) {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
const failure = this.childAdmissionFailure()
this.post(HostToWorkerType.ChildStartError, {
callId,
rendered: failure?.rendered ?? renderThrown(error),
})
return
}
const failure = this.childAdmissionFailure()
if (failure !== undefined) {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: failure.rendered })
try {
await run.dispose()
} catch (error: unknown) {
this.ctx.logger.warn(`workflow-workerthread: refused child dispose failed: ${renderThrown(error)}`)
}
return
}
this.children.set(callId, run)
const childId = run.id
// Observe settlement IMMEDIATELY, before readiness.
const record: ChildRecord = { run }
this.children.set(callId, record)
// Attach result forwarding before publishing the child handle. Because the
// callback itself runs in a later microtask, ChildStarted is still posted
// first even for an already-settled scripted provider.
const forwardResult = run.result.then<() => void, () => void>(
(result) => {
try {
const snapshot: ChildResult = structuredClone({
const snapshot = snapshotJsonValue<ChildResult>({
output: result.output,
...result.structured !== undefined ? { structured: result.structured } : {},
stopReason: result.stopReason,
})
if (snapshot === undefined) throw new TypeError('child result is not losslessly JSON-serializable')
return () => { this.post(HostToWorkerType.ChildSettled, { callId, result: snapshot }) }
} catch (error: unknown) {
const rendered = `workflow child result could not cross the worker boundary: ${renderThrown(error)}`
@@ -252,88 +433,104 @@ export class WorkerRun implements WorkflowRun {
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
},
)
// The provider owns the publication boundary.
void run.started.then(
() => {
this.post(HostToWorkerType.ChildStarted, { callId, childId })
void forwardResult.then((forward) => { forward() })
},
(error: unknown) => {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
if (this.children.get(callId) === run) void this.disposeChild(callId, run)
},
)
this.post(HostToWorkerType.ChildStarted, { callId, childId: run.id })
void forwardResult.then((forward) => { forward() })
}
private onChildDispose(callId: number): void {
const run = this.children.get(callId)
if (run === undefined) {
const record = this.children.get(callId)
if (record === undefined) {
// Already disposed host-side (a dispose() drive or a death reap beat
// the RPC) — the ack is still owed (the worker-side wrapper awaits it).
this.post(HostToWorkerType.ChildDisposed, { callId })
return
}
// disposeChild never rejects (containment is inside), so the ack always follows.
void this.disposeChild(callId, run).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
void this.disposeChild(callId, record).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
}
/**
* Start (or join) one registered child's disposal; the registry entry leaves when it
* settles.
*
* Start (or join) one registered child's disposal; the registry entry
* leaves when it settles. Memoized per callId: the worker's dispose RPC,
* the dispose() host drive, and the reap can all land on the same child —
* the child's `dispose()` runs once and every caller awaits that one
* settlement. A rejection is contained (the subagent seam's dispose() is
* not supposed to reject, but a backend that does anyway must not break
* quiescence): logged, and the child still leaves the registry.
* @param callId - the child's registry key.
* @param run - the registered child (the caller looked it up).
* @param record - the registered child (the caller looked it up).
* @returns resolves when the disposal settled either way; never rejects.
*/
private disposeChild(callId: number, run: SubagentRun): Promise<void> {
let disposal = this.childDisposals.get(callId)
if (disposal === undefined) {
// The seam promises a Promise, but invoke inside an async boundary so a
// contract-violating synchronous throw is contained exactly like a
// rejected disposal and cannot break host quiescence.
disposal = (async () => { await run.dispose() })().then(
() => { this.finishChild(callId) },
(error: unknown) => {
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)
this.finishChild(callId)
},
)
this.childDisposals.set(callId, disposal)
}
return disposal
private disposeChild(callId: number, record: ChildRecord): Promise<void> {
if (record.disposal !== undefined) return record.disposal
record.disposal = Promise.resolve()
.then(() => record.run.dispose())
.catch((error: unknown) => {
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)
})
.then(() => { this.finishChild(callId) })
return record.disposal
}
/** Drop a child from the registry (and its disposal memo), releasing quiescence waiters at zero. */
/** Drop a child record and release quiescence waiters when all work ends. */
private finishChild(callId: number): void {
this.children.delete(callId)
this.childDisposals.delete(callId)
if (this.children.size === 0) {
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
}
this.notifyChildQuiescence()
}
/** Resolves once the child registry is empty (every disposal settled). */
/** Retire one provider startup transaction. */
private finishPendingStart(task: Promise<void>): void {
this.pendingStarts.delete(task)
this.notifyChildQuiescence()
}
/** Release waiters only after both pending starts and published children end. */
private notifyChildQuiescence(): void {
if (this.children.size !== 0 || this.pendingStarts.size !== 0) return
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
}
/** Resolves once every pending start and published child has reached quiescence. */
private childQuiescence(): Promise<void> {
if (this.children.size === 0) return Promise.resolve()
if (this.children.size === 0 && this.pendingStarts.size === 0) return Promise.resolve()
return new Promise((resolve) => { this.quiescenceWaiters.push(resolve) })
}
/** Abort + dispose every registered child (worker death / final teardown); disposal is contained, not awaited. */
private reapChildren(reason: string): void {
this.controller.abort(this.cancelReason ?? reason)
for (const [callId, run] of [...this.children]) {
run.cancel(this.cancelReason ?? reason)
void this.disposeChild(callId, run)
this.abortChildren(this.cancelReason ?? reason)
for (const [callId, record] of [...this.children]) {
void this.disposeChild(callId, record)
}
}
/** Abort the one canonical signal shared by pending and published children. */
private abortChildren(reason: string): void {
if (!this.controller.signal.aborted) this.controller.abort(reason)
}
private onResult(result: WorkflowResult): void {
// The worker's settle-reap already child-cancel()s every stray; this
// abort fires the seam signal too, for providers that only honor the
// request signal (both channels, on every path).
if (this.cancelReason === undefined) this.controller.abort('workflow settled')
if (this.cancelReason !== undefined && result.stopReason !== 'cancelled') {
// The owned worker session sends one Result. Keep a late duplicate or a
// Result queued behind another terminal source completely side-effect-free.
if (this.terminalClaimed) return
// First-wins is decided when the Result message reaches the host. If no
// external cancellation was already in flight, this result won. Reaping a
// stray child below may synchronously reenter cancel() through provider
// callbacks, but that internal post-result cleanup must not retroactively
// rewrite the worker result that arrived first.
const cancellationWasRequested = this.cancelReason !== undefined
// Claim before settlement cleanup invokes provider disposal. Once Result
// won, a later cancellation cannot rewrite it.
this.terminalClaimed = true
// Abort pending starts and begin disposing published children before the
// workflow becomes externally settled. Cleanup remains independently
// tracked by childQuiescence and the holder's dispose().
this.reapChildren('workflow settled')
if (!cancellationWasRequested) {
this.settleResult(result)
return
}
if (result.stopReason !== 'cancelled') {
// The script settled while our cancel was crossing the thread boundary
// — the seam-visible result had NOT settled when cancellation was
// requested, so report cancelled (the vm drive()'s post-settle check,
@@ -344,21 +541,39 @@ export class WorkerRun implements WorkflowRun {
this.settleResult(result)
}
/** An unexpected worker death (or the expected exit after termination). */
private onWorkerDeath(message: string): void {
// Whatever the worker left behind must not leak — abort + dispose it all.
if (this.children.size > 0) this.reapChildren('workflow worker gone')
// The thread is gone: no more worker-authored agent-ends can arrive —
// pair every stranded start (a start that crossed between the grace
// force-settle and this exit included) before the run settles.
this.endStrandedAgents()
// settleResult no-ops on an already-settled run (the expected exit after
// a dispose's terminate lands here too).
if (this.cancelReason !== undefined) {
this.settleResult(this.cancelledResult(this.hostStarted))
return
/** Process an error/messageerror/exit signal; `exit` also performs the final disposal sweep. */
private onWorkerDeath(message: string, isExit: boolean): void {
if (!this.workerDeathObserved) {
// Close message admission BEFORE cleanup callbacks: Node can deliver a
// message queued before the crash after its `error` event. Treating the
// first death signal as a logical barrier prevents that late message
// from creating work or narrating after workflow/end.
this.workerDeathObserved = true
const outcomeWasClaimed = this.terminalClaimed
const cancellationWasRequested = this.cancelReason !== undefined
// When death is itself the terminal source, claim BEFORE child reap or
// synthesized observer callbacks. Either can reenter cancel(); a death
// that arrived first remains an error, while a cancellation already
// accepted before death remains cancelled. If Result/grace already won,
// preserve it while still performing prompt failure-time cleanup.
if (!outcomeWasClaimed) this.terminalClaimed = true
if (this.children.size > 0 || this.pendingStarts.size > 0) this.reapChildren('workflow worker gone')
this.endStrandedAgents()
if (!outcomeWasClaimed) {
if (cancellationWasRequested) {
this.settleResult(this.cancelledResult(this.hostStarted))
} else {
this.settleResult({ value: null, stopReason: 'error', error: message, agentsStarted: this.hostStarted })
}
}
}
this.settleResult({ value: null, stopReason: 'error', error: message, agentsStarted: this.hostStarted })
if (!isExit) return
// `error` is not Node's physical delivery barrier: a queued message may
// precede `exit`. Admission is already closed, so this final sweep only
// joins/starts disposal for registry survivors; it deliberately does not
// repeat explicit provider cancellation.
for (const [callId, record] of [...this.children]) void this.disposeChild(callId, record)
this.endStrandedAgents()
}
/**
@@ -377,11 +592,14 @@ export class WorkerRun implements WorkflowRun {
/**
* Synthesize the missing `agent-end` for every started-but-unpaired agent,
* outcome `'cancelled'`: the reap cancels every child, and a real
* settlement racing the force-settle loses to the cancellation — the same
* first-wins override {@link onResult} applies to the run's own result.
* settlement racing the force-settle loses to that already-started external
* cancellation. The atomic terminal boundaries in {@link onResult} and
* {@link onWorkerDeath} deliberately exclude teardown callbacks as contenders.
* Called where the worker can no longer speak (the grace force-settle,
* worker death), BEFORE settleResult, so the paired ends reach observers
* before `workflow/end`.
* worker death, physical exit). When grace/death is the terminal source it
* runs before settleResult, so already-known pairs precede `workflow/end`;
* after an earlier Result, exit cleanup may close a survivor afterward.
* The ledger preserves exactly-once pairing in both orders.
*/
private endStrandedAgents(): void {
for (const info of [...this.liveAgents.values()]) {
@@ -397,10 +615,25 @@ export class WorkerRun implements WorkflowRun {
return { value: null, stopReason: 'cancelled', error: `workflow run cancelled: ${reason}`, agentsStarted }
}
/** First settle wins; disarms the grace timer. */
/** Remove the exact abort callback installed on the caller's start signal. */
private detachInputSignal(): void {
const signal = this.inputSignal
const onAbort = this.inputSignalAbort
if (signal === undefined || onAbort === undefined) return
this.inputSignal = undefined
this.inputSignalAbort = undefined
signal.removeEventListener('abort', onAbort)
}
/** First settle wins; disarms the grace timer and releases the caller signal. */
private settleResult(result: WorkflowResult): void {
// Every current terminal source claims ownership before calling here; keep
// the fallback local so a future caller cannot resolve twice.
/* v8 ignore next -- defensive fallback outside the claimed state machine */
if (this.settled) return
this.terminalClaimed = true
this.settled = true
this.detachInputSignal()
clearTimeout(this.graceTimer)
this.settleResolve(result)
}

17
packages/workflow/workflow-workerthread/src/index.ts
View File

@@ -115,9 +115,7 @@ export class WorkerWorkflowEngine extends WorkflowService {
const meta = validateMeta(request.meta)
assertBodyParses(request.script, meta.name)
const id = WorkflowRunId(randomUUID())
// The event payloads and the run handle get SEPARATE meta clones: a
// listener mutating its snapshot must not corrupt the holder's view.
const info: WorkflowRunInfo = { id, meta: structuredClone(meta) }
const info: WorkflowRunInfo = { id, meta }
const limits: WorkerLimits = {
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
? Math.min(16, Math.max(1, availableParallelism() - 2))
@@ -132,10 +130,19 @@ export class WorkerWorkflowEngine extends WorkflowService {
...request.args !== undefined ? { args: request.args } : {},
limits,
}
// Capture the dependency while this service call is still traced through
// the start() holder. Cordis strips the engine-provider shadow when it
// returns the SubagentService handle, so an already-returned run can keep
// starting children after an engine HMR unload removes ctx.workflows.
// Re-resolving `this.ctx.subagents` later from WorkerRun would instead walk
// the now-inactive engine fiber and break the seam's holder-owned lifetime.
const runCtx = this.ctx
const subagents = runCtx.subagents
const workerRun = new WorkerRun(
this.ctx,
runCtx,
subagents,
id,
structuredClone(meta),
meta,
request.parent,
init,
this.config.provider,

8
packages/workflow/workflow-workerthread/src/protocol.ts
View File

@@ -22,8 +22,6 @@ export enum WorkerToHostType {
AgentEnd = 'agent-end',
/** Child RPC: start a child on the host (answered by ChildStarted or ChildStartError). */
ChildStart = 'child-start',
/** Child RPC: cancel a started child (fire-and-forget). */
ChildCancel = 'child-cancel',
/** Child RPC: dispose a started child (answered by ChildDisposed). */
ChildDispose = 'child-dispose',
/** The run's single terminal result. */
@@ -44,8 +42,6 @@ export interface WorkerToHostPayloads {
[WorkerToHostType.AgentEnd]: { info: WorkflowAgentEndInfo }
/** The RPC correlation id and the prompt plus validated options. */
[WorkerToHostType.ChildStart]: { callId: number; request: ChildStartRequest }
/** The RPC correlation id and the cancel reason (undefined = unspecified). */
[WorkerToHostType.ChildCancel]: { callId: number; reason: string | undefined }
/** The RPC correlation id of the child to dispose. */
[WorkerToHostType.ChildDispose]: { callId: number }
/** The run's terminal outcome. */
@@ -58,9 +54,9 @@ export enum HostToWorkerType {
Go = 'go',
/** Cancel the run: hooks start throwing and the script dies at its next await. */
Cancel = 'cancel',
/** Child RPC reply: provider publication/readiness fulfilled (exactly one start reply per ChildStart). */
/** Child RPC reply: the provider fulfilled with a ready run (exactly one start reply per ChildStart). */
ChildStarted = 'child-started',
/** Child RPC reply: synchronous start or asynchronous publication/readiness failed. */
/** Child RPC reply: the provider's asynchronous start failed. */
ChildStartError = 'child-start-error',
/** Child RPC: a started child's result RESOLVED (its JSON projection). */
ChildSettled = 'child-settled',

101
packages/workflow/workflow-workerthread/src/runtime.ts
View File

@@ -1,8 +1,39 @@
/**
* Per-run execution state for the engine's worker side: the script's vm context and its
* injected hooks (`agent`/`parallel`/`pipeline`/`phase`/ `log`/`args`), the concurrency
* semaphore and caps, cancellation, and the drive loop that turns a script settlement into a
* {@link WorkflowResult}.
* Per-run execution state for the engine's THREAD side: the script's vm
* context and its injected hooks (`agent`/`parallel`/`pipeline`/`phase`/
* `log`/`args`), the concurrency semaphore and caps, cancellation, and the
* drive loop that turns a script settlement into a {@link WorkflowResult}.
* Children are started by RPC to the host through a {@link ChildPort}, so
* this module never touches a cordis context — it runs inside the worker
* thread.
*
* Value boundary (the trust premise lives in ./realm.ts): values ENTERING the
* worker-side host code from the script (hook options, schemas, the return
* value) are materialized by `materializeFromRealm` — a plain walk that
* rejects loud everything JSON cannot carry, which also makes every value
* safe for the later postMessage hop. Values ENTERING the realm (`args`,
* `agent()` results, hook promises and their failures, combinator arrays) are
* handed over DIRECTLY as worker-realm values: the script is model-written
* and trusted, so outer prototypes are not a leak. `args` is cloned once at
* start so a script scribbling on it cannot mutate the session's init object
* (a benign-bug guard; the postMessage clone already isolated the caller).
*
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
* unsupported options/schemas, tripped caps, synchronous start refusal,
* provider-start failure, ready-child result rejection, and
* cancellation) ALWAYS propagate through
* `parallel`/`pipeline` — recognized by `instanceof` against this realm's
* class, which a script inside the vm context cannot forge — and the per-item
* `null` is reserved for child-run failures and ordinary in-stage script
* errors. Every hook-returned promise gets a no-op rejection consumer, so a
* dropped promise cannot surface an unhandled rejection (which would kill the
* worker and read as an engine fault).
*
* There is deliberately NO worker-side abandon channel: a script that never
* settles after a cancel simply never posts a result, and the HOST enforces
* the settles-within-grace guarantee by force-settling `cancelled` and
* terminating the worker — the real kill an in-process engine could not have.
*
* @module @deepseek-ai/dsh-workflow-workerthread/runtime
*/
@@ -52,7 +83,8 @@ function defaultLabel(prompt: string): string {
/**
* One live script execution inside the worker. Constructed per run by the
* session; `drive()` is called exactly once and NEVER rejects — every failure
* becomes a {@link WorkflowResult} with a non-`completed` stop reason.
* becomes a {@link WorkflowResult} with a non-`completed` stop reason. The
* host owns cancellation and cleanup of any dropped child work.
*/
export class WorkflowExecution {
/** 1-based count of `agent()` calls started (the `agentsStarted` result field). */
@@ -61,7 +93,6 @@ export class WorkflowExecution {
private readonly slotWaiters: { resolve(): void; reject(error: unknown): void }[] = []
private cancelReason: string | undefined
private cancelError: WorkflowError | undefined
private readonly controller = new AbortController()
private currentPhase: string | undefined
private readonly context: vm.Context
private readonly compiled: vm.Script
@@ -74,8 +105,12 @@ export class WorkflowExecution {
private readonly observer: ExecutionObserver,
private readonly children: ChildPort,
) {
// Compile FIRST: a body syntax error must throw out of the constructor before any realm
// state exists.
// Compile FIRST: a body syntax error must throw out of the constructor
// before any realm state exists. The host pre-parses the identical
// wrapper, so under one Node version this throw is unreachable in
// production — the session still maps it to an error result defensively.
// lineOffset compensates for the wrapper line, so stack traces carry the
// script's own line numbers.
try {
this.compiled = new vm.Script(`(async () => {\n${body}\n})()`, {
filename: `workflow:${meta.name}`,
@@ -93,11 +128,8 @@ export class WorkflowExecution {
pipeline: (items: unknown, ...stages: unknown[]) => this.contain(this.pipeline(items, stages)),
phase: (title: unknown) => { this.phase(title) },
log: (message: unknown) => { this.log(message) },
// Cloned once: a script scribbling on args must not mutate the
// session's init object (a benign-bug guard; args is plain JSON by the
// seam contract and already crossed one structured clone as workerData,
// so this clone is total).
args: args === undefined ? undefined : structuredClone(args),
// workerData already performed the real cross-thread structured clone.
args,
}
for (const [key, value] of Object.entries(globals)) {
// Data properties on the contextified global; frozen shape not required —
@@ -128,19 +160,18 @@ export class WorkflowExecution {
}
/**
* Cancel the run: in-flight children get a cancel RPC (the shared abort fanout), waiting
* `agent()` slots reject, and every future hook call throws `CANCELLED` — the script dies at
* its next await.
*
* @param reason - human-readable cause, carried on the CANCELLED error and
* into child cancel RPCs. Required: every caller (the session's cancel
* message, drive()'s settle-reap) has a concrete reason.
* Cancel the run: waiting `agent()` slots reject and every future hook call
* throws `CANCELLED` — the script dies at its next await. A script that
* never settles anyway (parked on a promise no hook owns) is the HOST's
* problem: its grace timer force-settles the run and terminates the
* worker. Idempotent; the first reason wins.
* @param reason - human-readable cause carried on the CANCELLED error. The
* host independently aborts the required signal shared by every child.
*/
cancel(reason: string): void {
if (this.cancelReason !== undefined) return
this.cancelReason = reason
this.cancelError = new WorkflowError(`workflow run cancelled: ${this.cancelReason}`, 'CANCELLED')
this.controller.abort(this.cancelReason)
for (const waiter of this.slotWaiters.splice(0)) waiter.reject(this.cancelledError())
}
@@ -148,8 +179,8 @@ export class WorkflowExecution {
* Run the script to settlement. Resolves — never rejects — with the run's
* {@link WorkflowResult}: the materialized return value on `completed`, the
* failure message on `error`, and `cancelled` when the script died of
* cancellation. After settlement, any stray children a script fired without
* awaiting are cancelled (their `agent()` wrappers dispose them via RPC).
* cancellation. This method only chooses the result; the session publishes
* it and the host owns terminal child cancellation.
* @returns the settled outcome — this promise NEVER rejects (the seam's
* `result`-never-rejects contract); every failure maps to a variant.
*/
@@ -177,10 +208,6 @@ export class WorkflowExecution {
// cannot throw — drive() resolving is the `result` never-rejects seam
// contract.
return { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: this.started }
} finally {
// Reap strays: a script that fired agent() calls without awaiting them leaves live
// children behind after settlement — cancel them all.
if (this.cancelReason === undefined) this.cancel('workflow settled')
}
}
@@ -264,7 +291,11 @@ export class WorkflowExecution {
await this.acquireSlot()
try {
// Recheck cancellation after semaphore acquisition because acquire always yields.
// Re-check after the acquire: the await yields at least one microtask
// tick even when a slot is free, and a queued waiter resumes a tick
// after its release — a cancel() landing in either window must not
// reach the host (which would refuse anyway, but the refusal reads as
// a start failure rather than the cancellation it is).
this.throwIfCancelled()
let run: ChildHandle
try {
@@ -285,24 +316,21 @@ export class WorkflowExecution {
// wind the fresh child down instead of leaving it live behind a dead
// script.
if (this.isCancelled()) {
run.cancel(this.cancelReason)
await run.dispose()
throw this.cancelledError()
}
const info: WorkflowAgentInfo = { seq, label, ...phase !== undefined ? { phase } : {}, childId: AgentId(run.id) }
this.observer.agentStart(info)
// Cancellation reaches the child through an explicit cancel RPC per
// child (the host also aborts its own per-run signal, but the seam
// leaves a provider free to honor either channel, so both are driven).
const onAbort = (): void => { run.cancel(this.cancelReason) }
this.controller.signal.addEventListener('abort', onAbort, { once: true })
try {
let result
try {
result = await run.result
} catch (error: unknown) {
// A rejected child result is an INFRASTRUCTURE fault relayed by the host — distinct
// from a child that failed and resolved.
// A rejected child result is an INFRASTRUCTURE fault relayed by the
// host — distinct from a child that failed and resolved. Pair the
// lifecycle before propagating, and propagate FATAL: an ordinary
// throw would dissolve to a per-item null inside the combinators,
// and a broken provider must not read as a failed child.
if (this.isCancelled()) {
this.observer.agentEnd({ ...info, outcome: 'cancelled' })
throw this.cancelledError()
@@ -333,7 +361,6 @@ export class WorkflowExecution {
this.observer.agentEnd({ ...info, outcome: 'failed' })
return null
} finally {
this.controller.signal.removeEventListener('abort', onAbort)
await run.dispose()
}
} finally {

12
packages/workflow/workflow-workerthread/src/session.ts
View File

@@ -47,10 +47,6 @@ class RpcChildHandle implements ChildHandle {
this.result = entry.settled.promise
}
cancel(reason?: string): void {
this.post(WorkerToHostType.ChildCancel, { callId: this.callId, reason })
}
dispose(): Promise<void> {
this.post(WorkerToHostType.ChildDispose, { callId: this.callId })
return this.entry.disposed.promise
@@ -59,7 +55,7 @@ class RpcChildHandle implements ChildHandle {
/**
* The worker-side child-RPC bridge ({@link ChildPort}): allocates callIds,
* posts the start/cancel/dispose RPCs, and owns the per-call pending
* posts the start/dispose RPCs, and owns the per-call pending
* book-keeping the session's message handler settles via the `onChild*`
* entry points.
*/
@@ -77,10 +73,10 @@ class ChildRpcBridge implements ChildPort {
settled: Promise.withResolvers<ChildResult>(),
disposed: Promise.withResolvers<void>(),
}
// Containment: when synchronous start or asynchronous readiness fails (or
// Containment: when asynchronous provider start fails (or
// the run is torn down), the settled promise may never gain a consumer —
// it must not surface as an unhandled rejection and kill the worker.
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start/readiness */ })
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start */ })
this.pending.set(callId, entry)
this.post(WorkerToHostType.ChildStart, { callId, request })
const childId = await entry.started.promise
@@ -92,7 +88,7 @@ class ChildRpcBridge implements ChildPort {
this.pending.get(callId)?.started.resolve(childId)
}
/** Synchronous start or asynchronous readiness failed; reject and retire the pending RPC. */
/** Asynchronous provider start failed; reject and retire the pending RPC. */
onChildStartError(callId: number, rendered: string): void {
const entry = this.pending.get(callId)
this.pending.delete(callId)

4
packages/workflow/workflow-workerthread/src/types.ts
View File

@@ -72,8 +72,6 @@ export interface ChildHandle {
* failed for its own reasons resolves with a non-`completed` stop reason.
*/
readonly result: Promise<ChildResult>
/** Ask the host to cancel the child (fire-and-forget). */
cancel(reason?: string): void
/** Ask the host to dispose the child; resolves on the host's ack. */
dispose(): Promise<void>
}
@@ -87,7 +85,7 @@ export interface ChildPort {
* Start one child agent on the host (the `agent()` hook's start half).
* @param request - the prompt and validated options.
* @returns the ready child handle; rejects when synchronous start or the
* provider's asynchronous publication/readiness boundary fails.
* provider's asynchronous start fails.
*/
startAgent(request: ChildStartRequest): Promise<ChildHandle>
}

2
packages/workflow/workflow-workerthread/tests/integration.spec.ts
View File

@@ -49,7 +49,7 @@ describe('dsh-workflow-workerthread over the real in-process stack', () => {
])
const childIds: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => {
// The workflow bridge must honor SubagentRun.started: a start observer
// The workflow bridge must await asynchronous provider start: an observer
// sees the real spawn child already published, never a reserved id.
expect(ctx.agents.get(agent.childId)).toBeDefined()
childIds.push(agent.childId)

6
packages/workflow/workflow-workerthread/tests/session.spec.ts
View File

@@ -212,7 +212,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
host.close()
})
it('cancel mid-run: in-flight children get cancel RPCs, hooks throw at entry, the run reports cancelled', async () => {
it('cancel mid-run: hooks throw at entry and the run reports cancelled', async () => {
const host = fakeHost()
void runWorkerSession(host.port, init(`
phase('before')
@@ -232,7 +232,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
const result = await host.result()
expect(result.stopReason).toBe('cancelled')
expect(result.error).toContain('stop everything')
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
// No post-cancel narration left the runtime (the hooks threw at entry).
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['before'])
@@ -433,7 +432,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
host.close()
})
it('a cancel landing DURING the start round-trip winds the fresh child down (cancel + dispose) and dies cancelled', async () => {
it('a cancel landing DURING the start round-trip disposes the fresh child and dies cancelled', async () => {
const host = fakeHost({ manual: true })
void runWorkerSession(host.port, init("return await agent('p')"))
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
@@ -447,7 +446,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
const result = await host.result()
expect(result.stopReason).toBe('cancelled')
await vi.waitFor(() => {
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId)
})
// The child never became an agent-start: it was wound down pre-lifecycle.

38
packages/workflow/workflow-workerthread/tests/source-worker.compat.spec.ts Normal file
View File

@@ -0,0 +1,38 @@
/**
* Keyless runtime smoke for the source-mode workflow worker. The Node
* compatibility matrix runs this WHOLE file, so renaming or removing its test
* cannot turn the runtime proof into a successful zero-match filter.
*/
import { expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import { AgentId } from '@deepseek-ai/dsh-agent'
import type { Agent } from '@deepseek-ai/dsh-agent'
import SubagentService from '@deepseek-ai/dsh-subagent'
import WorkerWorkflowEngine from '../src/index.ts'
// A fresh thread compiles the source runtime. Leave contention headroom on
// shared CI runners without weakening any engine-level timeout assertion.
vi.setConfig({ testTimeout: 30_000 })
it('runs the default config through the source worker', async () => {
const ctx = new Context()
const subagents = await ctx.plugin(SubagentService)
const engine = await ctx.plugin(WorkerWorkflowEngine, {})
const parent = { id: AgentId('workflow-compat-parent'), options: {} } as unknown as Agent
try {
const run = ctx.workflows.start({
script: 'return 6 * 7',
meta: { name: 'source-worker-compat', description: 'exercise the unbuilt worker entry' },
parent,
})
try {
await expect(run.result).resolves.toMatchObject({ value: 42, stopReason: 'completed', agentsStarted: 0 })
} finally {
await run.dispose()
}
} finally {
await engine.dispose()
await subagents.dispose()
}
})

512
packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts
View File

@@ -1,5 +1,6 @@
import { describe, expect, it, vi } from 'vitest'
import { fileURLToPath } from 'node:url'
import type { Worker } from 'node:worker_threads'
import { Context } from 'cordis'
import Loader from '@cordisjs/plugin-loader'
import { AgentId } from '@deepseek-ai/dsh-agent'
@@ -8,7 +9,7 @@ import SubagentService from '@deepseek-ai/dsh-subagent'
import type { SubagentCapabilities, SubagentProvider, SubagentResult, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
import * as workerEngineModule from '../src/index.ts'
import WorkerWorkflowEngine, { HostToWorkerType, type Config } from '../src/index.ts'
import WorkerWorkflowEngine, { HostToWorkerType, WorkerToHostType, type Config } from '../src/index.ts'
/** A minimal parent stand-in: the engine only threads it through to the provider. */
function fakeParent(): Agent {
@@ -21,9 +22,21 @@ function fakeParent(): Agent {
vi.setConfig({ testTimeout: 30_000 })
/**
* `vi.waitFor` with a contention-proof default timeout: the 1s default flaked repeatedly on
* the CI coverage lane, where worker-thread cold start (CPU-bound — a fresh thread compiles
* the runtime) competes with three sibling vitest workers for CPU.
* `vi.waitFor` with a contention-proof default timeout: the 1s default
* flaked repeatedly on the CI coverage lane, where worker-thread cold start
* (CPU-bound — a fresh thread compiles the runtime) competes with three
* sibling vitest workers for CPU. The 10s default is for exactly those
* races — waiting for a worker to start, run its first script line, or
* deliver an async child-registration message to the host. It is NOT for a
* wait that asserts the HOST reacted PROMPTLY to something that already
* happened (a settled result, an observed worker death): those keep an
* explicit tight override below, or the generous default would silently
* accept a multi-second regression in host-side reap latency as passing
* (proven by injecting a 6s delay into one such reap and watching the
* un-overridden version of this helper still pass in ~6s).
* @param assertion - retried until it stops throwing or the timeout elapses.
* @param timeout - override for a wait that must stay deliberately tight.
* @returns resolves when the assertion passes.
*/
function waitFor(assertion: () => void, timeout = 10_000): Promise<void> {
return vi.waitFor(assertion, { timeout, interval: 50 })
@@ -35,9 +48,9 @@ const ESCAPE = "globalThis.constructor.constructor('return process')()"
/** One controllable child run: the test (or auto mode) settles it. */
interface ControlledRun {
request: SubagentStartRequest
/** Fulfill the provider publication/readiness boundary. */
/** Fulfill the provider's async start with a ready child. */
publish(): void
/** Reject the provider publication/readiness boundary. */
/** Reject the provider's async start before ownership transfer. */
rejectStart(error: unknown): void
settle(result: SubagentResult): void
rejectResult(error: unknown): void
@@ -62,15 +75,19 @@ class StubProvider implements SubagentProvider {
private readonly reply?: (request: SubagentStartRequest, index: number) => SubagentResult,
private readonly disposeDelayMs = 0,
private readonly deferStart = false,
private readonly onAbortString?: (reason: string | undefined, index: number) => void,
private readonly onSignalAbort?: (reason: unknown, index: number) => void,
) {}
start(request: SubagentStartRequest): SubagentRun {
const readiness = Promise.withResolvers<undefined>()
async start(request: SubagentStartRequest): Promise<SubagentRun> {
const startGate = Promise.withResolvers<undefined>()
const terminal = Promise.withResolvers<SubagentResult>()
terminal.promise.catch(() => { /* provider owns early settlement until publication */ })
let published = false
const controlled: ControlledRun = {
request,
publish: () => { readiness.resolve(undefined) },
rejectStart: (error) => { readiness.reject(error) },
publish: () => { published = true; startGate.resolve(undefined) },
rejectStart: (error) => { startGate.reject(error) },
settle: (result) => { terminal.resolve(result) },
rejectResult: (error) => { terminal.reject(error) },
cancelled: undefined,
@@ -79,20 +96,29 @@ class StubProvider implements SubagentProvider {
}
this.runs.push(controlled)
const index = this.runs.length - 1
request.signal?.addEventListener('abort', () => { terminal.resolve({ output: [], stopReason: 'aborted' }) }, { once: true })
if (!this.deferStart) readiness.resolve(undefined)
request.signal.addEventListener('abort', () => {
controlled.cancelled = String(request.signal.reason ?? 'cancelled')
this.onAbortString?.(String(request.signal.reason ?? 'cancelled'), index)
this.onSignalAbort?.(request.signal.reason, index)
if (published) terminal.resolve({ output: [], stopReason: 'aborted' })
else startGate.reject(new Error('child start aborted before publication'))
}, { once: true })
if (!this.deferStart) controlled.publish()
if (this.reply) {
const reply = this.reply
queueMicrotask(() => { terminal.resolve(reply(request, index)) })
}
try {
await startGate.promise
} catch (error: unknown) {
controlled.disposeCalls += 1
controlled.disposed = true
throw error
}
if (request.signal.aborted) throw new Error('child start aborted before publication')
return {
id: AgentId(`stub-child-${index}`),
started: readiness.promise,
result: terminal.promise,
cancel: (reason?: string) => {
controlled.cancelled = reason ?? 'cancelled'
terminal.resolve({ output: [], stopReason: 'aborted' })
},
dispose: () => {
controlled.disposeCalls += 1
if (this.disposeDelayMs === 0) {
@@ -121,6 +147,8 @@ interface SetupOptions {
manual?: boolean
disposeDelayMs?: number
deferStart?: boolean
onChildAbortString?: (reason: string | undefined, index: number) => void
onChildSignalAbort?: (reason: unknown, index: number) => void
}
async function setup(options?: SetupOptions) {
@@ -131,13 +159,15 @@ async function setup(options?: SetupOptions) {
options?.manual ? undefined : options?.reply ?? (() => text('stub reply')),
options?.disposeDelayMs ?? 0,
options?.deferStart ?? false,
options?.onChildAbortString,
options?.onChildSignalAbort,
)
ctx.subagents.registerProvider(provider)
// A fixed concurrency ceiling: the auto-resolved default is machine-derived
// (cores - 2, floored at 1), so tests that expect N children in flight
// would wedge on small CI runners.
await ctx.plugin(WorkerWorkflowEngine, { provider: 'stub', maxConcurrentAgents: 8, ...options?.config })
return { ctx, provider, parent: fakeParent() }
const engineFiber = await ctx.plugin(WorkerWorkflowEngine, { provider: 'stub', maxConcurrentAgents: 8, ...options?.config })
return { ctx, provider, parent: fakeParent(), engineFiber }
}
/** The standard test meta plus a body, spread into a start request. */
@@ -220,7 +250,7 @@ describe('dsh-workflow-workerthread', () => {
expect(result.error).toContain('agent() could not start a child')
})
it('waits for child readiness before announcing it and snapshots a result that settled early', async () => {
it('waits for async provider start before announcing a result that settled early', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const order: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => { order.push(`start:${agent.seq}`) })
@@ -231,10 +261,8 @@ describe('dsh-workflow-workerthread', () => {
await waitFor(() => { expect(provider.runs.length).toBe(1) })
const early = text('accepted value')
provider.runs[0]!.settle(early)
// Let the host observe + snapshot result while readiness remains pending.
// The provider still owns this early result while start is pending.
await new Promise(resolve => setTimeout(resolve, 0))
const earlyText = early.output[0] as { type: 'text'; text: string }
earlyText.text = 'mutated after settlement'
expect(order).toEqual([])
provider.runs[0]!.publish()
@@ -245,7 +273,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('observes an early result rejection but sends ChildStarted before ChildFailed after readiness', async () => {
it('observes an early result rejection but sends ChildStarted before ChildFailed after start fulfills', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -276,7 +304,7 @@ describe('dsh-workflow-workerthread', () => {
await handle.dispose()
})
it('classifies readiness rejection as AGENT_START, drops an early result, and emits no false lifecycle pair', async () => {
it('classifies provider start rejection as AGENT_START, drops an early result, and emits no false lifecycle pair', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -288,7 +316,7 @@ describe('dsh-workflow-workerthread', () => {
})
await waitFor(() => { expect(provider.runs.length).toBe(1) })
// ACP-style failure can settle result(error) before its session/publication
// boundary rejects. Readiness must dominate that buffered child outcome.
// boundary rejects. Start rejection must dominate that buffered child outcome.
provider.runs[0]!.settle({ output: [], stopReason: 'error' })
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('publication rolled back'))
@@ -305,7 +333,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('cancels and disposes a readiness-pending child once without publishing workflow lifecycle', async () => {
it('aborts a pending provider start once without publishing workflow lifecycle', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true, config: { disposeGraceMs: 500 } })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -319,7 +347,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposed).toBe(true)
})
// Ensure the host-driven disposal removed the registry entry before the
// late readiness rejection; its callback must not invoke dispose again.
// late start rejection; its callback must not invoke dispose again.
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('cancelled before publication'))
@@ -337,11 +365,9 @@ describe('dsh-workflow-workerthread', () => {
name: 'rejecting',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('reject-child'),
started: Promise.resolve(),
result: Promise.reject(new Error('backend exploded')),
cancel: () => { /* nothing in flight */ },
dispose: () => Promise.resolve(),
}),
}
@@ -354,15 +380,39 @@ describe('dsh-workflow-workerthread', () => {
expect((result.value as { message: string }).message).toContain('backend exploded')
})
it('maps an uncloneable ready-child result to fatal AGENT_RESULT instead of wedging the bridge', async () => {
it('maps a non-JSON ready-child result to fatal AGENT_RESULT instead of wedging the bridge', async () => {
const { ctx, parent } = await setup({
reply: () => ({ output: [], structured: () => { /* deliberately not cloneable */ }, stopReason: 'completed' }),
reply: () => ({ output: [], structured: () => { /* deliberately outside lossless JSON */ }, stopReason: 'completed' }),
})
const result = await run(ctx, parent, scripted(`
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }
`))
expect(result.value).toMatchObject({ code: 'AGENT_RESULT' })
expect((result.value as { message: string }).message).toContain('could not cross the worker boundary')
expect((result.value as { message: string }).message).toContain('workflow child result could not cross the worker boundary')
})
it('contains a non-JSON result even if the injected subagent service violates its normalization contract', async () => {
// The real worker boundary must reject a non-JSON same-process result.
const { ctx, parent } = await setup()
const invalid = {
output: [],
structured: () => { /* deliberately outside lossless JSON */ },
stopReason: 'completed',
} as unknown as SubagentResult
const start = vi.spyOn(ctx.subagents, 'start').mockResolvedValue({
id: AgentId('raw-invalid-child'),
result: Promise.resolve(invalid),
dispose: () => Promise.resolve(),
})
const result = await run(ctx, parent, scripted(`
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }
`))
expect(start).toHaveBeenCalledOnce()
expect(result.value).toMatchObject({ code: 'AGENT_RESULT' })
expect((result.value as { message: string }).message)
.toContain('workflow child result could not cross the worker boundary')
})
it('a child whose dispose() throws synchronously cannot wedge the script (the host acks anyway)', async () => {
@@ -372,9 +422,8 @@ describe('dsh-workflow-workerthread', () => {
name: 'bad-dispose',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('bad-dispose-child'),
started: Promise.resolve(),
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
cancel: () => { /* settled already */ },
dispose: () => { throw new Error('dispose exploded') },
@@ -394,9 +443,8 @@ describe('dsh-workflow-workerthread', () => {
name: 'coercion-trap-dispose',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('trap-child'),
started: Promise.resolve(),
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
cancel: () => { /* settled already */ },
// The rejection VALUE's own coercion throws: a warn built with bare
@@ -520,11 +568,48 @@ describe('dsh-workflow-workerthread', () => {
await second.dispose()
})
it('removes the exact external abort callback on first settlement or teardown', async () => {
const { ctx, parent } = await setup()
const settledController = new AbortController()
const settledAdd = vi.spyOn(settledController.signal, 'addEventListener')
const settledRemove = vi.spyOn(settledController.signal, 'removeEventListener')
const completed = ctx.workflows.start({ ...scripted('return 123'), parent, signal: settledController.signal })
const settledAbort = settledAdd.mock.calls.find(([type]) => type === 'abort')?.[1]
expect(typeof settledAbort).toBe('function')
await expect(completed.result).resolves.toMatchObject({ value: 123, stopReason: 'completed' })
expect(settledRemove).toHaveBeenCalledWith('abort', settledAbort)
const cancelAfterSettle = vi.spyOn(completed, 'cancel')
settledController.abort()
expect(cancelAfterSettle).not.toHaveBeenCalled()
cancelAfterSettle.mockRestore()
await completed.dispose()
const manual = await setup({ manual: true })
const teardownController = new AbortController()
const teardownAdd = vi.spyOn(teardownController.signal, 'addEventListener')
const teardownRemove = vi.spyOn(teardownController.signal, 'removeEventListener')
const tornDown = manual.ctx.workflows.start({
...scripted("return await agent('job')"),
parent: manual.parent,
signal: teardownController.signal,
})
await waitFor(() => { expect(manual.provider.runs).toHaveLength(1) })
const teardownAbort = teardownAdd.mock.calls.find(([type]) => type === 'abort')?.[1]
expect(typeof teardownAbort).toBe('function')
const disposing = tornDown.dispose()
expect(teardownRemove).toHaveBeenCalledWith('abort', teardownAbort)
await disposing
})
it('a child-start racing the host cancel is refused: no child starts after cancellation', async () => {
const { ctx, parent, provider } = await setup({ manual: true })
// Cancel from inside the log listener: the worker has already posted its child-start
// (queued right behind the log message), so the host processes it with cancelReason set —
// the refusal arm no real-world timing can hit reliably.
// Cancel from INSIDE the log listener: the worker has already posted
// its child-start (queued right behind the log message), so the host
// processes it with cancelReason set — the refusal arm no real-world
// timing can hit reliably. (The closure runs only after `handle` below
// is initialized — the listener fires on the worker's first message.)
ctx.on('workflow/log', () => { handle.cancel('cancelled from the log listener') })
const handle = ctx.workflows.start({ ...scripted("log('mark')\nreturn await agent('late')"), parent })
const result = await handle.result
@@ -539,9 +624,12 @@ describe('dsh-workflow-workerthread', () => {
ctx.on('workflow/log', (_info, message) => { narration.push(message) })
ctx.on('workflow/phase', (_info, title) => { narration.push(`phase:${title}`) })
const handle = ctx.workflows.start({
// The sync spin keeps the worker's loop busy so the cancel message cannot be processed
// before the script settles `completed` — the worker posts a completed result that must
// LOSE to the in-flight host cancellation.
// The sync spin keeps the worker's loop busy so the cancel message
// cannot be processed before the script settles `completed` — the
// worker posts a completed result that must LOSE to the in-flight
// host cancellation. The trailing narration exercises host-side
// suppression: posted pre-cancel-processing worker-side, arriving
// post-cancel host-side.
...scripted(`
log('started')
const end = Date.now() + 1000
@@ -639,6 +727,34 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposed).toBe(true)
})
it('result settlement reaps a registered stray even when the worker cannot relay disposal', async () => {
const { ctx, parent, provider } = await setup({
manual: true,
config: { provider: 'stub', disposeGraceMs: 30_000 },
})
const handle = ctx.workflows.start({
...scripted("agent('stray')\nawait new Promise(() => {})"),
parent,
})
await waitFor(() => { expect(provider.runs).toHaveLength(1) })
// Claim the host result while the real worker remains wedged, so it can
// send neither ChildDispose nor an exit. This leaves the accepted child
// in the host registry when public disposal begins.
const worker = (handle as unknown as { worker: Worker }).worker
worker.emit('message', {
type: WorkerToHostType.Result,
result: { value: 'synthetic completion', stopReason: 'completed', agentsStarted: 1 },
})
await expect(handle.result).resolves.toMatchObject({ stopReason: 'completed' })
await waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) }, 1000)
const disposal = handle.dispose()
await disposal
expect(provider.runs[0]!.disposeCalls).toBe(1)
await ctx.fiber.dispose()
})
it('the settle-reap fires the request signal too: a provider honoring ONLY the signal winds its stray down promptly', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
@@ -647,21 +763,16 @@ describe('dsh-workflow-workerthread', () => {
name: 'signal-only',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: (request) => {
start: async (request) => {
let settle!: (result: SubagentResult) => void
const result = new Promise<SubagentResult>((resolve) => { settle = resolve })
request.signal?.addEventListener('abort', () => {
aborted.push(String(request.signal?.reason))
request.signal.addEventListener('abort', () => {
aborted.push(String(request.signal.reason))
settle({ output: [], stopReason: 'aborted' })
}, { once: true })
return {
id: AgentId('signal-only-child'),
started: Promise.resolve(),
result,
// The seam leaves a provider free to honor EITHER cancel channel;
// this one deliberately ignores run.cancel() — only the request
// signal can wind it down.
cancel: () => { /* signal-only by design */ },
dispose: () => Promise.resolve(),
}
},
@@ -677,59 +788,104 @@ describe('dsh-workflow-workerthread', () => {
})
const result = await handle.result
expect(result.stopReason).toBe('completed')
// before dispose(): the settlement itself must have aborted the signal — without it this
// child would stay live until dispose's terminate.
// BEFORE dispose(): the settlement itself must have aborted the signal —
// without it this child would stay live until dispose's terminate. This
// is a HOST-PROMPTNESS claim, not a cold-start race — a tight explicit
// bound (unlike the file default) so a multi-second reap regression
// cannot pass by outlasting the wait.
await waitFor(() => { expect(aborted).toEqual(['workflow settled']) }, 1000)
await handle.dispose()
})
it("cancel() drives each child's explicit cancel() host-side: a wedged worker cannot delay it", async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
let starts = 0
const cancelled: string[] = []
const provider: SubagentProvider = {
name: 'cancel-only',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => {
starts += 1
return {
id: AgentId('cancel-only-child'),
started: Promise.resolve(),
result: new Promise(() => { /* only cancel() ends this child */ }),
// Deliberately ignores the request signal — the seam leaves a
// provider free to honor ONLY the explicit cancel() channel.
cancel: (reason?: string) => { cancelled.push(reason ?? 'cancelled') },
dispose: () => Promise.resolve(),
}
},
}
ctx.subagents.registerProvider(provider)
// A deliberately huge grace: if only the grace/terminate reap could
// reach this child, the assertion below would time out first.
await ctx.plugin(WorkerWorkflowEngine, { provider: 'cancel-only', maxConcurrentAgents: 2, disposeGraceMs: 30_000 })
it('the settle-reap aborts a pending provider start before workflow/end', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const childLifecycle: string[] = []
let cancellationAtWorkflowEnd: string | undefined
ctx.on('workflow/agent-start', () => { childLifecycle.push('start') })
ctx.on('workflow/agent-end', () => { childLifecycle.push('end') })
ctx.on('workflow/end', () => {
cancellationAtWorkflowEnd = provider.runs[0]?.cancelled
})
const handle = ctx.workflows.start({
// The stray child's start RPC reaches the host, then the script wedges its own worker
// in a synchronous spin: the worker cannot process the Cancel message, so it can relay
// NO ChildCancel RPC — only the host's own children loop can deliver the explicit
// cancel in time.
...scripted(`
agent('wedged child')
agent('start-pending stray')
return 'done'
`),
parent,
})
const result = await handle.result
expect(result.stopReason).toBe('completed')
expect(provider.runs).toHaveLength(1)
expect(provider.runs[0]!.request.signal?.aborted).toBe(true)
expect(provider.runs[0]!.request.signal?.reason).toBe('workflow settled')
expect(provider.runs[0]!.cancelled).toBe('workflow settled')
expect(cancellationAtWorkflowEnd).toBe('workflow settled')
expect(childLifecycle).toEqual([])
await handle.dispose()
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('a duplicate Result after the terminal claim cannot repeat cleanup or rewrite the outcome', async () => {
let signalAborts = 0
const { ctx, parent, provider } = await setup({
manual: true,
onChildAbortString: (_reason, index) => { if (index === 0) signalAborts += 1 },
})
const handle = ctx.workflows.start({
...scripted("agent('stray')\nawait new Promise(() => {})"),
parent,
})
await waitFor(() => { expect(provider.runs).toHaveLength(1) })
const worker = (handle as unknown as { worker: Worker }).worker
worker.emit('message', {
type: WorkerToHostType.Result,
result: { value: 'first', stopReason: 'completed', agentsStarted: 1 },
})
worker.emit('message', {
type: WorkerToHostType.Result,
result: { value: 'late', stopReason: 'completed', agentsStarted: 1 },
})
await expect(handle.result).resolves.toMatchObject({ value: 'first', stopReason: 'completed' })
expect(signalAborts).toBe(1)
await handle.dispose()
expect(signalAborts).toBe(1)
await ctx.fiber.dispose()
})
it('a grace-terminated worker reaps its child on exit without waiting for consumer dispose()', async () => {
const { ctx, parent, provider } = await setup({
manual: true,
config: { provider: 'stub', maxConcurrentAgents: 2, disposeGraceMs: 100 },
})
const handle = ctx.workflows.start({
// Let child-start cross, then make the worker unable to process its
// Cancel message. Grace settles the result and terminates the thread;
// that exit must independently own the host registry's disposal pass.
...scripted(`
agent('survives until exit reap')
for (let i = 0; i < 20; i++) await null
const end = Date.now() + 1500
while (Date.now() < end) {}
return 'raced'
return 'unreachable'
`),
parent: fakeParent(),
parent,
})
await waitFor(() => { expect(starts).toBe(1) })
handle.cancel('stop now')
await waitFor(() => { expect(cancelled).toEqual(['stop now']) }, 800)
// The wedged worker's own completion loses to the in-flight cancel.
await waitFor(() => { expect(provider.runs).toHaveLength(1) })
handle.cancel('force termination')
const result = await handle.result
expect(result.stopReason).toBe('cancelled')
// Deliberately assert before handle.dispose(): host-owned worker exit,
// not consumer courtesy, is responsible for this resource guarantee.
await waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) }, 1000)
expect(provider.runs[0]!.disposeCalls).toBe(1)
await handle.dispose()
await ctx.fiber.dispose()
}, 15_000)
it('dispose() on a wedged worker host-drives child disposal inside the grace: it returns with the children DISPOSED, not with their teardown still in flight', async () => {
@@ -739,7 +895,11 @@ describe('dsh-workflow-workerthread', () => {
config: { provider: 'stub', maxConcurrentAgents: 8, disposeGraceMs: 400 },
})
const handle = ctx.workflows.start({
// A wedged worker leaves host disposal as the only path to child quiescence.
// Same shape as the wedged-cancel test above: the child's start RPC
// reaches the host, then the script seizes its worker's loop, so the
// worker can relay NO dispose RPC — the host's own dispose() drive is
// the only thing that can start (and finish) this child's disposal
// before the grace runs out.
...scripted(`
agent('wedged child')
for (let i = 0; i < 20; i++) await null
@@ -855,23 +1015,124 @@ describe('dsh-workflow-workerthread', () => {
})
describe('worker death', () => {
it('the first death signal closes admission to messages Node delivers before exit', async () => {
const { ctx, parent, provider } = await setup({ manual: true })
const phases: string[] = []
ctx.on('workflow/phase', (_info, title) => { phases.push(title) })
const handle = ctx.workflows.start({
...scripted('await new Promise(() => {})'),
parent,
})
const worker = (handle as unknown as { worker: Worker }).worker
// Node may physically emit error -> queued message -> exit. Reproduce
// that ordering deterministically at the Worker event boundary: the
// late protocol data must not create work, narrate, or rewrite error.
worker.emit('error', new Error('synthetic error-before-message'))
worker.emit('message', { type: WorkerToHostType.Phase, title: 'late phase' })
worker.emit('message', {
type: WorkerToHostType.ChildStart,
callId: 999,
request: { prompt: 'late child' },
})
worker.emit('message', {
type: WorkerToHostType.Result,
result: { value: 'late', stopReason: 'completed', agentsStarted: 1 },
})
const result = await handle.result
expect(result.stopReason).toBe('error')
expect(result.error).toContain('synthetic error-before-message')
expect(provider.runs).toHaveLength(0)
expect(phases).toEqual([])
await handle.dispose()
await ctx.fiber.dispose()
})
it('refuses and disposes a provider run that becomes ready after its real worker dies', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const requested = Promise.withResolvers<SubagentStartRequest>()
const ready = Promise.withResolvers<SubagentRun>()
let disposeCalls = 0
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => ctx.logger)
const provider: SubagentProvider = {
name: 'late-ready',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: (request) => {
requested.resolve(request)
// Model a backend whose independent startup boundary cannot be
// interrupted promptly. The host must still reject ownership if the
// worker dies before this promise transfers the ready run.
return ready.promise
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, { provider: 'late-ready', maxConcurrentAgents: 1 })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', () => { lifecycle.push('end') })
const handle = ctx.workflows.start({
...scripted("return await agent('pending startup')"),
parent: fakeParent(),
})
const request = await requested.promise
const worker = (handle as unknown as { worker: Worker }).worker
// Kill the actual Worker while provider startup is independently
// pending. Death closes admission and aborts the shared signal, but this
// deliberately uncooperative provider still fulfills afterward.
await worker.terminate()
const result = await handle.result
expect(result.stopReason).toBe('error')
expect(result.error).toContain('exit code')
expect(request.signal.aborted).toBe(true)
expect(request.signal.reason).toBe('workflow worker gone')
ready.resolve({
id: AgentId('late-ready-child'),
result: Promise.resolve({ output: [], stopReason: 'aborted' }),
dispose: () => {
disposeCalls += 1
return Promise.reject(new Error('late ready dispose failed'))
},
})
await waitFor(() => {
expect(disposeCalls).toBe(1)
expect(warn).toHaveBeenCalledWith(expect.stringContaining('refused child dispose failed: Error: late ready dispose failed'))
}, 1000)
expect(lifecycle).toEqual([])
await handle.dispose()
expect(disposeCalls).toBe(1)
await ctx.fiber.dispose()
})
it('a worker that exits before settling reports an error result and reaps its children', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
// The child's dispose() REJECTS on top of the worker death: the reap
// must contain it (warn, not crash) while still emptying the registry.
const cancelled: string[] = []
const signalAborts: unknown[] = []
const provider: SubagentProvider = {
name: 'doomed',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
id: AgentId('doomed-child'),
started: Promise.resolve(),
result: new Promise(() => { /* never settles; the reap is the teardown */ }),
cancel: (reason?: string) => { cancelled.push(reason ?? 'cancelled') },
dispose: () => Promise.reject(new Error('dispose exploded during reap')),
}),
start: async (request) => {
request.signal.addEventListener('abort', () => {
signalAborts.push(request.signal.reason)
// The death claim precedes the shared-signal fanout. This
// synchronous callback cannot turn death into cancellation.
handle.cancel('reentered from worker-death signal cleanup')
}, { once: true })
return {
id: AgentId('doomed-child'),
result: new Promise(() => { /* never settles; the reap is the teardown */ }),
dispose: () => Promise.reject(new Error('dispose exploded during reap')),
}
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, { provider: 'doomed', maxConcurrentAgents: 2 })
@@ -899,7 +1160,11 @@ describe('dsh-workflow-workerthread', () => {
expect(runEnds).toEqual([{ stopReason: 'error', error: result.error, agentsStarted: 1 }])
// Result already settled — this is the reap's promptness, not a
// cold-start race; tight explicit bound (see the helper's doc comment).
await waitFor(() => { expect(cancelled.length).toBe(1) }, 1000)
await waitFor(() => {
expect(signalAborts).toEqual(['workflow worker gone'])
}, 1000)
await Promise.resolve()
expect(result.stopReason).toBe('error')
await handle.dispose()
}, 15_000)
@@ -940,9 +1205,10 @@ describe('dsh-workflow-workerthread', () => {
})
ctx.on('workflow/end', () => { order.push('run-end') })
const handle = ctx.workflows.start({
// Same choreography as the force-settle pairing test, but the worker DIES (the
// documented vm escape) instead of being terminated: the exit path must close slow's
// pair from the ledger too.
// Same choreography as the force-settle pairing test, but the worker
// DIES (the documented vm escape) instead of being terminated: the
// exit path must close slow's pair from the ledger too. The escaped
// setTimeout lets the already-posted messages flush before the kill.
...scripted(`
const p = agent('slow')
await agent('fast')
@@ -1021,33 +1287,57 @@ describe('dsh-workflow-workerthread', () => {
})
describe('service surface', () => {
it('run ids are unique per start; the run handle and event payloads hold SEPARATE meta clones', async () => {
it('run ids are unique and lifecycle meta is the run\'s borrowed immutable value', async () => {
const { ctx, parent } = await setup()
let eventMeta: WorkflowRunInfo | undefined
ctx.on('workflow/start', (info) => { eventMeta = info })
const first = ctx.workflows.start({ ...scripted('return 1'), parent })
const second = ctx.workflows.start({ ...scripted('return 2'), parent })
expect(first.id).not.toBe(second.id)
eventMeta!.meta.name = 'corrupted'
expect(eventMeta!.meta).toBe(second.meta)
expect(second.meta.name).toBe('test-flow')
await Promise.all([first.result, second.result])
await first.dispose()
await second.dispose()
})
it('unregisters ctx.workflows when the engine fiber is disposed (HMR safety), and default config runs (auto concurrency)', async () => {
it('unregisters ctx.workflows when the engine fiber is disposed (HMR safety)', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const fiber = await ctx.plugin(WorkerWorkflowEngine, {})
expect(ctx.get('workflows')).toBeDefined()
// A zero-agent run through the DEFAULT config exercises the auto
// concurrency resolution (cores - 2, capped) in start().
const result = await run(ctx, fakeParent(), scripted('return 6 * 7'))
expect(result.value).toBe(42)
await fiber.dispose()
expect(ctx.get('workflows')).toBeUndefined()
})
it('keeps a holder-owned run usable when the engine unloads before its child starts', async () => {
const { ctx, parent, provider, engineFiber } = await setup({ reply: () => text('survived reload') })
let handle!: ReturnType<typeof ctx.workflows.start>
const holder = await ctx.plugin(Object.assign((inner: Context) => {
handle = inner.workflows.start({ ...scripted("return await agent('after reload')"), parent })
}, { inject: ['workflows'] }))
try {
// A real worker cannot deliver child-start in the synchronous start()
// slice. Unload the provider before that message arrives: the returned
// run belongs to `holder`, not to the engine fiber being reloaded.
expect(provider.runs).toHaveLength(0)
await engineFiber.dispose()
expect(ctx.get('workflows')).toBeUndefined()
await expect(handle.result).resolves.toEqual({
value: 'survived reload',
stopReason: 'completed',
agentsStarted: 1,
})
expect(provider.runs).toHaveLength(1)
} finally {
await handle.dispose()
await holder.dispose()
await ctx.fiber.dispose()
}
})
it('has the class-plugin export shape (default = the engine service class)', () => {
expect(workerEngineModule.default).toBe(WorkerWorkflowEngine)
const loader = Object.create(Loader.prototype) as Loader

3
packages/workflow/workflow-workerthread/tsconfig.json
View File

@@ -26,6 +26,9 @@
{
"path": "../../llm/llm"
},
{
"path": "../../core/session"
},
{
"path": "../../subagent/subagent"
},

46
packages/workflow/workflow/README.md
View File

@@ -1,29 +1,43 @@
# @deepseek-ai/dsh-workflow
The **workflow seam** (`ctx.workflows`): an abstract service defining WHAT a workflow engine does — execute a model-written orchestration script that fans out subagents — without saying HOW. The bash-shaped third of the [workflow family](../README.md): implementations subclass `WorkflowService` and register as the `workflows` service (one per context); [`dsh-workflow-workerthread`](../workflow-workerthread/README.md) (one worker thread per run) is the implementation, and [`dsh-tool-workflow`](../tool-workflow/README.md) is the model-facing consumer.
The workflow seam (`ctx.workflows`) executes a model-written orchestration script that can fan out subagents. The seam defines the script, run, result, error, and event contracts; an engine decides how to isolate and execute the script.
## Service: `WorkflowService` (abstract)
`@deepseek-ai/dsh-workflow-workerthread` is the current engine and `@deepseek-ai/dsh-tool-workflow` is the model-facing consumer. A future process or sandbox engine can replace the implementation without changing the tool.
`start(request: WorkflowStartRequest): WorkflowRun` — parse and execute a script. Throws synchronously (`SCRIPT_PARSE`/`META_INVALID`) for a script that cannot begin; once a run is returned, its `result` NEVER rejects — every failure resolves with `stopReason: 'error'` (or `'cancelled'`) — and once the run is cancelled, `result` settles within the implementation's bounded grace even if the script itself never settles (a consumer awaiting `result` must never be wedged past a cancellation). `dispose()` must reach quiescence within a bounded grace (cancel → wait for the script to settle and its children to finish disposing → abandon), never hanging its caller. Runs are HOLDER-owned: the engine does not track its live runs, so disposing the engine's fiber mid-run leaves each run to its holder's teardown.
## Service and run contract
The protected `emitWorkflowEvent` helper dispatches the `workflow/*` events with PER-LISTENER containment and PER-LISTENER payload snapshots (a throwing subscriber is logged, never propagated, and cannot starve later listeners; each subscriber gets its own clone of the payload, so mutating it corrupts neither the engine nor other listeners) — the same containment guarantee as the subagent seam's lifecycle emits.
`WorkflowService.start(request): WorkflowRun` validates enough synchronously to reject a malformed meta block or unparseable script before a run exists. Once returned, `WorkflowRun.result` never rejects: execution failures resolve with `stopReason: 'error'`, and cancellation resolves with `cancelled` within the engine's bounded grace.
## Vocabulary
A run is holder-owned. Engine-plugin unload prevents new starts but does not revoke accepted runs. The holder must call `dispose()` on every path; disposal cancels remaining work and reaches or abandons quiescence within the documented bound.
- `WorkflowStartRequest` — `{ script, args?, parent: Agent, signal? }`. `parent` is REQUIRED: every child the script spawns is attributed to it. `args` must be plain host-realm JSON data.
- `WorkflowMeta` / `WorkflowPhase` — the workflow's identity block, carried as plain JSON data on the start request (Claude Code meta vocabulary: required `name`/`description`, optional `whenToUse`/`phases`) and shape-validated by the engine.
- `WorkflowRun` — `{ id, meta, result, cancel(reason?), dispose() }`; the consumer awaits `result` and MUST `dispose` on every path.
- `WorkflowResult` — `{ value, stopReason: 'completed'|'cancelled'|'error', error?, agentsStarted }`; `value` is the script's materialized return (plain JSON data; `null` for no return).
- `WorkflowError` — `HarnessError` with a `WorkflowErrorCode` and a `fatal` flag driving the combinator discipline: a fatal error (bad hook arguments, unsupported options/schemas, tripped caps, seam start failures, cancellation) always propagates through `parallel()`/`pipeline()` instead of dissolving into a per-item `null`. `isFatalWorkflowError(error)` is the catch-site predicate.
`WorkflowStartRequest` contains `{ meta, script, args?, parent, signal? }`. `parent` attributes every child agent to the invoking agent. `meta` and `args` are plain data, not script fragments.
`WorkflowRun` exposes `{ id, meta, result, cancel(reason?), dispose() }`. `WorkflowResult` contains `{ value, stopReason, error?, agentsStarted }`; `value` is plain JSON data or `null`.
## Events
All observe-only emits carrying DATA SNAPSHOTS (`WorkflowRunInfo` = id + meta) — never the live `WorkflowRun`, so a listener cannot gain `cancel`/`dispose`; control stays with the `start()` caller:
Workflow events are observe-only. They carry `WorkflowRunInfo` (`id` plus `meta`) rather than the live run, so listeners cannot acquire cancellation or disposal authority.
- `workflow/start`(info) / `workflow/end`(info, resultInfo) — run lifecycle; `resultInfo` deliberately omits the value.
- `workflow/phase`(info, title) / `workflow/log`(info, message) — script narration.
- `workflow/agent-start`(info, agent) / `workflow/agent-end`(info, agent + outcome) — ready-child lifecycle correlated by `seq`; the [generated event contract](../../../docs/cordis-catalog/events.md#workflowagent-start--emit) defines publication and pairing.
- `workflow/start` / `workflow/end` pair the run.
- `workflow/phase` and `workflow/log` expose script narration.
- `workflow/agent-start` / `workflow/agent-end` pair each child call by `seq`; a child whose async provider start rejects emits neither.
## Non-goals (this cut)
Same-process event payloads are borrowed immutable values. Every listener is independently contained: a synchronous throw or rejected returned promise is logged without starving peers or changing execution.
Background collection, journaling/resume, saved workflows, nested `workflow()`, token budgets — see the [RFC's deferred section](../../../docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md).
## Failure discipline
`WorkflowError` carries a code and a `fatal` flag. Fatal errors always escape `parallel()` and `pipeline()` instead of becoming an ordinary per-item `null`:
- `SCRIPT_PARSE` / `META_INVALID` — the workflow cannot start.
- `INVALID_ARGUMENT` / `UNSUPPORTED_OPTION` / `UNSUPPORTED_SCHEMA` — a hook call violates the engine contract.
- `AGENT_CAP` / `ITEM_CAP` — configured safety limits were exceeded.
- `AGENT_START` — the provider's async start rejected.
- `AGENT_RESULT` — a ready child's result rejected with an infrastructure fault.
- `RESULT_UNSERIALIZABLE` — a script/worker value is not plain JSON data.
- `CANCELLED` — cancellation owns the run and pending/future hooks reject.
A child that resolves normally with a non-completed stop reason is not an infrastructure exception: `agent()` returns `null`, allowing the script to handle an ordinary child failure.
## Non-goals
Background collection, journaling/resume, saved workflows, nested `workflow()`, and token budgets are deferred. See the [dynamic-workflows RFC](../../../docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md).

98
packages/workflow/workflow/src/index.ts
View File

@@ -1,7 +1,21 @@
/**
* The workflow capability seam (`ctx.workflows`): an abstract service defining what a workflow
* engine does — execute a model-written orchestration script that fans out subagents — without
* saying how.
* The workflow capability seam (`ctx.workflows`): an abstract service defining
* WHAT a workflow engine does — execute a model-written orchestration script
* that fans out subagents — without saying HOW. Implementations subclass
* {@link WorkflowService} and register as the `workflows` service (one
* implementation per context, cordis' standard duplicate-service behavior);
* the implementation is `@deepseek-ai/dsh-workflow-workerthread`, which runs each
* script in its own worker thread. Hardened engines (an isolated-vm or
* separate-process sandbox) swap in without touching the model-facing tool
* that consumes them (`@deepseek-ai/dsh-tool-workflow`).
*
* The `workflow/*` lifecycle events are OBSERVE-ONLY data: they
* carry {@link WorkflowRunInfo} (id + meta), never the live {@link WorkflowRun}
* — a listener must not gain `cancel`/`dispose`; control stays with the
* `start()` caller holding the run. Same-process payloads are borrowed
* immutable values. Every listener is independently contained, so a throw or
* rejected promise can neither strand a run nor starve peers.
*
* @module @deepseek-ai/dsh-workflow
*/
@@ -62,7 +76,7 @@ declare module 'cordis' {
/**
* One `agent()` call established a ready child run. Paired with
* {@link Events['workflow/agent-end']} by `agent.seq`. A call that never
* crosses the provider's publication/readiness boundary emits neither
* receives a ready run from the provider emits neither
* event in this pair.
* @param info - the run's identity snapshot.
* @param agent - the call's sequence number, label, phase, and child id.
@@ -103,9 +117,27 @@ export type WorkflowEventName =
| 'workflow/end'
/**
* The workflow-seam error codes. Every one of these is FATAL when it reaches a script (see
* {@link WorkflowError.fatal}): the combinators re-throw it instead of dissolving it into an
* ordinary per-item `null`.
* The workflow-seam error codes. Every one of these is FATAL when it reaches
* a script (see {@link WorkflowError.fatal}): the combinators re-throw it
* instead of dissolving it into an ordinary per-item `null`.
*
* - `SCRIPT_PARSE` — the script (or its meta statement) does not parse.
* - `META_INVALID` — the meta block evaluated but fails the shape contract.
* - `INVALID_ARGUMENT` — a hook was called with malformed arguments.
* - `UNSUPPORTED_OPTION` — an `agent()` option this engine does not support
* (deferred: `effort`/`isolation`/`agentType`) or does not know.
* - `UNSUPPORTED_SCHEMA` — an `agent()` schema outside the structured-output
* subset (see dsh-tools).
* - `AGENT_CAP` / `ITEM_CAP` — the run/agent caps tripped.
* - `AGENT_START` — the provider's asynchronous start rejected before
* cancellation took precedence.
* - `AGENT_RESULT` — a ready run had its `result` REJECT: an infrastructure
* fault at the subagent seam. This is distinct from a child that failed and resolved
* (which is the per-item `null`, never an error).
* - `RESULT_UNSERIALIZABLE` — a value crossing the script/host value boundary
* is not plain JSON data.
* - `CANCELLED` — the run was cancelled; pending and future hooks reject
* with this (the script-kill mechanism).
*/
export type WorkflowErrorCode =
| 'SCRIPT_PARSE'
@@ -150,9 +182,31 @@ export function isFatalWorkflowError(error: unknown): boolean {
}
/**
* Abstract workflow execution service. Subclass, implement {@link start}, and load the
* subclass as a plugin — it registers as `ctx.workflows` (one implementation per context;
* loading a second throws, cordis' standard duplicate-service behavior).
* Abstract workflow execution service. Subclass, implement {@link start}, and
* load the subclass as a plugin — it registers as `ctx.workflows` (one
* implementation per context; loading a second throws, cordis' standard
* duplicate-service behavior).
*
* Semantics every implementation must honor:
* - {@link start} throws synchronously for a request that cannot begin (an
* unparseable script, an invalid meta block). Once it returns a
* {@link WorkflowRun}, `result` NEVER rejects — every failure resolves with
* `stopReason: 'error'` (or `'cancelled'`) — and once the run is cancelled,
* `result` SETTLES within the implementation's bounded grace even if the
* script itself never settles (a consumer awaiting `result` must never be
* wedged past a cancellation).
* - The `workflow/*` events fire through {@link emitWorkflowEvent} (borrowed
* immutable data, per-listener containment); `workflow/end` fires exactly once
* per started run, after `result` is settled or as it settles.
* - `dispose()` reaches quiescence within a bounded grace: it cancels, waits
* for the script to settle AND its started children to finish disposing,
* and abandons whatever is left rather than hanging its caller (the engine
* documents what abandonment leaves behind).
* - Runs are HOLDER-OWNED: the engine hands control (`cancel`/`dispose`) to
* the `start()` caller and does not track its live runs — disposing the
* engine's own fiber mid-run deliberately leaves those runs to their
* holders' teardown, so an engine reload cannot yank a run out from under
* the consumer awaiting it.
*/
export abstract class WorkflowService extends Service {
constructor(ctx: Context) {
@@ -168,14 +222,24 @@ export abstract class WorkflowService extends Service {
abstract start(request: WorkflowStartRequest): WorkflowRun
/**
* Emit isolated payload snapshots and contain each lifecycle listener independently.
* Emit one `workflow/*` lifecycle event with per-listener containment. Each
* subscriber receives the same borrowed immutable payload; a throw or
* asynchronously rejected listener is logged (never propagated — the logging
* itself is total, even for a thrown value whose own string coercion
* throws), so one bad subscriber can neither fail the engine mid-run,
* surface as an unhandled rejection on a detached settle hook, nor starve
* the listeners registered after it (cordis `emit` halts on the first throw
* — same guarantee as the subagent seam's lifecycle emits).
* @param name - the `workflow/*` event to dispatch.
* @param args - the event's payload, matching its declared signature.
*/
protected emitWorkflowEvent(name: WorkflowEventName, ...args: unknown[]): void {
for (const callback of this.ctx.events.dispatch('emit', [name, ...args])) {
try {
;(callback as (...payload: unknown[]) => void)(...structuredClone(args))
const returned: unknown = (callback as (...payload: unknown[]) => unknown)(...args)
void Promise.resolve(returned).catch((error: unknown) => {
this.ctx.logger.warn(`workflow: ${name} listener rejected: ${renderListenerError(error)}`)
})
} catch (error: unknown) {
this.ctx.logger.warn(`workflow: ${name} listener threw: ${renderListenerError(error)}`)
}
@@ -184,15 +248,19 @@ export abstract class WorkflowService extends Service {
}
/**
* Render a thrown value without weakening listener containment.
* Total renderer for a listener-thrown value: the containment catch must never
* itself throw, and `String(error)` does when the value's own `toString` /
* `Symbol.toPrimitive` throws. Local rather than an engine package's renderer
* — the seam sits below every engine and cannot import one.
* @param error - any thrown value.
* @returns string form or a fixed fallback when coercion throws.
* @returns `String(error)`, or a fixed label when even coercion throws.
*/
function renderListenerError(error: unknown): string {
try {
return String(error)
} catch {
// String coercion itself is untrusted.
// Only a throwing toString/Symbol.toPrimitive lands here; the fixed label
// keeps the containment guarantee total.
return '[unrenderable thrown value]'
}
}

2
packages/workflow/workflow/src/types.ts
View File

@@ -119,7 +119,7 @@ export interface WorkflowRun {
dispose(): Promise<void>
}
/** Identifying detail for a run, carried by every `workflow/*` event (a data snapshot, never the live run). */
/** Identifying detail for a run, carried by every `workflow/*` event as borrowed immutable data, never the live run. */
export interface WorkflowRunInfo {
/** The run's id. */
id: WorkflowRunId

25
packages/workflow/workflow/tests/workflow.spec.ts
View File

@@ -66,26 +66,21 @@ describe('dsh-workflow (interface)', () => {
])
})
it('gives each listener its OWN payload snapshot: mutation corrupts neither peers nor the caller', async () => {
it('contains an asynchronously rejected listener without starving peers', async () => {
const ctx = new Context()
await ctx.plugin(StubEngine)
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => ctx.logger)
const seen: string[] = []
ctx.on('workflow/agent-start', (info, agent) => {
agent.label = 'HACKED'
info.meta.name = 'HACKED'
seen.push('mutator')
})
ctx.on('workflow/agent-start', (info, agent) => {
seen.push(`${info.meta.name}/${agent.label}`)
})
// Runtime listeners may return thenables even though the declaration's observable result is void.
// eslint-disable-next-line @typescript-eslint/no-misused-promises -- exercises rejected-listener containment
ctx.on('workflow/agent-start', async () => { throw new Error('async observer failed') })
ctx.on('workflow/agent-start', (_info, agent) => { seen.push(agent.label) })
const engine = ctx.workflows as StubEngine
const info: WorkflowRunInfo = { id: WorkflowRunId('run-2'), meta: { name: 'w', description: 'd' } }
const payload = { seq: 1, label: 'original', childId: 'c' }
engine.emit('workflow/agent-start', info, payload)
expect(seen).toEqual(['mutator', 'w/original'])
// The caller's own objects are pristine too — no listener ever saw them.
expect(info.meta.name).toBe('w')
expect(payload.label).toBe('original')
engine.emit('workflow/agent-start', INFO, payload)
await Promise.resolve()
expect(seen).toEqual(['original'])
expect(String(warn.mock.calls[0]![0])).toContain('listener rejected')
})
it('contains a throwing listener PER LISTENER: later listeners still run, nothing propagates', async () => {