Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(subagent): unify async readiness and cancellation

Browse Source
This commit is contained in:
Tianyi Cui
2026-07-12 22:41:59 +08:00
parent 02ca71db57
commit bb3f6bd736
49 changed files with 1350 additions and 4147 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
packages/workflow/workflow-workerthread/README.md
View File

@@ -1,59 +1,80 @@
# @deepseek-ai/dsh-workflow-workerthread
The [`WorkflowService`](../workflow/README.md) implementation, on **`node:worker_threads`**: each run gets its OWN worker thread (one run = one worker, no pooling — a run is heavyweight, so the ~tens-of-ms thread spin-up is noise), the script executes in a vm context INSIDE that worker with the workflow hooks injected, and every `agent()` call bridges back over the message port to [`ctx.subagents`](../../subagent/README.md) on the host. Child agents are I/O-bound LLM loops and stay on the host event loop; the thread isolates the SCRIPT, the only part that can spin synchronously.
This package implements `WorkflowService` with one Node worker thread per run. The worker executes the orchestration script; child agents remain on the host and are reached through `ctx.subagents` over a typed host/worker protocol.
## Trust premise: what the thread buys (and what it does not)
The split has one primary purpose: a synchronous script loop cannot block the harness event loop, and a script that ignores cancellation can be terminated with its worker. It is not a security sandbox.
Workflow scripts are **model-written** — the same trust level as the model's existing bash access — so this engine defends against **buggy** scripts, never hostile ones. A worker thread is NOT a security boundary: the vm context inside it is escapable by construction (`node:vm` shares object machinery with its surrounding realm, so a script can reach the `Function` constructor via `globalThis.constructor.constructor` and from it `process` and every Node builtin), and an escapee holds the same process privileges as the host — Node's permission model is process-wide. The absent globals are API surface that keeps honest scripts portable, not walls. What the thread concretely buys:
## Trust and isolation boundary
- **The host never blocks**: `start()` returns without running any script code on the host; a synchronous spin anywhere in the script occupies the worker's loop, not the harness's.
- **Termination is real**: a script that outlives its post-cancel grace is `worker.terminate()`d — nothing of it survives `dispose()`, where an in-process engine could only abandon the spin on its own loop.
- **No ambient credentials**: the worker spawns with an EMPTY environment (`env: {}` plus hermetic `execArgv`, the same stance as `dsh-code-runtime-worker`; the unbuilt dev shape forwards exactly one loader variable, `TSX_TSCONFIG_PATH` — path plumbing, not a secret), so an escapee reading `process.env` finds no harness secrets — ambient-channel hardening only; the process-wide privileges above (fs and the rest) remain, so a genuine sandbox is still the engine swap.
- **Serialization by construction**: everything crossing the thread is structured-clone data, and plain JSON before that — the `materializeFromRealm` walk rejects loud what JSON cannot carry, which is also what makes every postMessage hop total.
Workflow scripts are model-written and have the same trust premise as the model's existing bash access. `node:vm` inside a worker is an API-shaping mechanism, not a security boundary: an escaped script can recover Node capabilities with the host process's privileges.
What the seam guarantees regardless, because benign scripts hit these constantly: `result` never rejects, a dropped hook promise never becomes an unhandled rejection, values JSON cannot carry are rejected **loud** instead of silently mangled, and hook misuse is fatal instead of dissolving into a per-item `null`. Genuine sandboxing (containing what an escaped script may touch) remains an isolated-vm/separate-process engine swap behind the seam, still deferred.
The worker still provides useful containment:
## The script contract it executes
- Script CPU work and synchronous spins stay off the host event loop.
- `worker.terminate()` gives disposal a real final stop.
- The worker starts with an empty environment, except unbuilt loader plumbing, so ambient credentials do not cross through `process.env`.
- Host/worker messages use structured-clone data, with plain-JSON validation at the script boundary.
- **Meta as DATA** (`validateMeta`, host-side): the workflow's identity arrives on the start request as plain JSON (the tool carries it as its schema-validated `meta` parameter — never as script text) and is shape-validated loud, every violation named (`name`/`description` required; unknown fields rejected). The engine deliberately evaluates NO script text to obtain meta: an evaluated meta literal could smuggle getters that run on the host outside any vm timeout — the exact spin the worker thread exists to isolate. A body that still opens with a Claude Code-style `export const meta` statement is rejected with a pointed `SCRIPT_PARSE` message.
- **Hooks**: `agent(prompt, {label, phase, schema, model})` (schema = the [structured-output subset](../../core/tools/README.md), forwarded as `outputSchema`; result = validated object, or final text without a schema; a failed child resolves `null`), `parallel(thunks)`, `pipeline(items, ...stages)` with NO cross-stage barrier and `(prev, item, index)` stage callbacks, `phase(title)`, `log(message)`, and the `args` global. Anything else — `effort`/`isolation`/`agentType`, unknown options, malformed arguments, schemas outside the subset — throws a FATAL `WorkflowError` that `parallel`/`pipeline` re-throw rather than nulling (see the seam README's failure discipline).
- **No ambient APIs**: no timers, filesystem, or Node APIs are injected into the context (absence is API surface, not containment — see the trust premise).
A genuinely untrusted-script sandbox would require a different engine behind the same workflow seam.
## How a run executes
## Script contract
`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (unbuilt: a JavaScript data-URL bootstrap registers tsx's ESM and CommonJS transforms inside the worker before importing `src/worker.ts`, giving the whole mixed-module source graph full TypeScript and tsconfig-path transformation on every supported Node line; built: the sibling `lib/worker.js` bundle) with the meta, body, `args`, and worker-side limits as `workerData`.
The workflow's `meta` is host-provided data, not evaluated script text. The engine validates its required `name` and `description`, rejects unknown fields, and parse-checks the body before returning a run.
Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**. `agent()` sends `child-start`, and the host starts the child through the holder-bound `SubagentService` handle captured synchronously by `start()`, with parent attribution, the shared per-run abort signal, and `outputSchema`/`model` pass-through. This capture is part of the seam's holder-owned lifetime: unloading the engine removes `ctx.workflows` for new calls but does not invalidate an already returned run whose worker starts another child afterward.
Inside the worker, the script receives `args` and these hooks:
The host observes `run.result` immediately but buffers its snapshotted wire projection until `run.started` fulfills. Provider `start()` is arbitrary code and can synchronously reenter workflow cancellation before its returned run reaches the host registry, so the host registers the run, attaches both promise observers, and re-checks admission after `start()` returns and again at readiness. A closed boundary never admits or announces the run to the worker: while the exact run remains registered, the host invokes explicit cancel once and disposes it; `child-start-error` is sent only while worker-message admission remains open. If the run was already retired, the identity guard sends no cleanup through the deleted call ID. An ordinary readiness rejection sends `child-start-error` while possible and disposes the provider attempt without adding an explicit cancellation. Otherwise the host replies `child-started` with the child id before forwarding settlement, so `workflow/agent-start` always names a ready child and precedes its end. The worker classifies a start error as fatal `AGENT_START` unless cancellation already owns the run. If readiness fulfills, an infrastructure result rejection crosses as `child-failed`/`AGENT_RESULT` regardless of whether that rejection settled before or after readiness. Child disposal acknowledgements complete the RPC.
- `agent(prompt, { label, phase, schema, model })` starts one host-side subagent. With a schema it returns the structured value; otherwise it returns final text. An ordinary failed child yields `null`.
- `parallel(thunks)` runs thunks under the configured concurrency limit.
- `pipeline(items, ...stages)` passes `(previous, item, index)` without a cross-stage barrier.
- `phase(title)` and `log(message)` emit observer narration.
Observer narration (`phase`/`log`/`agent-start`/`agent-end`) crosses as messages and re-emits as the seam's `workflow/*` events. A **ready→go handshake** gates the body: a cancellation racing worker boot arrives before `go`, so a run cancelled before start never executes the body at all.
Unknown options, malformed arguments, unsupported schemas, tripped caps, provider-start failures, and infrastructure result failures are fatal workflow errors. No timers, filesystem API, or Node globals are intentionally injected, though the trust caveat above still applies.
## The value boundary
## Run sequence
Values LEAVING the script (hook options/schemas, the script's return) are materialized by `materializeFromRealm`: a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested `undefined`), copying into plain containers via `defineProperty` so a `"__proto__"` key becomes a data property, never a prototype mutation. Getters are read ordinarily — the RESULT is what crosses; a read that throws fails loud; both run in the WORKER, never on the host. Values ENTERING the realm (`args`, `agent()` results, hook promises and their failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; `args` is cloned once at start so a script scribbling on it cannot mutate the caller's object. One script-visible consequence: an error thrown by a hook is built OUTSIDE the script's vm context, so `e instanceof Error` inside the script is `false` — branch on `e.name`/`e.code` instead (the combinators recognize fatality by `instanceof` against their own realm's class, which a script-built object can never pass, so fatal-vs-null cannot be forged or dissolved).
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
## Cancellation, death, disposal
For each `agent()` call:
Cancellation is bounded and host-driven. Per-run limits are a concurrency semaphore (`maxConcurrentAgents`), a total-`agent()` cap (`maxTotalAgents`), and a per-call item cap (`maxItemsPerCall`), all config. `cancel()` first records its reason, then posts to the worker (its hooks start throwing `CANCELLED`; the script dies at its next await) and cancels every host-side child NOW on **both seam channels**: the shared request signal aborts and each registered child's explicit `cancel()` runs host-side. The seam leaves a provider free to honor either channel, and a worker wedged in a synchronous spin could not relay its own per-child cancel RPCs. A host-side per-call gate turns the worker's later explicit-cancel relay into a no-op, because the seam does not require `SubagentRun.cancel()` to be idempotent. Each explicit child `cancel()` callback is exception-contained independently, post-cancel `phase`/`log` narration is suppressed host-side, and cancelled children still deliver paired `agent-end` events. The caller's optional start-signal callback is retained by exact identity only while the run is live and removed at the first settlement or teardown.
1. The worker sends `child-start` with a plain-data prompt and options.
2. The host calls the configured provider through async `SubagentService.start`, passing the workflow's parent and one canonical per-run abort signal.
3. If start rejects, the host sends `child-start-error`; provider startup has already reached quiescence and no child lifecycle event is emitted.
4. If start fulfills while the workflow still admits work, the host records the run, observes `result`, then sends `child-started`. Even an already-settled result is forwarded afterward, preserving start-before-result order.
5. The worker emits paired `workflow/agent-start` and `workflow/agent-end` narration and requests child disposal after collection.
Terminal arbitration is first-wins at explicit host-side claim points. A cancellation before ready→go reports `cancelled` without executing the body. For a later race, the worker queues Result before its settlement-reap `ChildCancel` messages; external `cancel()` records its reason before its fanout, while Result receipt snapshots any earlier cancellation and records the terminal outcome before settlement-cleanup fanout. Same-port FIFO and those claim points mean earlier caller/signal/dispose cancellation overrides a non-cancelled report, while an arrived report cannot be rewritten by a cleanup callback. Once Result has won, a losing reentrant `cancel()` has no state, message, child-fanout, or grace-timer effect. If no earlier terminal source settles the run, the grace callback claims `cancelled`, synthesizes missing lifecycle ends, settles the result, and terminates the worker after `disposeGraceMs`.
Provider starts are tracked separately from published children. If cancellation, worker death, or normal workflow settlement closes admission while a start is pending, the shared signal aborts it. A provider that nevertheless fulfills after closure is disposed by the host and never announced to the worker.
Worker death separates outcome ownership, message admission, and resource cleanup. An unexpected OOM, `error`, message failure, or premature exit claims `stopReason: 'error'` with diagnostics—or preserves an external cancellation already in flight—before reaping children or synthesizing observer events. Reentrant provider cancellation therefore cannot turn a death-first error into cancellation. The first death signal also closes worker-message admission because Node may deliver a queued `message` between `error` and `exit`; late protocol data cannot start a child, emit narration, or compete with the outcome. If Result or grace already claimed the outcome, death preserves it while still reaping promptly. The eventual `exit` then performs a final disposal-only sweep, joining any in-flight disposal without repeating explicit child cancellation. This separation lets grace settlement become observable before `worker.terminate()` reports exit without leaking the host-side registry.
## Value boundary
Disposal is the holder's bounded resource guarantee: cancel, begin host-driven disposal of every registered child immediately, wait for result plus child-registry quiescence up to the same grace, and unconditionally terminate the worker. `handle.dispose()` claims its public promise before that traversal invokes cancellation or disposal callbacks. Independently, every `disposeChild` path claims the call ID's promise before invoking the wrapped child disposer. Public-first reentry therefore returns the existing holder promise; worker-first reentry may begin holder disposal, whose child traversal joins the already-claimed call ID promise. Neither order can start a second provider disposal. A wedged worker can relay no dispose RPC, so host-driven teardown overlaps the grace; any later worker RPC joins the same per-child disposal. Before ordinary settlement becomes observable, the host also cancels every stray on both channels, including a fire-and-forget run still waiting on readiness. That work is settlement-only cleanup after the terminal claim, so provider reentry cannot rewrite the chosen result; `dispose()` then waits for its completion within the bound.
Values leaving the script pass through `materializeFromRealm`, which accepts plain, lossless JSON data and rejects exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, and nested `undefined`. The walk runs in the worker, and defines object keys as data properties so `__proto__` cannot mutate a prototype.
Lifecycle pairing is host-guaranteed independently of outcome arbitration. Forwarded starts live in a ledger and worker-reported ends pair them on graceful paths. When death or grace is the terminal source, the host synthesizes missing ends with outcome `cancelled` before `workflow/end`. If Result settled first, later death cleanup may synthesize a survivor's end afterward; a start already crossing force-settlement may likewise surface after `workflow/end`. The same ledger still pairs every forwarded start exactly once.
Child results are projected and snapshotted before crossing from the host to the worker. This is a real process-like serialization boundary; it is deliberately different from trusted same-process workflow and subagent event payloads, which are borrowed immutable values.
**Engine-specific limitations**: worker startup is paid per run; on a termination path `agentsStarted` reports the HOST-observed count (accepted `child-start`s — calls still queued worker-side for a concurrency slot are unknowable then); and a returned promise or thenable resolves per JavaScript semantics BEFORE materialization — that is what makes an un-awaited `return agent('x')` work — with the value-boundary guard applying to the resolution.
## Cancellation and disposal
`WorkflowRun.cancel()` records the first reason, tells the worker to cancel, aborts the one signal shared by every pending and published child, and arms the `disposeGraceMs` timer. Worker hooks then throw `CANCELLED` at their next await. If the run remains unsettled at the deadline, the host resolves it as cancelled, pairs stranded child lifecycle events, and terminates the worker.
The subagent seam has one cancellation channel: the request signal. There is no separate child-cancel RPC. Published child teardown uses `run.dispose()`; pending provider starts remain provider-owned until their promise rejects or fulfills.
Normal settlement also aborts pending starts and begins disposing any published fire-and-forget children before the result becomes externally settled. The host's quiescence condition includes both pending starts and published child disposals, so cleanup does not forget an async startup transaction.
`dispose()` is idempotent. It cancels the run, starts host-driven disposal immediately, waits for result plus child quiescence up to the same grace, terminates the worker unconditionally, and performs a final survivor sweep. Per-child disposal is memoized so worker RPC, host cancellation, death cleanup, and public disposal all join one operation.
## Outcome and event guarantees
Terminal outcome is first-wins at host claim points. An accepted external cancellation overrides a later non-cancelled worker result; a result or worker death that claims first cannot be rewritten by reentrant cleanup callbacks.
Worker error, message failure, or premature exit closes message admission before cleanup, then resolves `error` unless cancellation already owns the run. Late queued messages cannot create children or narrate after that logical boundary.
The host keeps a ledger of forwarded child starts. A graceful worker supplies their ends; death or force termination synthesizes any missing end as cancelled. Every forwarded `workflow/agent-start` is therefore paired exactly once, although cleanup after an already-arrived workflow result may complete afterward.
## Config
| Key | Default | Meaning |
|---|---|---|
| `provider` | `spawn` | The `ctx.subagents` provider children run on (host-side). |
| `maxConcurrentAgents` | `0` (auto) | Concurrent `agent()` ceiling; `0` resolves to `min(16, max(1, cores - 2))`. |
| `maxTotalAgents` | `1000` | Total `agent()` calls one run may start (runaway-loop backstop). |
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()`/`pipeline()` call. |
| `syncTimeoutMs` | `5000` | vm timeout for the script's initial synchronous slice (in the worker). |
| `disposeGraceMs` | `5000` | How long a cancelled run may stay unsettled before force-settle + terminate; also bounds `dispose()`. |
| `provider` | `spawn` | Host-side subagent provider used by `agent()`. |
| `maxConcurrentAgents` | `0` | Concurrent `agent()` ceiling; `0` resolves from available CPU parallelism. |
| `maxTotalAgents` | `1000` | Total `agent()` calls in one run. |
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()` or `pipeline()` call. |
| `syncTimeoutMs` | `5000` | VM timeout for the script's initial synchronous slice. |
| `disposeGraceMs` | `5000` | Bound before force-settlement/termination and for public disposal. |

305
packages/workflow/workflow-workerthread/src/host.ts
View File

@@ -25,33 +25,18 @@
* eventual exit performs a final disposal-only sweep without repeating child
* cancellation.
*
* Children live in a host-side registry (callId → run) as soon as the provider
* accepts them, so cancellation reaches even a pre-publication attempt. Both
* explicit run cancellation and the shared request signal are driven when the
* workflow is cancelled OR normally settles, so a fire-and-forget child cannot
* survive merely by honoring only one channel. A per-call gate invokes each
* explicit provider `cancel()` at most once even though host fanout and the
* worker's later relay can both request it. The host observes `result`
* immediately but acknowledges the child to the worker only after `started`
* fulfills; readiness failure is a start error and the host disposes the
* attempt because the worker never received a handle. The
* worker drives disposal by RPC on the graceful path, `dispose()` host-drives
* every registered child's disposal immediately (a wedged worker can relay no
* dispose RPC, and child teardown must overlap the grace, not start after it),
* and the registry lets the host abort and dispose every survivor when the
* worker dies or is terminated mid-flight. The three
* paths share ONE disposal per child (memoized by callId; the seam's
* dispose() is idempotent anyway, the memo keeps the bookkeeping and the
* containment warn single). Lifecycle pairing is host-guaranteed the same
* way: every forwarded `agent-start` lives in a ledger, and a start the dead
* or terminated worker never paired is closed exactly once by a synthesized
* `agent-end` (outcome `'cancelled'`). When death or grace is the terminal
* source, already-known pairs close before the run settles; cleanup after an
* earlier Result can close a survivor afterward. On a termination path
* `agentsStarted` reports the
* HOST-observed count (accepted `child-start` messages) — `agent()` calls
* still queued worker-side for a concurrency slot are unknowable then; the
* worker's own count rides the result message on every graceful path.
* Provider starts and published children are tracked separately. Every start
* receives one shared per-run abort signal; the provider owns partial setup
* until its promise fulfills. If admission closes while a start is pending,
* the signal aborts it; a late fulfillment is disposed without publication to
* the worker. Ready runs enter a callId registry whose memoized disposal is
* shared by graceful worker RPC, public disposal, normal-settlement reap, and
* worker-death cleanup. Quiescence requires both pending starts and published
* children to drain. Lifecycle pairing is host-guaranteed independently:
* every forwarded `agent-start` enters a ledger, and a dead or terminated
* worker's missing `agent-end` is synthesized exactly once as cancelled. On a
* termination path `agentsStarted` reports the host-observed child-start count;
* calls still queued worker-side for a concurrency slot are unknowable.
*
* @module @deepseek-ai/dsh-workflow-workerthread/host
*/
@@ -71,6 +56,12 @@ import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
import type { HostToWorkerPayloads, WorkerToHostMessage } from './protocol.ts'
import type { ChildResult, ChildStartRequest, WorkerInit } from './types.ts'
/** One published child and its shared quiescent-disposal transaction. */
interface ChildRecord {
readonly run: SubagentRun
disposal?: Promise<void>
}
/**
* Resolve the worker entry and spawn options for the current runtime shape.
* Unbuilt (tsx demos, vitest — `import.meta.url` points into `src/`), the
@@ -135,9 +126,9 @@ function resolveWorkerSpawn(init: WorkerInit): { entry: URL; options: WorkerOpti
/**
* One live worker-engine run — the seam's {@link WorkflowRun}, returned by
* `start()` directly. Owns the Worker, the child registry, and the result
* settlement; `result` never rejects. `meta` is this handle's OWN clone
* (event payloads carry separate clones), so a consumer mutating it corrupts
* nothing. The holder-bound SubagentService handle is captured before the
* settlement; `result` never rejects. `meta` is trusted same-process data
* borrowed as immutable by the handle and lifecycle events. The holder-bound
* SubagentService handle is captured before the
* engine returns this run, so unloading the engine removes only the ability to
* start another workflow; this run can still start and clean up its children.
*/
@@ -157,12 +148,10 @@ export class WorkerRun implements WorkflowRun {
private workerGone = false
/** Accepted `child-start` messages — the terminate-path `agentsStarted` (see module doc). */
private hostStarted = 0
/** Live children by callId; an entry leaves ONLY after its dispose settles (quiescence = empty). */
private readonly children = new Map<number, SubagentRun>()
/** In-flight child disposals by callId — the memo that gives every path (worker RPC, dispose(), reap) ONE shared disposal per child. */
private readonly childDisposals = new Map<number, Promise<void>>()
/** callIds whose explicit provider cancel callback has already been invoked. */
private readonly childCancellations = new Set<number>()
/** Published children by callId; an entry leaves only after disposal settles. */
private readonly children = new Map<number, ChildRecord>()
/** Provider starts that have not yet fulfilled or rejected. */
private readonly pendingStarts = new Set<Promise<void>>()
/** Started-but-not-ended agents by seq — the pairing ledger the HOST guarantees (see {@link endAgent}). */
private readonly liveAgents = new Map<number, WorkflowAgentInfo>()
private readonly quiescenceWaiters: (() => void)[] = []
@@ -214,11 +203,8 @@ export class WorkerRun implements WorkflowRun {
/**
* Cancel the run: the worker is told (its hooks start throwing and the
* script dies at its next await), every host-side child is cancelled NOW on
* BOTH seam channels — the shared request signal aborts and each registered
* child's explicit `cancel()` is called (the seam leaves a provider free to
* honor either, and a worker wedged in a synchronous spin could not relay
* its own per-child cancel RPCs until far too late) — and the grace timer
* script dies at its next await), the required signal shared by every child
* start is aborted, and the grace timer
* arms: a run still unsettled `disposeGraceMs` later force-settles
* `cancelled` and its worker is TERMINATED. Idempotent; the first reason
* wins.
@@ -234,11 +220,7 @@ export class WorkerRun implements WorkflowRun {
if (this.settled || this.terminalClaimed || this.cancelReason !== undefined) return
this.cancelReason = reason ?? 'workflow cancelled'
this.post(HostToWorkerType.Cancel, { reason: this.cancelReason })
// The explicit channel is driven host-side, not left to the worker: a
// provider honoring only run.cancel() must not wait on a wedged worker's
// ChildCancel relay (the per-call cancellation gate makes those later
// RPCs no-ops without imposing idempotence on the provider).
this.cancelChildren(this.cancelReason)
this.abortChildren(this.cancelReason)
this.graceTimer = setTimeout(() => {
// Cancellation already owns the race through cancelReason; close the
// terminal boundary explicitly before observer teardown callbacks.
@@ -351,12 +333,6 @@ export class WorkerRun implements WorkflowRun {
case WorkerToHostType.ChildStart:
this.onChildStart(message.callId, message.request)
break
case WorkerToHostType.ChildCancel:
{
const run = this.children.get(message.callId)
if (run !== undefined) this.cancelChild(message.callId, run, message.reason)
}
break
case WorkerToHostType.ChildDispose:
this.onChildDispose(message.callId)
break
@@ -369,7 +345,7 @@ export class WorkerRun implements WorkflowRun {
}
}
/** Why a child may no longer cross the provider readiness boundary. */
/** Why a ready provider result may no longer be admitted to the worker. */
private childAdmissionFailure(): { reason: string; rendered: string } | undefined {
if (this.cancelReason !== undefined) {
return { reason: this.cancelReason, rendered: `workflow run cancelled: ${this.cancelReason}` }
@@ -393,9 +369,20 @@ export class WorkerRun implements WorkflowRun {
return
}
this.hostStarted += 1
const task = this.startChild(callId, request)
this.pendingStarts.add(task)
void task.then(
() => { this.finishPendingStart(task) },
/* v8 ignore next -- startChild contains provider and cleanup failures */
() => { this.finishPendingStart(task) },
)
}
/** Await one provider-owned startup transaction and publish only while admitted. */
private async startChild(callId: number, request: ChildStartRequest): Promise<void> {
let run: SubagentRun
try {
run = this.subagents.start(this.provider, {
run = await this.subagents.start(this.provider, {
prompt: [{ type: 'text', text: request.prompt }],
parent: this.parent,
signal: this.controller.signal,
@@ -403,32 +390,36 @@ export class WorkerRun implements WorkflowRun {
...request.model !== undefined ? { agentOptions: { model: request.model } } : {},
})
} catch (error: unknown) {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
const failure = this.childAdmissionFailure()
this.post(HostToWorkerType.ChildStartError, {
callId,
rendered: failure?.rendered ?? renderThrown(error),
})
return
}
const failure = this.childAdmissionFailure()
if (failure !== undefined) {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: failure.rendered })
try {
await run.dispose()
} catch (error: unknown) {
this.ctx.logger.warn(`workflow-workerthread: refused child dispose failed: ${renderThrown(error)}`)
}
return
}
this.children.set(callId, run)
const childId = run.id
// Observe settlement IMMEDIATELY, before readiness. A provider may reject
// result and started in the same turn; delaying this handler would make the
// result transiently unhandled. Buffer a forwarding closure so the worker
// still sees ChildStarted before ChildSettled/ChildFailed. Snapshot a
// resolved result now: a provider mutating its resolved object while
// publication is pending must not change what crosses the worker boundary.
const record: ChildRecord = { run }
this.children.set(callId, record)
// Attach result forwarding before publishing the child handle. Because the
// callback itself runs in a later microtask, ChildStarted is still posted
// first even for an already-settled scripted provider.
const forwardResult = run.result.then<() => void, () => void>(
(result) => {
try {
// Capture every provider-owned field once, then materialize the
// worker-bound value in one lossless traversal. A stateful accessor
// cannot validate one result and send another, and an exotic value is
// rejected before any prototype-erasing clone.
const output = result.output
const structured = result.structured
const stopReason = result.stopReason
const snapshot = snapshotJsonValue<ChildResult>({
output,
...structured !== undefined ? { structured } : {},
stopReason,
output: result.output,
...result.structured !== undefined ? { structured: result.structured } : {},
stopReason: result.stopReason,
})
if (snapshot === undefined) throw new TypeError('child result is not losslessly JSON-serializable')
return () => { this.post(HostToWorkerType.ChildSettled, { callId, result: snapshot }) }
@@ -442,67 +433,20 @@ export class WorkerRun implements WorkflowRun {
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
},
)
// The provider owns the publication boundary. Observe both promises before
// invoking cancellation/disposal below: provider.start() itself is
// arbitrary code and may have reentered handle.cancel() before the returned
// run reached our registry. Exactly one branch answers this ChildStart.
let startReplySent = false
const refusePublication = (failure: { reason: string; rendered: string }): void => {
startReplySent = true
this.post(HostToWorkerType.ChildStartError, { callId, rendered: failure.rendered })
// A prior dispose/death can finish and remove this run while readiness
// is still pending. In that case teardown already owned cancellation and
// disposal; touching the retired callId would repeat cancel and orphan a
// fresh gate entry after finishChild deleted it.
if (this.children.get(callId) !== run) return
this.cancelChild(callId, run, failure.reason)
void this.disposeChild(callId, run)
}
// Only acknowledge the child after it is real, then flush any result that
// settled unusually early. Re-check admission at that exact boundary: a
// cancellation while readiness was pending is a refusal, not a late
// publication into a terminal workflow. A readiness rejection is a START
// failure, not AGENT_RESULT; the worker never receives a handle, so the
// host disposes the registered attempt. Identity guards preserve the one
// disposal memo against concurrent host teardown.
void run.started.then(
() => {
if (startReplySent) return
const failure = this.childAdmissionFailure()
if (failure !== undefined) {
refusePublication(failure)
return
}
startReplySent = true
this.post(HostToWorkerType.ChildStarted, { callId, childId })
void forwardResult.then((forward) => { forward() })
},
(error: unknown) => {
if (startReplySent) return
startReplySent = true
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
if (this.children.get(callId) === run) void this.disposeChild(callId, run)
},
)
// Close the synchronous hole around provider.start(): cancel()/dispose()
// can run before the returned run is visible to their children loop.
const reentrantFailure = this.childAdmissionFailure()
if (reentrantFailure !== undefined) refusePublication(reentrantFailure)
this.post(HostToWorkerType.ChildStarted, { callId, childId: run.id })
void forwardResult.then((forward) => { forward() })
}
private onChildDispose(callId: number): void {
const run = this.children.get(callId)
if (run === undefined) {
const record = this.children.get(callId)
if (record === undefined) {
// Already disposed host-side (a dispose() drive or a death reap beat
// the RPC) — the ack is still owed (the worker-side wrapper awaits it).
this.post(HostToWorkerType.ChildDisposed, { callId })
return
}
// disposeChild never rejects (containment is inside), so the ack always follows.
void this.disposeChild(callId, run).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
void this.disposeChild(callId, record).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
}
/**
@@ -514,79 +458,55 @@ export class WorkerRun implements WorkflowRun {
* not supposed to reject, but a backend that does anyway must not break
* quiescence): logged, and the child still leaves the registry.
* @param callId - the child's registry key.
* @param run - the registered child (the caller looked it up).
* @param record - the registered child (the caller looked it up).
* @returns resolves when the disposal settled either way; never rejects.
*/
private disposeChild(callId: number, run: SubagentRun): Promise<void> {
let disposal = this.childDisposals.get(callId)
if (disposal === undefined) {
// Claim before run.dispose() invokes provider code. Reentrant holder
// disposal then joins this exact child transaction instead of entering
// the provider wrapper twice before either memo is installed.
const claimed = Promise.withResolvers<undefined>()
disposal = claimed.promise
this.childDisposals.set(callId, disposal)
// The seam promises a Promise, but invoke inside an async boundary so a
// contract-violating synchronous throw is contained exactly like a
// rejected disposal and cannot break host quiescence.
void (async () => { await run.dispose() })().then(
() => {
this.finishChild(callId)
claimed.resolve(undefined)
},
(error: unknown) => {
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)
this.finishChild(callId)
claimed.resolve(undefined)
},
)
}
return disposal
private disposeChild(callId: number, record: ChildRecord): Promise<void> {
if (record.disposal !== undefined) return record.disposal
record.disposal = Promise.resolve()
.then(() => record.run.dispose())
.catch((error: unknown) => {
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)
})
.then(() => { this.finishChild(callId, record) })
return record.disposal
}
/** Drop a child from the registry (and its disposal memo), releasing quiescence waiters at zero. */
private finishChild(callId: number): void {
this.children.delete(callId)
this.childDisposals.delete(callId)
this.childCancellations.delete(callId)
if (this.children.size === 0) {
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
}
/** Drop an exact child record and release quiescence waiters when all work ends. */
private finishChild(callId: number, record: ChildRecord): void {
if (this.children.get(callId) === record) this.children.delete(callId)
this.notifyChildQuiescence()
}
/** Resolves once the child registry is empty (every disposal settled). */
/** Retire one provider startup transaction. */
private finishPendingStart(task: Promise<void>): void {
this.pendingStarts.delete(task)
this.notifyChildQuiescence()
}
/** Release waiters only after both pending starts and published children end. */
private notifyChildQuiescence(): void {
if (this.children.size !== 0 || this.pendingStarts.size !== 0) return
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
}
/** Resolves once every pending start and published child has reached quiescence. */
private childQuiescence(): Promise<void> {
if (this.children.size === 0) return Promise.resolve()
if (this.children.size === 0 && this.pendingStarts.size === 0) return Promise.resolve()
return new Promise((resolve) => { this.quiescenceWaiters.push(resolve) })
}
/** Abort + dispose every registered child (worker death / final teardown); disposal is contained, not awaited. */
private reapChildren(reason: string): void {
const cancellation = this.cancelReason ?? reason
this.cancelChildren(cancellation)
for (const [callId, run] of [...this.children]) {
void this.disposeChild(callId, run)
this.abortChildren(this.cancelReason ?? reason)
for (const [callId, record] of [...this.children]) {
void this.disposeChild(callId, record)
}
}
/** Drive both cancellation channels for every child already accepted by the host. */
private cancelChildren(reason: string): void {
this.controller.abort(reason)
for (const [callId, run] of this.children) this.cancelChild(callId, run, reason)
}
/** Invoke one provider-owned cancel callback at most once and contain its exception. */
private cancelChild(callId: number, run: SubagentRun, reason?: string): void {
// Host fanout and the worker's FIFO-later ChildCancel relay are two paths
// to the same provider callback. The seam does not require cancel() to be
// idempotent, so claim the callId before invoking arbitrary provider code.
if (this.childCancellations.has(callId)) return
this.childCancellations.add(callId)
try {
run.cancel(reason)
} catch (error: unknown) {
this.ctx.logger.warn(`workflow-workerthread: child cancel failed: ${renderThrown(error)}`)
}
/** Abort the one canonical signal shared by pending and published children. */
private abortChildren(reason: string): void {
if (!this.controller.signal.aborted) this.controller.abort(reason)
}
private onResult(result: WorkflowResult): void {
@@ -599,17 +519,14 @@ export class WorkerRun implements WorkflowRun {
// callbacks, but that internal post-result cleanup must not retroactively
// rewrite the worker result that arrived first.
const cancellationWasRequested = this.cancelReason !== undefined
// Claim before either settlement-cleanup cancellation channel invokes
// provider code. A provider callback can reenter cancel() synchronously or
// from a queued microtask; once Result won, that losing cancellation must
// have no state, message, child-fanout, or grace-timer side effects.
// Claim before settlement cleanup invokes provider disposal. Once Result
// won, a later cancellation cannot rewrite it.
this.terminalClaimed = true
// The worker cancels handles it already received, but a fire-and-forget
// child may still be waiting on readiness and therefore have no worker
// handle. Drive BOTH provider-permitted channels from the host before the
// workflow becomes externally settled.
// Abort pending starts and begin disposing published children before the
// workflow becomes externally settled. Cleanup remains independently
// tracked by childQuiescence and the holder's dispose().
this.reapChildren('workflow settled')
if (!cancellationWasRequested) {
this.cancelChildren('workflow settled')
this.settleResult(result)
return
}
@@ -640,7 +557,7 @@ export class WorkerRun implements WorkflowRun {
// accepted before death remains cancelled. If Result/grace already won,
// preserve it while still performing prompt failure-time cleanup.
if (!outcomeWasClaimed) this.terminalClaimed = true
if (this.children.size > 0) this.reapChildren('workflow worker gone')
if (this.children.size > 0 || this.pendingStarts.size > 0) this.reapChildren('workflow worker gone')
this.endStrandedAgents()
if (!outcomeWasClaimed) {
if (cancellationWasRequested) {
@@ -655,7 +572,7 @@ export class WorkerRun implements WorkflowRun {
// precede `exit`. Admission is already closed, so this final sweep only
// joins/starts disposal for registry survivors; it deliberately does not
// repeat explicit provider cancellation.
for (const [callId, run] of [...this.children]) void this.disposeChild(callId, run)
for (const [callId, record] of [...this.children]) void this.disposeChild(callId, record)
this.endStrandedAgents()
}

6
packages/workflow/workflow-workerthread/src/index.ts
View File

@@ -151,9 +151,7 @@ export class WorkerWorkflowEngine extends WorkflowService {
const meta = validateMeta(request.meta)
assertBodyParses(request.script, meta.name)
const id = WorkflowRunId(randomUUID())
// The event payloads and the run handle get SEPARATE meta clones: a
// listener mutating its snapshot must not corrupt the holder's view.
const info: WorkflowRunInfo = { id, meta: structuredClone(meta) }
const info: WorkflowRunInfo = { id, meta }
const limits: WorkerLimits = {
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
? Math.min(16, Math.max(1, availableParallelism() - 2))
@@ -180,7 +178,7 @@ export class WorkerWorkflowEngine extends WorkflowService {
runCtx,
subagents,
id,
structuredClone(meta),
meta,
request.parent,
init,
this.config.provider,

8
packages/workflow/workflow-workerthread/src/protocol.ts
View File

@@ -33,8 +33,6 @@ export enum WorkerToHostType {
AgentEnd = 'agent-end',
/** Child RPC: start a child on the host (answered by ChildStarted or ChildStartError). */
ChildStart = 'child-start',
/** Child RPC: cancel a started child (fire-and-forget). */
ChildCancel = 'child-cancel',
/** Child RPC: dispose a started child (answered by ChildDisposed). */
ChildDispose = 'child-dispose',
/** The run's single terminal result. */
@@ -55,8 +53,6 @@ export interface WorkerToHostPayloads {
[WorkerToHostType.AgentEnd]: { info: WorkflowAgentEndInfo }
/** The RPC correlation id and the prompt plus validated options. */
[WorkerToHostType.ChildStart]: { callId: number; request: ChildStartRequest }
/** The RPC correlation id and the cancel reason (undefined = unspecified). */
[WorkerToHostType.ChildCancel]: { callId: number; reason: string | undefined }
/** The RPC correlation id of the child to dispose. */
[WorkerToHostType.ChildDispose]: { callId: number }
/** The run's terminal outcome. */
@@ -69,9 +65,9 @@ export enum HostToWorkerType {
Go = 'go',
/** Cancel the run: hooks start throwing and the script dies at its next await. */
Cancel = 'cancel',
/** Child RPC reply: provider publication/readiness fulfilled (exactly one start reply per ChildStart). */
/** Child RPC reply: the provider fulfilled with a ready run (exactly one start reply per ChildStart). */
ChildStarted = 'child-started',
/** Child RPC reply: synchronous start or asynchronous publication/readiness failed. */
/** Child RPC reply: the provider's asynchronous start failed. */
ChildStartError = 'child-start-error',
/** Child RPC: a started child's result RESOLVED (its JSON projection). */
ChildSettled = 'child-settled',

47
packages/workflow/workflow-workerthread/src/runtime.ts
View File

@@ -20,7 +20,7 @@
*
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
* unsupported options/schemas, tripped caps, synchronous start refusal,
* pre-publication readiness failure, ready-child result rejection, and
* provider-start failure, ready-child result rejection, and
* cancellation) ALWAYS propagate through
* `parallel`/`pipeline` — recognized by `instanceof` against this realm's
* class, which a script inside the vm context cannot forge — and the per-item
@@ -83,9 +83,8 @@ function defaultLabel(prompt: string): string {
/**
* One live script execution inside the worker. Constructed per run by the
* session; `drive()` is called exactly once and NEVER rejects — every failure
* becomes a {@link WorkflowResult} with a non-`completed` stop reason. After
* the session publishes that result it calls {@link reapAfterResult} exactly
* once to cancel any dropped child work without racing terminal publication.
* becomes a {@link WorkflowResult} with a non-`completed` stop reason. The
* host owns cancellation and cleanup of any dropped child work.
*/
export class WorkflowExecution {
/** 1-based count of `agent()` calls started (the `agentsStarted` result field). */
@@ -94,7 +93,6 @@ export class WorkflowExecution {
private readonly slotWaiters: { resolve(): void; reject(error: unknown): void }[] = []
private cancelReason: string | undefined
private cancelError: WorkflowError | undefined
private readonly controller = new AbortController()
private currentPhase: string | undefined
private readonly context: vm.Context
private readonly compiled: vm.Script
@@ -130,11 +128,8 @@ export class WorkflowExecution {
pipeline: (items: unknown, ...stages: unknown[]) => this.contain(this.pipeline(items, stages)),
phase: (title: unknown) => { this.phase(title) },
log: (message: unknown) => { this.log(message) },
// Cloned once: a script scribbling on args must not mutate the
// session's init object (a benign-bug guard; args is plain JSON by the
// seam contract and already crossed one structured clone as workerData,
// so this clone is total).
args: args === undefined ? undefined : structuredClone(args),
// workerData already performed the real cross-thread structured clone.
args,
}
for (const [key, value] of Object.entries(globals)) {
// Data properties on the contextified global; frozen shape not required —
@@ -165,22 +160,18 @@ export class WorkflowExecution {
}
/**
* Cancel the run: in-flight children get a cancel RPC (the shared abort
* fanout), waiting `agent()` slots reject, and every future hook call
* Cancel the run: waiting `agent()` slots reject and every future hook call
* throws `CANCELLED` — the script dies at its next await. A script that
* never settles anyway (parked on a promise no hook owns) is the HOST's
* problem: its grace timer force-settles the run and terminates the
* worker. Idempotent; the first reason wins.
* @param reason - human-readable cause, carried on the CANCELLED error and
* into child cancel RPCs. Required: every caller (the session's cancel
* message and its post-result {@link reapAfterResult} call) has a concrete
* reason.
* @param reason - human-readable cause carried on the CANCELLED error. The
* host independently aborts the required signal shared by every child.
*/
cancel(reason: string): void {
if (this.cancelReason !== undefined) return
this.cancelReason = reason
this.cancelError = new WorkflowError(`workflow run cancelled: ${this.cancelReason}`, 'CANCELLED')
this.controller.abort(this.cancelReason)
for (const waiter of this.slotWaiters.splice(0)) waiter.reject(this.cancelledError())
}
@@ -188,9 +179,8 @@ export class WorkflowExecution {
* Run the script to settlement. Resolves — never rejects — with the run's
* {@link WorkflowResult}: the materialized return value on `completed`, the
* failure message on `error`, and `cancelled` when the script died of
* cancellation. This method only chooses the result; the session must publish
* it and then call {@link reapAfterResult}, so the terminal message precedes
* settlement-only child cancellation on the worker-to-host FIFO channel.
* cancellation. This method only chooses the result; the session publishes
* it and the host owns terminal child cancellation.
* @returns the settled outcome — this promise NEVER rejects (the seam's
* `result`-never-rejects contract); every failure maps to a variant.
*/
@@ -221,16 +211,6 @@ export class WorkflowExecution {
}
}
/**
* Reap strays only after the caller publishes the chosen terminal result.
* Aborting the controller synchronously sends child-cancel RPCs, so calling
* this before publication would let a provider callback reenter host
* cancellation and misclassify a result the script had already chosen.
*/
reapAfterResult(): void {
if (this.cancelReason === undefined) this.cancel('workflow settled')
}
/**
* Attach a no-op rejection consumer WITHOUT changing what the caller
* receives: if the script drops the promise (no await), cancellation cannot
@@ -336,17 +316,11 @@ export class WorkflowExecution {
// wind the fresh child down instead of leaving it live behind a dead
// script.
if (this.isCancelled()) {
run.cancel(this.cancelReason)
await run.dispose()
throw this.cancelledError()
}
const info: WorkflowAgentInfo = { seq, label, ...phase !== undefined ? { phase } : {}, childId: AgentId(run.id) }
this.observer.agentStart(info)
// Cancellation reaches the child through an explicit cancel RPC per
// child (the host also aborts its own per-run signal, but the seam
// leaves a provider free to honor either channel, so both are driven).
const onAbort = (): void => { run.cancel(this.cancelReason) }
this.controller.signal.addEventListener('abort', onAbort, { once: true })
try {
let result
try {
@@ -387,7 +361,6 @@ export class WorkflowExecution {
this.observer.agentEnd({ ...info, outcome: 'failed' })
return null
} finally {
this.controller.signal.removeEventListener('abort', onAbort)
await run.dispose()
}
} finally {

26
packages/workflow/workflow-workerthread/src/session.ts
View File

@@ -14,11 +14,6 @@
* A `cancel` arriving instead of `go` still releases the gate: `drive()`
* sees the cancelled state and settles without running the body.
*
* Terminal ordering is Result first, settlement cleanup second. The session
* queues the Result message before asking the execution to reap stray children;
* MessagePort FIFO therefore lets the host atomically claim the result before a
* cleanup ChildCancel can invoke arbitrary provider code.
*
* @module @deepseek-ai/dsh-workflow-workerthread/session
*/
@@ -64,10 +59,6 @@ class RpcChildHandle implements ChildHandle {
this.result = entry.settled.promise
}
cancel(reason?: string): void {
this.post(WorkerToHostType.ChildCancel, { callId: this.callId, reason })
}
dispose(): Promise<void> {
this.post(WorkerToHostType.ChildDispose, { callId: this.callId })
return this.entry.disposed.promise
@@ -76,7 +67,7 @@ class RpcChildHandle implements ChildHandle {
/**
* The worker-side child-RPC bridge ({@link ChildPort}): allocates callIds,
* posts the start/cancel/dispose RPCs, and owns the per-call pending
* posts the start/dispose RPCs, and owns the per-call pending
* book-keeping the session's message handler settles via the `onChild*`
* entry points.
*/
@@ -94,10 +85,10 @@ class ChildRpcBridge implements ChildPort {
settled: Promise.withResolvers<ChildResult>(),
disposed: Promise.withResolvers<void>(),
}
// Containment: when synchronous start or asynchronous readiness fails (or
// Containment: when asynchronous provider start fails (or
// the run is torn down), the settled promise may never gain a consumer —
// it must not surface as an unhandled rejection and kill the worker.
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start/readiness */ })
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start */ })
this.pending.set(callId, entry)
this.post(WorkerToHostType.ChildStart, { callId, request })
const childId = await entry.started.promise
@@ -109,7 +100,7 @@ class ChildRpcBridge implements ChildPort {
this.pending.get(callId)?.started.resolve(childId)
}
/** Synchronous start or asynchronous readiness failed; reject and retire the pending RPC. */
/** Asynchronous provider start failed; reject and retire the pending RPC. */
onChildStartError(callId: number, rendered: string): void {
const entry = this.pending.get(callId)
this.pending.delete(callId)
@@ -213,12 +204,5 @@ export async function runWorkerSession(port: MessagePort, init: WorkerInit): Pro
post(WorkerToHostType.Ready, {})
await gate.promise
const result = await execution.drive()
try {
// This post is the worker's terminal claim. Queue it BEFORE aborting stray
// children: MessagePort FIFO then guarantees the host claims Result before
// any settlement-only ChildCancel can invoke arbitrary provider callbacks.
post(WorkerToHostType.Result, { result })
} finally {
execution.reapAfterResult()
}
post(WorkerToHostType.Result, { result })
}

4
packages/workflow/workflow-workerthread/src/types.ts
View File

@@ -77,8 +77,6 @@ export interface ChildHandle {
* failed for its own reasons resolves with a non-`completed` stop reason.
*/
readonly result: Promise<ChildResult>
/** Ask the host to cancel the child (fire-and-forget). */
cancel(reason?: string): void
/** Ask the host to dispose the child; resolves on the host's ack. */
dispose(): Promise<void>
}
@@ -92,7 +90,7 @@ export interface ChildPort {
* Start one child agent on the host (the `agent()` hook's start half).
* @param request - the prompt and validated options.
* @returns the ready child handle; rejects when synchronous start or the
* provider's asynchronous publication/readiness boundary fails.
* provider's asynchronous start fails.
*/
startAgent(request: ChildStartRequest): Promise<ChildHandle>
}

2
packages/workflow/workflow-workerthread/tests/integration.spec.ts
View File

@@ -49,7 +49,7 @@ describe('dsh-workflow-workerthread over the real in-process stack', () => {
])
const childIds: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => {
// The workflow bridge must honor SubagentRun.started: a start observer
// The workflow bridge must await asynchronous provider start: an observer
// sees the real spawn child already published, never a reserved id.
expect(ctx.agents.get(agent.childId)).toBeDefined()
childIds.push(agent.childId)

36
packages/workflow/workflow-workerthread/tests/session.spec.ts
View File

@@ -212,7 +212,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
host.close()
})
it('cancel mid-run: in-flight children get cancel RPCs, hooks throw at entry, the run reports cancelled', async () => {
it('cancel mid-run: hooks throw at entry and the run reports cancelled', async () => {
const host = fakeHost()
void runWorkerSession(host.port, init(`
phase('before')
@@ -232,7 +232,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
const result = await host.result()
expect(result.stopReason).toBe('cancelled')
expect(result.error).toContain('stop everything')
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
// No post-cancel narration left the runtime (the hooks threw at entry).
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['before'])
@@ -281,36 +280,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
}
})
it('queues Result before settlement-only cancellation of a ready stray', async () => {
const host = fakeHost({ manual: true })
const session = runWorkerSession(host.port, init(`
agent('ready stray')
return await agent('gate')
`))
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart)).toHaveLength(2) })
const starts = host.ofType(WorkerToHostType.ChildStart)
const stray = starts.find(message => message.request.prompt === 'ready stray')!
const gate = starts.find(message => message.request.prompt === 'gate')!
host.send({ type: HostToWorkerType.ChildStarted, callId: stray.callId, childId: 'stray-child' })
host.send({ type: HostToWorkerType.ChildStarted, callId: gate.callId, childId: 'gate-child' })
host.send({ type: HostToWorkerType.ChildSettled, callId: gate.callId, result: text('gate completed') })
const result = await host.result()
await session
await vi.waitFor(() => {
expect(host.ofType(WorkerToHostType.ChildCancel).map(message => message.callId)).toContain(stray.callId)
})
expect(result).toMatchObject({ value: 'gate completed', stopReason: 'completed', agentsStarted: 2 })
const resultIndex = host.messages.findIndex(message => message.type === WorkerToHostType.Result)
const strayCancelIndex = host.messages.findIndex(message =>
message.type === WorkerToHostType.ChildCancel && message.callId === stray.callId)
expect(resultIndex).toBeGreaterThanOrEqual(0)
expect(strayCancelIndex).toBeGreaterThan(resultIndex)
host.send({ type: HostToWorkerType.ChildSettled, callId: stray.callId, result: { output: [], stopReason: 'aborted' } })
host.close()
})
it('an unparseable body settles an error result instead of dying without one (host pre-parse skew guard)', async () => {
const host = fakeHost()
await runWorkerSession(host.port, init('return ((('))
@@ -463,7 +432,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
host.close()
})
it('a cancel landing DURING the start round-trip winds the fresh child down (cancel + dispose) and dies cancelled', async () => {
it('a cancel landing DURING the start round-trip disposes the fresh child and dies cancelled', async () => {
const host = fakeHost({ manual: true })
void runWorkerSession(host.port, init("return await agent('p')"))
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
@@ -477,7 +446,6 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
const result = await host.result()
expect(result.stopReason).toBe('cancelled')
await vi.waitFor(() => {
expect(host.ofType(WorkerToHostType.ChildCancel).map(m => m.callId)).toContain(callId)
expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId)
})
// The child never became an agent-start: it was wound down pre-lifecycle.

531
packages/workflow/workflow-workerthread/tests/workflow-workerthread.spec.ts
View File

@@ -48,9 +48,9 @@ const ESCAPE = "globalThis.constructor.constructor('return process')()"
/** One controllable child run: the test (or auto mode) settles it. */
interface ControlledRun {
request: SubagentStartRequest
/** Fulfill the provider publication/readiness boundary. */
/** Fulfill the provider's async start with a ready child. */
publish(): void
/** Reject the provider publication/readiness boundary. */
/** Reject the provider's async start before ownership transfer. */
rejectStart(error: unknown): void
settle(result: SubagentResult): void
rejectResult(error: unknown): void
@@ -75,17 +75,19 @@ class StubProvider implements SubagentProvider {
private readonly reply?: (request: SubagentStartRequest, index: number) => SubagentResult,
private readonly disposeDelayMs = 0,
private readonly deferStart = false,
private readonly onCancel?: (reason: string | undefined, index: number) => void,
private readonly onAbortString?: (reason: string | undefined, index: number) => void,
private readonly onSignalAbort?: (reason: unknown, index: number) => void,
) {}
start(request: SubagentStartRequest): SubagentRun {
const readiness = Promise.withResolvers<undefined>()
async start(request: SubagentStartRequest): Promise<SubagentRun> {
const startGate = Promise.withResolvers<undefined>()
const terminal = Promise.withResolvers<SubagentResult>()
terminal.promise.catch(() => { /* provider owns early settlement until publication */ })
let published = false
const controlled: ControlledRun = {
request,
publish: () => { readiness.resolve(undefined) },
rejectStart: (error) => { readiness.reject(error) },
publish: () => { published = true; startGate.resolve(undefined) },
rejectStart: (error) => { startGate.reject(error) },
settle: (result) => { terminal.resolve(result) },
rejectResult: (error) => { terminal.reject(error) },
cancelled: undefined,
@@ -94,24 +96,29 @@ class StubProvider implements SubagentProvider {
}
this.runs.push(controlled)
const index = this.runs.length - 1
request.signal?.addEventListener('abort', () => {
this.onSignalAbort?.(request.signal?.reason, index)
terminal.resolve({ output: [], stopReason: 'aborted' })
request.signal.addEventListener('abort', () => {
controlled.cancelled = String(request.signal.reason ?? 'cancelled')
this.onAbortString?.(String(request.signal.reason ?? 'cancelled'), index)
this.onSignalAbort?.(request.signal.reason, index)
if (published) terminal.resolve({ output: [], stopReason: 'aborted' })
else startGate.reject(new Error('child start aborted before publication'))
}, { once: true })
if (!this.deferStart) readiness.resolve(undefined)
if (!this.deferStart) controlled.publish()
if (this.reply) {
const reply = this.reply
queueMicrotask(() => { terminal.resolve(reply(request, index)) })
}
try {
await startGate.promise
} catch (error: unknown) {
controlled.disposeCalls += 1
controlled.disposed = true
throw error
}
if (request.signal.aborted) throw new Error('child start aborted before publication')
return {
id: AgentId(`stub-child-${index}`),
started: readiness.promise,
result: terminal.promise,
cancel: (reason?: string) => {
controlled.cancelled = reason ?? 'cancelled'
this.onCancel?.(reason, index)
terminal.resolve({ output: [], stopReason: 'aborted' })
},
dispose: () => {
controlled.disposeCalls += 1
if (this.disposeDelayMs === 0) {
@@ -140,7 +147,7 @@ interface SetupOptions {
manual?: boolean
disposeDelayMs?: number
deferStart?: boolean
onChildCancel?: (reason: string | undefined, index: number) => void
onChildAbortString?: (reason: string | undefined, index: number) => void
onChildSignalAbort?: (reason: unknown, index: number) => void
}
@@ -152,7 +159,7 @@ async function setup(options?: SetupOptions) {
options?.manual ? undefined : options?.reply ?? (() => text('stub reply')),
options?.disposeDelayMs ?? 0,
options?.deferStart ?? false,
options?.onChildCancel,
options?.onChildAbortString,
options?.onChildSignalAbort,
)
ctx.subagents.registerProvider(provider)
@@ -243,7 +250,7 @@ describe('dsh-workflow-workerthread', () => {
expect(result.error).toContain('agent() could not start a child')
})
it('waits for child readiness before announcing it and snapshots a result that settled early', async () => {
it('waits for async provider start before announcing a result that settled early', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const order: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => { order.push(`start:${agent.seq}`) })
@@ -254,10 +261,8 @@ describe('dsh-workflow-workerthread', () => {
await waitFor(() => { expect(provider.runs.length).toBe(1) })
const early = text('accepted value')
provider.runs[0]!.settle(early)
// Let the host observe + snapshot result while readiness remains pending.
// The provider still owns this early result while start is pending.
await new Promise(resolve => setTimeout(resolve, 0))
const earlyText = early.output[0] as { type: 'text'; text: string }
earlyText.text = 'mutated after settlement'
expect(order).toEqual([])
provider.runs[0]!.publish()
@@ -268,7 +273,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('observes an early result rejection but sends ChildStarted before ChildFailed after readiness', async () => {
it('observes an early result rejection but sends ChildStarted before ChildFailed after start fulfills', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -299,7 +304,7 @@ describe('dsh-workflow-workerthread', () => {
await handle.dispose()
})
it('classifies readiness rejection as AGENT_START, drops an early result, and emits no false lifecycle pair', async () => {
it('classifies provider start rejection as AGENT_START, drops an early result, and emits no false lifecycle pair', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -311,7 +316,7 @@ describe('dsh-workflow-workerthread', () => {
})
await waitFor(() => { expect(provider.runs.length).toBe(1) })
// ACP-style failure can settle result(error) before its session/publication
// boundary rejects. Readiness must dominate that buffered child outcome.
// boundary rejects. Start rejection must dominate that buffered child outcome.
provider.runs[0]!.settle({ output: [], stopReason: 'error' })
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('publication rolled back'))
@@ -328,7 +333,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('cancels and disposes a readiness-pending child once without publishing workflow lifecycle', async () => {
it('aborts a pending provider start once without publishing workflow lifecycle', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true, config: { disposeGraceMs: 500 } })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
@@ -342,7 +347,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposed).toBe(true)
})
// Ensure the host-driven disposal removed the registry entry before the
// late readiness rejection; its callback must not invoke dispose again.
// late start rejection; its callback must not invoke dispose again.
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('cancelled before publication'))
@@ -360,11 +365,9 @@ describe('dsh-workflow-workerthread', () => {
name: 'rejecting',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('reject-child'),
started: Promise.resolve(),
result: Promise.reject(new Error('backend exploded')),
cancel: () => { /* nothing in flight */ },
dispose: () => Promise.resolve(),
}),
}
@@ -385,24 +388,20 @@ describe('dsh-workflow-workerthread', () => {
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }
`))
expect(result.value).toMatchObject({ code: 'AGENT_RESULT' })
expect((result.value as { message: string }).message).toContain('subagent result must be losslessly JSON-serializable')
expect((result.value as { message: string }).message).toContain('workflow child result could not cross the worker boundary')
})
it('contains a non-JSON result even if the injected subagent service violates its normalization contract', async () => {
// SubagentService normally rejects this before the workflow sees it. Stub
// the injected seam itself so the host's defensive worker-boundary guard
// remains independently covered rather than becoming dead, untested code.
// The real worker boundary must reject a non-JSON same-process result.
const { ctx, parent } = await setup()
const invalid = {
output: [],
structured: () => { /* deliberately outside lossless JSON */ },
stopReason: 'completed',
} as unknown as SubagentResult
const start = vi.spyOn(ctx.subagents, 'start').mockReturnValue({
const start = vi.spyOn(ctx.subagents, 'start').mockResolvedValue({
id: AgentId('raw-invalid-child'),
started: Promise.resolve(),
result: Promise.resolve(invalid),
cancel: () => { /* already settled */ },
dispose: () => Promise.resolve(),
})
@@ -416,31 +415,6 @@ describe('dsh-workflow-workerthread', () => {
.toContain('workflow child result could not cross the worker boundary')
})
it('reads each resolved child-result field once before crossing the worker boundary', async () => {
let structuredReads = 0
class DriftedStructured { readonly value = 'drifted' }
const { ctx, parent } = await setup({
reply: () => ({
output: [],
get structured() {
structuredReads += 1
return structuredReads === 1 ? { value: 'accepted' } : new DriftedStructured()
},
stopReason: 'completed',
}),
})
const result = await run(ctx, parent, scripted(`
const found = await agent('p', {
schema: { type: 'object', properties: { value: { type: 'string' } }, required: ['value'] }
})
return found.value
`))
expect(result.value).toBe('accepted')
expect(structuredReads).toBe(1)
})
it('a child whose dispose() throws synchronously cannot wedge the script (the host acks anyway)', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
@@ -448,9 +422,8 @@ describe('dsh-workflow-workerthread', () => {
name: 'bad-dispose',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('bad-dispose-child'),
started: Promise.resolve(),
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
cancel: () => { /* settled already */ },
dispose: () => { throw new Error('dispose exploded') },
@@ -470,9 +443,8 @@ describe('dsh-workflow-workerthread', () => {
name: 'coercion-trap-dispose',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => ({
start: async () => ({
id: AgentId('trap-child'),
started: Promise.resolve(),
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
cancel: () => { /* settled already */ },
// The rejection VALUE's own coercion throws: a warn built with bare
@@ -755,7 +727,7 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposed).toBe(true)
})
it('dispose() reaps a registered stray after result settlement even when the worker cannot relay disposal', async () => {
it('result settlement reaps a registered stray even when the worker cannot relay disposal', async () => {
const { ctx, parent, provider } = await setup({
manual: true,
config: { provider: 'stub', disposeGraceMs: 30_000 },
@@ -775,13 +747,9 @@ describe('dsh-workflow-workerthread', () => {
result: { value: 'synthetic completion', stopReason: 'completed', agentsStarted: 1 },
})
await expect(handle.result).resolves.toMatchObject({ stopReason: 'completed' })
expect(provider.runs[0]!.disposed).toBe(false)
await waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) }, 1000)
const disposal = handle.dispose()
// A 30-second grace makes this assertion mutation-sensitive: without the
// settled-path host reap, no worker message can start child disposal and
// this bounded wait fails long before the grace fallback.
await waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) }, 1000)
await disposal
expect(provider.runs[0]!.disposeCalls).toBe(1)
await ctx.fiber.dispose()
@@ -795,21 +763,16 @@ describe('dsh-workflow-workerthread', () => {
name: 'signal-only',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: (request) => {
start: async (request) => {
let settle!: (result: SubagentResult) => void
const result = new Promise<SubagentResult>((resolve) => { settle = resolve })
request.signal?.addEventListener('abort', () => {
aborted.push(String(request.signal?.reason))
request.signal.addEventListener('abort', () => {
aborted.push(String(request.signal.reason))
settle({ output: [], stopReason: 'aborted' })
}, { once: true })
return {
id: AgentId('signal-only-child'),
started: Promise.resolve(),
result,
// The seam leaves a provider free to honor EITHER cancel channel;
// this one deliberately ignores run.cancel() — only the request
// signal can wind it down.
cancel: () => { /* signal-only by design */ },
dispose: () => Promise.resolve(),
}
},
@@ -834,7 +797,7 @@ describe('dsh-workflow-workerthread', () => {
await handle.dispose()
})
it('the settle-reap explicitly cancels a readiness-pending stray before workflow/end', async () => {
it('the settle-reap aborts a pending provider start before workflow/end', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const childLifecycle: string[] = []
let cancellationAtWorkflowEnd: string | undefined
@@ -845,7 +808,7 @@ describe('dsh-workflow-workerthread', () => {
})
const handle = ctx.workflows.start({
...scripted(`
agent('readiness-pending stray')
agent('start-pending stray')
return 'done'
`),
parent,
@@ -864,129 +827,11 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('post-result child cleanup cannot reentrantly rewrite a completed workflow as cancelled', async () => {
let cancelCallbacks = 0
let signalCallbacks = 0
const { ctx, parent, provider } = await setup({
manual: true,
deferStart: true,
onChildCancel: () => {
cancelCallbacks += 1
// The first callback is host cleanup for the already-arrived Result.
// Reentering cancel() here is later than that message and must not
// retroactively win the result race. Its nested child cancel is
// intentionally ignored to keep the adversarial callback finite.
if (cancelCallbacks === 1) handle.cancel('reentrant child cleanup')
},
onChildSignalAbort: () => {
signalCallbacks += 1
handle.cancel('reentrant signal cleanup')
},
})
const handle = ctx.workflows.start({
...scripted(`
agent('readiness-pending stray')
return 'completed first'
`),
parent,
})
const result = await handle.result
expect(result).toMatchObject({ value: 'completed first', stopReason: 'completed', agentsStarted: 1 })
expect(signalCallbacks).toBe(1)
expect(cancelCallbacks).toBe(1)
// Readiness crossing after Result is a terminal-admission refusal: no
// ChildStarted/lifecycle publication, and host-owned disposal begins.
provider.runs[0]!.publish()
await waitFor(() => { expect(provider.runs[0]!.disposed).toBe(true) }, 1000)
expect(cancelCallbacks).toBe(1)
await handle.dispose()
await ctx.fiber.dispose()
})
it('late readiness after completed disposal cannot cancel or dispose the retired child twice', async () => {
let explicitCancels = 0
const lifecycle: string[] = []
const { ctx, parent, provider } = await setup({
manual: true,
deferStart: true,
onChildCancel: () => { explicitCancels += 1 },
})
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', () => { lifecycle.push('end') })
const handle = ctx.workflows.start({
...scripted("agent('retired readiness')\nreturn 'done'"),
parent,
})
await expect(handle.result).resolves.toMatchObject({ stopReason: 'completed' })
expect(explicitCancels).toBe(1)
await handle.dispose()
expect(provider.runs[0]!.disposed).toBe(true)
expect(provider.runs[0]!.disposeCalls).toBe(1)
// The Promise may still fulfill after its run left every host ledger.
// Refusal replies once but must not recreate the deleted cancel gate.
provider.runs[0]!.publish()
await Promise.resolve()
await Promise.resolve()
expect(explicitCancels).toBe(1)
expect(provider.runs[0]!.disposeCalls).toBe(1)
expect(lifecycle).toEqual([])
await ctx.fiber.dispose()
})
it.each([
['synchronous', (cancel: () => void) => { cancel() }],
['microtask', (cancel: () => void) => { queueMicrotask(cancel) }],
])('a ready stray %s cleanup callback cannot beat the earlier worker result claim', async (_mode, reenter) => {
let reentered = false
const explicitCancels = new Map<number, number>()
const { ctx, parent, provider } = await setup({
manual: true,
onChildCancel: (_reason, index) => {
explicitCancels.set(index, (explicitCancels.get(index) ?? 0) + 1)
if (index !== 0 || reentered) return
reentered = true
reenter(() => { handle.cancel('reentered from child cleanup') })
},
})
const handle = ctx.workflows.start({
...scripted(`
agent('ready stray')
return await agent('gate')
`),
parent,
})
const cancelChildSpy = vi.spyOn(handle as unknown as {
cancelChild(callId: number, run: SubagentRun, reason?: string): void
}, 'cancelChild')
await waitFor(() => { expect(provider.runs).toHaveLength(2) })
provider.runs[1]!.settle(text('gate completed'))
const result = await handle.result
await Promise.resolve()
expect(result).toMatchObject({ value: 'gate completed', stopReason: 'completed', agentsStarted: 2 })
expect(reentered).toBe(true)
// The host claim and worker's FIFO-later ChildCancel both reach the
// routing gate, but the provider callback is not an idempotent seam:
// invoke it exactly once for this callId.
await waitFor(() => {
expect(cancelChildSpy.mock.calls.filter(([callId]) => callId === 1)).toHaveLength(2)
}, 1000)
expect(explicitCancels.get(0)).toBe(1)
cancelChildSpy.mockRestore()
await handle.dispose()
await ctx.fiber.dispose()
})
it('a duplicate Result after the terminal claim cannot repeat cleanup or rewrite the outcome', async () => {
let explicitCancels = 0
let signalAborts = 0
const { ctx, parent, provider } = await setup({
manual: true,
onChildCancel: (_reason, index) => { if (index === 0) explicitCancels += 1 },
onChildAbortString: (_reason, index) => { if (index === 0) signalAborts += 1 },
})
const handle = ctx.workflows.start({
...scripted("agent('stray')\nawait new Promise(() => {})"),
@@ -1005,266 +850,9 @@ describe('dsh-workflow-workerthread', () => {
})
await expect(handle.result).resolves.toMatchObject({ value: 'first', stopReason: 'completed' })
expect(explicitCancels).toBe(1)
expect(signalAborts).toBe(1)
await handle.dispose()
expect(explicitCancels).toBe(1)
await ctx.fiber.dispose()
})
it('contains a throwing child cancel and still settles after cancelling peer strays', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
let starts = 0
const cancelled: string[] = []
const warnings: string[] = []
ctx.logger.warn = ((message: unknown) => { warnings.push(String(message)) }) as typeof ctx.logger.warn
const provider: SubagentProvider = {
name: 'throwing-cancel',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => {
const index = starts++
return {
id: AgentId(`throwing-cancel-${index}`),
started: new Promise(() => { /* readiness stays pending */ }),
result: new Promise(() => { /* cancellation callback owns settlement */ }),
cancel: (reason?: string) => {
if (index === 0) throw new Error('cancel callback broke')
cancelled.push(`${index}:${reason ?? 'cancelled'}`)
},
dispose: () => Promise.resolve(),
}
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, { provider: 'throwing-cancel', maxConcurrentAgents: 2 })
const handle = ctx.workflows.start({
...scripted(`
agent('first stray')
agent('second stray')
return 'done'
`),
parent: fakeParent(),
})
const result = await handle.result
expect(result.stopReason).toBe('completed')
expect(starts).toBe(2)
expect(cancelled).toContain('1:workflow settled')
expect(warnings.some(message => message.includes('cancel callback broke'))).toBe(true)
await handle.dispose()
})
it("cancel() drives each child's explicit cancel() host-side: a wedged worker cannot delay it", async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
let starts = 0
const cancelled: string[] = []
const provider: SubagentProvider = {
name: 'cancel-only',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => {
starts += 1
return {
id: AgentId('cancel-only-child'),
started: Promise.resolve(),
result: new Promise(() => { /* only cancel() ends this child */ }),
// Deliberately ignores the request signal — the seam leaves a
// provider free to honor ONLY the explicit cancel() channel.
cancel: (reason?: string) => { cancelled.push(reason ?? 'cancelled') },
dispose: () => Promise.resolve(),
}
},
}
ctx.subagents.registerProvider(provider)
// A deliberately huge grace: if only the grace/terminate reap could
// reach this child, the assertion below would time out first.
await ctx.plugin(WorkerWorkflowEngine, { provider: 'cancel-only', maxConcurrentAgents: 2, disposeGraceMs: 30_000 })
const handle = ctx.workflows.start({
// The stray child's start RPC reaches the host, then the script wedges
// its own worker in a synchronous spin: the worker cannot process the
// Cancel message, so it can relay NO ChildCancel RPC — only the host's
// own children loop can deliver the explicit cancel in time. The
// microtask yields let the agent() continuation POST its child-start
// before the spin seizes the worker's loop (the posted message needs
// no further worker-loop turns to reach the host).
...scripted(`
agent('wedged child')
for (let i = 0; i < 20; i++) await null
const end = Date.now() + 1500
while (Date.now() < end) {}
return 'raced'
`),
parent: fakeParent(),
})
await waitFor(() => { expect(starts).toBe(1) })
handle.cancel('stop now')
await waitFor(() => { expect(cancelled).toEqual(['stop now']) }, 800)
// The wedged worker's own completion loses to the in-flight cancel.
const result = await handle.result
expect(result.stopReason).toBe('cancelled')
await handle.dispose()
}, 15_000)
it.each(['fulfills', 'rejects'] as const)('provider.start() reentrant cancellation refuses the run when readiness later %s', async (readinessOutcome) => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const readiness = Promise.withResolvers<undefined>()
let starts = 0
let explicitCancels = 0
let disposals = 0
let sawAbortedSignal = false
const lifecycle: string[] = []
const provider: SubagentProvider = {
name: 'start-reentry',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: (request) => {
starts += 1
// This arbitrary provider callback runs before onChildStart can put
// the returned run in its registry. Cancellation must be rechecked
// after return instead of trusting the pre-start admission check.
handle.cancel('provider start reentered cancellation')
sawAbortedSignal = request.signal?.aborted === true
return {
id: AgentId('start-reentry-child'),
started: readiness.promise,
result: new Promise(() => { /* refusal owns teardown */ }),
// Deliberately honors only the explicit channel. It must still be
// reached promptly even though the first host fanout saw no run.
cancel: () => { explicitCancels += 1 },
dispose: () => {
disposals += 1
return Promise.resolve()
},
}
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, {
provider: 'start-reentry',
maxConcurrentAgents: 2,
disposeGraceMs: 30_000,
})
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', () => { lifecycle.push('end') })
const handle = ctx.workflows.start({
...scripted("await agent('reentrant provider')\nreturn 'unreachable'"),
parent: fakeParent(),
})
await waitFor(() => { expect(starts).toBe(1) })
// Either later readiness settlement must not answer the already-refused
// start again or emit a workflow lifecycle pair.
if (readinessOutcome === 'fulfills') readiness.resolve(undefined)
else readiness.reject(new Error('late readiness rejection after refusal'))
let result: WorkflowResult | undefined
void handle.result.then((value) => { result = value })
await waitFor(() => {
expect(explicitCancels).toBe(1)
expect(disposals).toBe(1)
expect(result?.stopReason).toBe('cancelled')
}, 1000)
expect(sawAbortedSignal).toBe(true)
expect(lifecycle).toEqual([])
await handle.dispose()
expect(explicitCancels).toBe(1)
expect(disposals).toBe(1)
await ctx.fiber.dispose()
})
it('claims workflow and child disposal before a raw provider disposer reenters handle.dispose()', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const terminal = Promise.withResolvers<SubagentResult>()
const observed: { reentrant?: Promise<void> } = {}
let starts = 0
let rawDisposeCalls = 0
const provider: SubagentProvider = {
name: 'dispose-reentry',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => {
starts += 1
return {
id: AgentId('dispose-reentry-child'),
started: Promise.resolve(),
result: terminal.promise,
cancel: () => { terminal.resolve({ output: [], stopReason: 'aborted' }) },
dispose: () => {
rawDisposeCalls += 1
observed.reentrant = handle.dispose()
return Promise.resolve()
},
}
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, { provider: 'dispose-reentry', maxConcurrentAgents: 2 })
const handle = ctx.workflows.start({
...scripted("await agent('live child')\nreturn 'unreachable'"),
parent: fakeParent(),
})
await waitFor(() => { expect(starts).toBe(1) })
const disposal = handle.dispose()
expect(observed.reentrant).toBe(disposal)
await disposal
expect(rawDisposeCalls).toBe(1)
await expect(handle.result).resolves.toMatchObject({ stopReason: 'cancelled' })
await ctx.fiber.dispose()
})
it('claims worker-originated child disposal before its raw disposer reenters holder disposal', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const terminal = Promise.withResolvers<SubagentResult>()
const observed: { reentrant?: Promise<void> } = {}
let starts = 0
let rawDisposeCalls = 0
const provider: SubagentProvider = {
name: 'child-dispose-reentry',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: () => {
starts += 1
return {
id: AgentId('child-dispose-reentry-child'),
started: Promise.resolve(),
result: terminal.promise,
cancel: () => { terminal.resolve({ output: [], stopReason: 'aborted' }) },
dispose: () => {
rawDisposeCalls += 1
// This begins holder disposal from the worker's ChildDispose
// callback, before any public handle.dispose() call exists.
observed.reentrant = handle.dispose()
return Promise.resolve()
},
}
},
}
ctx.subagents.registerProvider(provider)
await ctx.plugin(WorkerWorkflowEngine, { provider: 'child-dispose-reentry', maxConcurrentAgents: 2 })
const handle = ctx.workflows.start({
...scripted("return await agent('settling child')"),
parent: fakeParent(),
})
const finishChildSpy = vi.spyOn(handle as unknown as {
finishChild(callId: number): void
}, 'finishChild')
await waitFor(() => { expect(starts).toBe(1) })
terminal.resolve({ output: [{ type: 'text', text: 'done' }], stopReason: 'completed' })
await waitFor(() => { expect(observed.reentrant).toBeDefined() }, 1000)
await observed.reentrant
expect(rawDisposeCalls).toBe(1)
expect(finishChildSpy.mock.calls.filter(([callId]) => callId === 1)).toHaveLength(1)
finishChildSpy.mockRestore()
await expect(handle.result).resolves.toMatchObject({ stopReason: 'cancelled' })
expect(signalAborts).toBe(1)
await ctx.fiber.dispose()
})
@@ -1466,29 +1054,21 @@ describe('dsh-workflow-workerthread', () => {
await ctx.plugin(SubagentService)
// The child's dispose() REJECTS on top of the worker death: the reap
// must contain it (warn, not crash) while still emptying the registry.
const cancelled: string[] = []
const signalAborts: unknown[] = []
const provider: SubagentProvider = {
name: 'doomed',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: false },
inheritsParentContext: false,
start: (request) => {
request.signal?.addEventListener('abort', () => {
signalAborts.push(request.signal?.reason)
start: async (request) => {
request.signal.addEventListener('abort', () => {
signalAborts.push(request.signal.reason)
// The death claim precedes the shared-signal fanout. This
// synchronous callback cannot turn death into cancellation.
handle.cancel('reentered from worker-death signal cleanup')
}, { once: true })
return {
id: AgentId('doomed-child'),
started: Promise.resolve(),
result: new Promise(() => { /* never settles; the reap is the teardown */ }),
cancel: (reason?: string) => {
cancelled.push(reason ?? 'cancelled')
// Exercise the later microtask case too: terminal ownership
// remains closed after the death callback returns.
queueMicrotask(() => { handle.cancel('reentered from worker-death child cleanup') })
},
dispose: () => Promise.reject(new Error('dispose exploded during reap')),
}
},
@@ -1521,7 +1101,6 @@ describe('dsh-workflow-workerthread', () => {
// cold-start race; tight explicit bound (see the helper's doc comment).
await waitFor(() => {
expect(signalAborts).toEqual(['workflow worker gone'])
expect(cancelled).toEqual(['workflow worker gone'])
}, 1000)
await Promise.resolve()
expect(result.stopReason).toBe('error')
@@ -1647,14 +1226,14 @@ describe('dsh-workflow-workerthread', () => {
})
describe('service surface', () => {
it('run ids are unique per start; the run handle and event payloads hold SEPARATE meta clones', async () => {
it('run ids are unique and lifecycle meta is the run\'s borrowed immutable value', async () => {
const { ctx, parent } = await setup()
let eventMeta: WorkflowRunInfo | undefined
ctx.on('workflow/start', (info) => { eventMeta = info })
const first = ctx.workflows.start({ ...scripted('return 1'), parent })
const second = ctx.workflows.start({ ...scripted('return 2'), parent })
expect(first.id).not.toBe(second.id)
eventMeta!.meta.name = 'corrupted'
expect(eventMeta!.meta).toBe(second.meta)
expect(second.meta.name).toBe('test-flow')
await Promise.all([first.result, second.result])
await first.dispose()