Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

workflow: simplify to the trust premise; settle result on cancellation

Browse Source 
Two review responses that belong together — the same review argued the
engine was defending the wrong threat while a benign-input bug wedged
the product.

1) Drop hostile-value containment; state the trust premise.

Scripts are model-written — the same trust level as the model's bash
access — yet successive pre-push review rounds had ratcheted in defenses
that only matter against an adversarial author: trap-free proxy
rejection, accessor-never-invoked descriptor walks, realm-side
pre-rendering of thrown values, realm-built promises/arrays/error clones
with structural fatal recognition. That same author keeps a documented,
accepted, unkillable event-loop spin, so containing its error VALUES is
cost without a threat model — and the planned hardened engine
(worker/isolated-vm) gets value isolation by serialization and deletes
all of this machinery anyway.

What stays, because benign scripts hit it constantly: result never
rejects; dropped hook promises cannot become unhandled rejections; the
value boundary rejects LOUD everything JSON cannot carry (now a plain
recursive walk — getters are read ordinarily and their result is what
crosses; a throwing read fails loud); a "__proto__" key still copies as
a data property; the fatal-vs-null combinator discipline (now host
instanceof — unforgeable from the realm and simpler than clone-shape
recognition). What changes for scripts (documented in the engine
README): hooks hand back host values and host errors — in-script
`instanceof Error` on a hook failure is false (branch on e.name/e.code)
— and args are host-cloned once so a script cannot mutate the caller's
object. realm.ts drops 289 → 173 lines; the hostile-value test tables go
with it. The premise now leads the engine module doc, the README, and
the RFC's engine section, with the removed machinery recorded under
What was rejected.

2) result settles within the dispose grace of a cancellation.

Review finding (verified through the real registry + tool + engine): a
script parked on a promise no hook owns — `await new Promise(() => {})`,
`await Promise.race([])`, a returned never-settling thenable — could not
be settled by cancel(): hooks reject and children abort, but nothing
touches a promise the engine does not own, so `result` stayed pending
FOREVER (the previous cut even pinned that as intended). The tool awaits
run.result BEFORE its disposing finally, the registry awaits the tool,
the loop awaits the registry — one such script wedged the whole agent
turn past any abort, unrecoverable in-process; the mock engine in the
tool's abort test settles result on cancel, which is exactly the
behavior the real engine lacked, so no existing test could see it.

The seam contract now says it out loud: once a run is cancelled, result
SETTLES within the implementation's bounded grace even if the script
never does. The vm engine arms an abandon channel in cancel(); drive()
races the script against it, force-settling 'cancelled' at the grace
(the abandoned settlement stays contained; a post-slice synchronous spin
remains the documented limitation). dispose()'s outer race now exists
for child quiescence only, and `workflow/end` again fires exactly once
per started run. The old 'result stays pending' pin is FLIPPED to the
new contract (the pinned behavior was the bug); new regressions cover
cancel-then-settle on a parked script, a never-settling returned
thenable, and the full composition through the REAL registry + tool +
vm engine (tool-workflow gains workflow-vm/subagent devDeps for it).
agentsStarted JSDoc clarified while touching the vocabulary (accepted
calls, including ones still queued at cancellation).
This commit is contained in:
Tianyi Cui
2026-07-06 00:48:49 +08:00
parent 7234d41b91
commit 2accf85714
16 changed files with 352 additions and 609 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
packages/workflow/workflow-vm/src/index.ts
View File

@@ -4,25 +4,33 @@
* body in a fresh in-process vm context with the workflow hooks injected, and
* fans `agent()` calls out to `ctx.subagents`.
*
* Engine limitations, documented as the accepted cost of the in-process
* mechanism (the interface/implementation seam exists precisely so a
* worker-thread or isolated-vm engine can swap in if these ever matter):
* TRUST PREMISE: scripts are MODEL-WRITTEN — the same trust level as the
* model's existing bash access — so this engine defends against BUGGY
* scripts, never hostile ones. vm is NOT a security boundary and no attempt
* is made to contain adversarial values (see ./realm.ts); genuine sandboxing
* is an engine swap behind the seam (worker-thread/isolated-vm), not
* incremental host-side defenses here.
*
* - vm is NOT a security boundary. Scripts are model-written — the same trust
* level as the model's bash access — and the realm-boundary materialization
* is correctness containment, not a sandbox.
* - The vm `timeout` covers only the initial SYNCHRONOUS slice of the script;
* realm code that runs past that slice — an await continuation, a
* thenable's `then` invoked by promise resolution (including one the script
* RETURNS: a returned thenable resolves per JavaScript semantics before
* materialization, which is what makes an un-awaited `return agent('x')`
* work) — is beyond the timeout, so a pathological synchronous spin there
* cannot be killed in-process. `dispose()` waits a bounded grace for the
* script to settle AND its children (stray `agent()` calls included) to
* finish disposing, then ABANDONS whatever is left: pending hook promises
* are already rejected and the script's settlement is contained (no
* unhandled rejection), but an abandoned synchronous spin would still
* occupy the event loop.
* Engine limitations, documented as the accepted cost of the in-process
* mechanism:
*
* - `start()` runs the script's initial SYNCHRONOUS slice inline, so the
* CALLER blocks on the host event loop until the script's first await (or
* the vm `timeout` kills the slice); the meta-literal evaluation has its
* own timeout budget on the same call.
* - The vm `timeout` covers only that initial slice; realm code running past
* it — an await continuation, a thenable's `then` invoked by promise
* resolution (including one the script RETURNS: a returned thenable
* resolves per JavaScript semantics before materialization, which is what
* makes an un-awaited `return agent('x')` work) — is beyond the timeout, so
* a synchronous spin there cannot be killed in-process, and neither can
* script code the host invokes while rendering a failure (a getter on a
* thrown value). `dispose()` waits a bounded grace for the script to settle
* AND its children (stray `agent()` calls included) to finish disposing,
* then ABANDONS whatever is left: pending hook promises are already
* rejected and the script's settlement is contained (no unhandled
* rejection), but an abandoned synchronous spin would still occupy the
* event loop.
*
* Plugin export shape: a default-exported {@link WorkflowService} subclass
* (the class-based service form, like `dsh-bash-local`).
@@ -55,7 +63,11 @@ export interface Config {
maxItemsPerCall?: number
/** vm timeout for the script's initial synchronous slice AND the meta-literal evaluation (default 5000 ms). */
syncTimeoutMs?: number
/** How long `dispose()` waits for a cancelled script to settle before abandoning it (default 5000 ms). */
/**
* How long after a cancellation an unsettled script may keep running before
* it is abandoned and `result` force-settles `cancelled` (default 5000 ms);
* also bounds `dispose()`.
*/
disposeGraceMs?: number
}
@@ -110,6 +122,7 @@ export class VmWorkflowEngine extends WorkflowService {
maxTotalAgents: this.config.maxTotalAgents,
maxItemsPerCall: this.config.maxItemsPerCall,
syncTimeoutMs: this.config.syncTimeoutMs,
disposeGraceMs: this.config.disposeGraceMs,
}
const execution = new WorkflowExecution(
this.ctx,
@@ -149,9 +162,11 @@ export class VmWorkflowEngine extends WorkflowService {
},
dispose: (): Promise<void> => {
// Idempotent: cancel, then wait min(settle + child quiescence, grace).
// `result` and `quiesce()` never reject, so the race needs no
// rejection handling; a script or child still unsettled past the grace
// is abandoned per the module contract.
// The cancel itself bounds `result` (the execution abandons a script
// still unsettled `disposeGraceMs` later), so this outer race exists
// for CHILD quiescence: a slow-disposing child must not hold dispose
// past the grace. `result` and `quiesce()` never reject, so the race
// needs no rejection handling.
disposed ??= (async () => {
execution.cancel('workflow disposed')
await Promise.race([

15
packages/workflow/workflow-vm/src/meta.ts
View File

@@ -20,7 +20,7 @@
import * as vm from 'node:vm'
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
import type { WorkflowMeta, WorkflowPhase } from '@deepseek-ai/dsh-workflow'
import { materializeFromRealm, MaterializeError, describeThrown, thrownRendering, REALM_THROWN_RENDERER_SOURCE } from './realm.ts'
import { materializeFromRealm, MaterializeError, renderThrown } from './realm.ts'
/** The result of {@link extractMeta}: the validated meta and the runnable body. */
export interface ExtractedScript {
@@ -171,18 +171,11 @@ export function extractMeta(script: string, evalTimeoutMs: number): ExtractedScr
// An EMPTY context: any non-literal reference (a variable, a call) throws
// here. The result — data only — is what the contract checks; a getter or
// IIFE can still run, which is why the timeout and the materialization
// below are part of the same boundary. A thrown value is pre-rendered by
// the realm-side catch INSIDE the timed window, so a hostile
// stack/message/toString can neither run on the host catch path nor
// outlive the timeout.
evaluated = vm.runInNewContext(
`(() => { try { return (${literal}) } catch (e) { throw (${REALM_THROWN_RENDERER_SOURCE})(e) } })()`,
undefined,
{ timeout: evalTimeoutMs },
)
// below are part of the same boundary.
evaluated = vm.runInNewContext(`(${literal})`, undefined, { timeout: evalTimeoutMs })
} catch (error: unknown) {
throw new WorkflowError(
`meta block failed to evaluate as a pure literal: ${thrownRendering(error) ?? describeThrown(error)}`,
`meta block failed to evaluate as a pure literal: ${renderThrown(error)}`,
'META_INVALID',
{ cause: error },
)

246
packages/workflow/workflow-vm/src/realm.ts
View File

@@ -1,48 +1,34 @@
/**
* Realm-boundary materialization for the vm engine.
* The vm engine's value boundary: copy script-realm values into plain host
* JSON data — loud about everything JSON cannot carry — and render thrown
* script values to failure text.
*
* Values produced INSIDE the script realm (the meta literal, hook arguments,
* the script's return value) must become plain host-realm JSON data before the
* host touches them. The repo's `isJsonValue` guard cannot run first: it is
* prototype-strict (any cross-realm object fails it) and it INVOKES getters
* (letting realm code run outside the vm's timed window). So this module walks
* own-property DESCRIPTORS — never invoking accessors — and copies data into
* host containers, rejecting loud everything JSON cannot carry:
* accessor properties, non-plain prototypes, functions, symbols (keys or
* values), bigints, non-finite numbers, `undefined` values, cycles, sparse
* arrays, arrays with non-index own properties, and proxies. Proxies are
* rejected via the trap-free native `util.types.isProxy` check BEFORE any
* other inspection — a descriptor walk over a proxy would otherwise run its
* realm-side traps (`ownKeys`, `getOwnPropertyDescriptor`, `getPrototypeOf`)
* on the host stack, outside the vm's timed window, and a throwing trap would
* escape as a raw realm error instead of a {@link MaterializeError}. The same
* check guards the PROTOTYPE position (an object whose prototype is a proxy).
* TRUST PREMISE (everything in this module hangs on it): workflow scripts are
* MODEL-WRITTEN, the same trust level as the model's existing bash access, so
* this boundary guards against BUGGY scripts, not hostile ones. It rejects
* loud what JSON would silently mangle — functions, symbols, bigints,
* non-finite numbers, nested `undefined`, cycles, sparse arrays, exotic
* prototypes — because accepted-then-ignored is this repo's banned failure
* mode. It does NOT defend against adversarial values: the walk reads
* properties ordinarily (a getter runs, and whatever it returns is what
* crosses), {@link renderThrown} reads `stack`/`message`/`String()` directly,
* and a proxy is walked through its traps. A hostile script gains nothing
* worth defending here — it can already occupy the event loop forever with a
* synchronous spin past the first await (the engine's documented, accepted
* limitation) — so host-side hostile-value containment would be cost without
* a threat model; genuine hardening is an ENGINE SWAP (worker/isolated-vm,
* where the boundary is serialization by construction), not incremental
* defenses here.
*
* Host objects are built with `Object.defineProperty` into a fresh `{}` —
* never plain `target[key] =` assignment, which a `"__proto__"` key would turn
* into prototype mutation instead of a data property.
*
* The host→realm direction deliberately does NOT live here: a host object
* handed into the realm would expose host intrinsics through its prototype
* chain, so the engine rebuilds inbound values INSIDE the realm via the
* context's own `JSON.parse` (see the runtime).
*
* {@link REALM_THROWN_RENDERER_SOURCE}, {@link thrownRendering}, and
* {@link describeThrown} are the same discipline for the one place realm
* values reach the host WITHOUT materialization: a thrown value crossing into
* a host catch block. The renderer runs INSIDE the realm's own execution
* window (compiled into the script wrapper), so reading a hostile
* accessor/`toString` there is subject to the vm sync-slice timeout exactly
* like any other script code; the host side only descriptor-reads the
* pre-rendered string, or falls back to {@link describeThrown}, which invokes
* no getter whose function identity is not the host realm's own native stack
* getter.
* The host→realm direction needs no machinery at all: hooks hand the script
* plain host values, host prototypes included — the script is trusted. One
* consequence is documented in the engine README: an error thrown by a hook
* is a HOST error, so an in-script `instanceof Error` check is false; read
* `name`/`code`/`message` instead.
*
* @module @deepseek-ai/dsh-workflow-vm/realm
*/
import { types } from 'node:util'
/** Thrown by {@link materializeFromRealm}; the caller wraps it into the right `WorkflowError` code. */
export class MaterializeError extends Error {
constructor(public readonly path: string, public readonly reason: string) {
@@ -52,154 +38,63 @@ export class MaterializeError extends Error {
}
/**
* Realm-SOURCE text (an arrow-function expression) the engine compiles into
* its script wrappers: `throw (RENDERER)(e)` inside a catch around the whole
* body/literal. It renders the thrown value to a string INSIDE the realm's
* own execution window — a hostile `stack`/`message` accessor or `toString`
* invoked here is subject to the vm sync-slice timeout like any other script
* code (and post-await it is the engine's accepted spin limitation, identical
* to a script reading `e.stack` in its own catch). Host `WorkflowError`s
* thrown by hooks pass through unwrapped (duck-checked by name — a realm
* forgery fails the host's `instanceof` and merely renders data-only);
* everything else becomes `{ __wfThrown: <string> }`, whose only consumer is
* {@link thrownRendering}. Every read is individually contained, so the
* renderer itself never throws.
*/
export const REALM_THROWN_RENDERER_SOURCE = `(e) => {
try { if (e && e.name === 'WorkflowError') return e } catch { /* hostile name getter: fall through to rendering */ }
const rendered = (() => {
try { if (e && typeof e.stack === 'string' && e.stack.length > 0) return e.stack } catch { /* hostile stack getter */ }
try { if (e && typeof e.message === 'string') return e.message } catch { /* hostile message getter */ }
try { return String(e) } catch { /* hostile toString/Symbol.toPrimitive */ }
return '[unrenderable thrown value]'
})()
return { __wfThrown: rendered }
}`
/**
* The pre-rendered failure text carried by a realm-catch wrapper object
* (`{ __wfThrown: string }` from {@link REALM_THROWN_RENDERER_SOURCE}), or
* `undefined` when `error` is not such a wrapper. Descriptor-read and
* proxy-guarded: never invokes user code.
* @param error - the value a host catch received from script execution.
* @returns the realm-rendered string, or `undefined` to fall back to
* {@link describeThrown}.
*/
export function thrownRendering(error: unknown): string | undefined {
if (typeof error !== 'object' || error === null || types.isProxy(error)) return undefined
const value = ownDataProperty(error, '__wfThrown')
return typeof value === 'string' ? value : undefined
}
/**
* The host realm's own native `stack` getter (modern V8 makes `stack` an own
* ACCESSOR on Errors); `undefined` where it is a data property. Typed through
* a structural view of the descriptor — it is only ever identity-compared or
* `.call`ed on an explicit receiver, never invoked unbound.
*/
const HOST_STACK_GETTER: unknown = (Object.getOwnPropertyDescriptor(new Error(), 'stack') as { get?: unknown } | undefined)?.get
/**
* Render a thrown value HOST-SIDE without ever throwing and without running
* any code the host does not own: proxies become a fixed label (trap-free
* `isProxy` before any inspection); `stack` is read as an own data descriptor,
* or through its getter ONLY when that getter's function identity is the host
* realm's own native stack getter (an unforgeable check — realm code cannot
* hold that identity, and the host realm's `prepareStackTrace` is the host's
* own trust domain); `message` is an own-data read; anything else
* object-shaped renders as `[object Object]` untouched; only primitives
* (which cannot carry code) reach `String()`. Used for host-thrown errors
* (vm timeouts, `WorkflowError`s) and as the fallback for adversarial values
* that bypassed the realm-side renderer (e.g. a hostile thenable rejection);
* ordinary script failures arrive pre-rendered via {@link thrownRendering}.
* Render a thrown value to failure text without ever throwing: prefer the
* `stack` (host or realm — a realm error's `stack` is a plain string read),
* fall back to `message`, then `String()`. Reading those properties MAY run
* script code (a getter, `toString`) — accepted under the module's trust
* premise; if that code itself throws, a fixed label is returned instead.
* @param error - the thrown value, of any shape and any realm.
* @returns human-readable text for the failure report; prefers the stack.
*/
export function describeThrown(error: unknown): string {
switch (typeof error) {
case 'object':
break
case 'function':
return '[thrown function]'
default:
// Primitives (string/number/boolean/bigint/symbol/undefined): String()
// cannot reach user code on these.
return String(error)
export function renderThrown(error: unknown): string {
try {
const stack = (error as { stack?: unknown } | null | undefined)?.stack
if (typeof stack === 'string' && stack.length > 0) return stack
const message = (error as { message?: unknown } | null | undefined)?.message
if (typeof message === 'string' && message.length > 0) return message
return String(error)
} catch {
// A throwing accessor/toString on the thrown value — rendering must be
// total (drive()'s never-reject contract), so fall back to a fixed label.
return '[unrenderable thrown value]'
}
if (error === null) return 'null'
if (types.isProxy(error)) return '[thrown proxy]'
const stack = readStack(error)
if (typeof stack === 'string' && stack.length > 0) return stack
const message = ownDataProperty(error, 'message')
if (typeof message === 'string') return message
return '[object Object]'
}
/**
* Read `error.stack` without running foreign code: an own DATA descriptor is
* read directly; an accessor is invoked only on function identity with
* {@link HOST_STACK_GETTER} (never a realm or user function). The native
* getter returns `undefined` on a non-Error receiver rather than throwing.
*/
function readStack(error: object): unknown {
const descriptor = Object.getOwnPropertyDescriptor(error, 'stack')
if (descriptor === undefined) return undefined
if ('value' in descriptor) return descriptor.value
if (typeof descriptor.get !== 'function') return undefined
if (descriptor.get !== HOST_STACK_GETTER) return undefined
return descriptor.get.call(error)
}
/** An own DATA property's value (`undefined` for absent or accessor); never invokes user code on a non-proxy object. */
function ownDataProperty(value: object, key: string): unknown {
const descriptor = Object.getOwnPropertyDescriptor(value, key)
return descriptor !== undefined && 'value' in descriptor ? descriptor.value : undefined
}
/**
* Whether `error` is a FATAL realm-built `WorkflowError` clone — the shape the
* engine's hooks reject with (host errors are translated at the realm boundary
* so the script never holds host prototypes), duck-checked because a realm
* object cannot be an `instanceof` the host class. Proxy-guarded and
* descriptor-read, so a forged object cannot run code here; a script forging
* the shape only kills its own run (self-sabotage). Combinators use this to
* decide re-throw vs per-item `null`.
* @param error - the value a combinator caught from a realm thunk/stage.
* @returns `true` when the error must propagate and kill the script.
*/
export function isFatalWorkflowErrorClone(error: unknown): boolean {
if (typeof error !== 'object' || error === null || types.isProxy(error)) return false
return ownDataProperty(error, 'name') === 'WorkflowError' && ownDataProperty(error, 'fatal') === true
}
/**
* Whether an object's prototype chain is data-shaped: `null`, or a prototype
* whose own prototype is `null` (the realm's `Object.prototype` — which we
* cannot compare by identity across realms). A `Date`/`Map`/class instance
* has a longer chain and is rejected, as is a proxy sitting in the prototype
* position (checked trap-free BEFORE its own prototype is dereferenced).
* has a longer chain and is rejected.
*/
function hasPlainPrototype(value: object): boolean {
const proto: unknown = Object.getPrototypeOf(value)
if (proto === null) return true
if (types.isProxy(proto)) return false
return Object.getPrototypeOf(proto) === null
}
/**
* Copy `value` (typically from the vm realm) into plain host JSON data.
* Throws {@link MaterializeError} naming the offending path for anything JSON
* cannot carry losslessly. Accessors are detected via descriptors and NEVER
* invoked. `undefined` is accepted only at the ROOT (a script with no
* `return` value) — the caller decides what it means; an `undefined` nested
* INSIDE a container is a violation.
* cannot carry losslessly. Properties are read ordinarily — a getter runs and
* its RESULT is materialized; a read that throws surfaces as a
* {@link MaterializeError} carrying the rendered failure. `undefined` is
* accepted only at the ROOT (a script with no `return` value) — the caller
* decides what it means; an `undefined` nested INSIDE a container is a
* violation.
* @param value - the realm value to materialize.
* @param root - the path label for the root value (error messages).
* @returns the host-realm copy (plain objects/arrays/scalars only).
*/
export function materializeFromRealm(value: unknown, root = 'value'): unknown {
if (value === undefined) return undefined
return materialize(value, root, new Set())
try {
return materialize(value, root, new Set())
} catch (error: unknown) {
if (error instanceof MaterializeError) throw error
// A property read ran script code that threw; total-ize it so callers can
// keep the narrow MaterializeError contract.
throw new MaterializeError(root, `reading the value threw: ${renderThrown(error)}`)
}
}
function materialize(value: unknown, path: string, seen: Set<object>): unknown {
@@ -214,20 +109,15 @@ function materialize(value: unknown, path: string, seen: Set<object>): unknown {
case 'bigint':
throw new MaterializeError(path, 'bigints are not JSON data')
case 'function':
throw new MaterializeError(path, 'functions cannot cross the workflow realm boundary')
throw new MaterializeError(path, 'functions cannot cross the workflow value boundary')
case 'symbol':
throw new MaterializeError(path, 'symbols cannot cross the workflow realm boundary')
throw new MaterializeError(path, 'symbols cannot cross the workflow value boundary')
case 'undefined':
throw new MaterializeError(path, 'undefined is not JSON data')
case 'object':
break
}
if (value === null) return null
// BEFORE anything else touches the object: every inspection below —
// Array.isArray aside — can trigger a proxy trap, running realm code on the
// host stack (module doc). isProxy is a native internal-slot check (no
// traps, catches revoked proxies, realm-agnostic).
if (types.isProxy(value)) throw new MaterializeError(path, 'proxies cannot cross the workflow realm boundary')
const objectValue: object = value
if (seen.has(objectValue)) throw new MaterializeError(path, 'circular references are not JSON data')
seen.add(objectValue)
@@ -242,10 +132,8 @@ function materialize(value: unknown, path: string, seen: Set<object>): unknown {
function materializeArray(value: unknown[], path: string, seen: Set<object>): unknown[] {
const out: unknown[] = []
for (let index = 0; index < value.length; index++) {
const descriptor = Object.getOwnPropertyDescriptor(value, index)
if (descriptor === undefined) throw new MaterializeError(`${path}[${index}]`, 'sparse arrays are not JSON data')
if (!('value' in descriptor)) throw new MaterializeError(`${path}[${index}]`, 'accessor properties cannot cross the workflow realm boundary')
out.push(materialize(descriptor.value, `${path}[${index}]`, seen))
if (!(index in value)) throw new MaterializeError(`${path}[${index}]`, 'sparse arrays are not JSON data')
out.push(materialize(value[index], `${path}[${index}]`, seen))
}
// Own enumerable props beyond the indices (e.g. `arr.total = 3`) would be
// silently dropped by JSON — reject them instead.
@@ -256,7 +144,7 @@ function materializeArray(value: unknown[], path: string, seen: Set<object>): un
}
}
if (Object.getOwnPropertySymbols(value).length > 0) {
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow realm boundary')
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow value boundary')
}
return out
}
@@ -266,20 +154,16 @@ function materializeObject(value: object, path: string, seen: Set<object>): Reco
throw new MaterializeError(path, 'only plain objects and arrays are JSON data (exotic prototype)')
}
if (Object.getOwnPropertySymbols(value).length > 0) {
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow realm boundary')
throw new MaterializeError(path, 'symbol-keyed properties cannot cross the workflow value boundary')
}
const out: Record<string, unknown> = {}
for (const [key, descriptor] of Object.entries(Object.getOwnPropertyDescriptors(value))) {
// Non-enumerable own props never reach JSON output — skip them, matching
// JSON.stringify's contract exactly (documented in the module doc).
if (!descriptor.enumerable) continue
if (!('value' in descriptor)) {
throw new MaterializeError(`${path}.${key}`, 'accessor properties cannot cross the workflow realm boundary')
}
// Object.keys = own enumerable string keys, matching JSON.stringify's
// property selection exactly (non-enumerable props never reach JSON output).
for (const key of Object.keys(value)) {
// defineProperty, never assignment: a "__proto__" key must become an OWN
// data property of the copy, not a prototype mutation.
Object.defineProperty(out, key, {
value: materialize(descriptor.value, `${path}.${key}`, seen),
value: materialize((value as Record<string, unknown>)[key], `${path}.${key}`, seen),
enumerable: true,
writable: true,
configurable: true,

225
packages/workflow/workflow-vm/src/runtime.ts
View File

@@ -4,30 +4,27 @@
* concurrency semaphore and caps, cancellation, and the drive loop that turns
* a script settlement into a {@link WorkflowResult}.
*
* Realm discipline (see also ./realm.ts): values ENTERING the host from the
* script (hook options, schemas, the return value) are materialized via
* descriptor walks; values ENTERING the realm from the host (`args`, agent()
* results) are rebuilt INSIDE the realm through the context's own
* `JSON.parse`, so the script never holds an object whose prototype chain
* reaches host intrinsics. The same rule covers every other value a hook
* hands the script: the promises `agent`/`parallel`/`pipeline` return are
* realm promises (the realm's own `Promise.resolve` over the host promise),
* the arrays the combinators resolve to are realm-built (their ELEMENTS are
* realm values already — only the container needs rebuilding), and a hook
* failure — rejection or synchronous `phase`/`log` throw — crosses as a
* realm-built clone carrying name/code/message/fatal. Realm functions
* (pipeline stages, parallel thunks) are called, not materialized — their
* values stay realm-side.
* Value boundary (the trust premise lives in ./realm.ts): values ENTERING the
* host from the script (hook options, schemas, the return value) are
* materialized by `materializeFromRealm` — a plain walk that rejects loud
* everything JSON cannot carry. Values ENTERING the realm (`args`, `agent()`
* results, hook promises and their failures, combinator arrays) are handed
* over DIRECTLY as host values: the script is model-written and trusted, so
* host prototypes are not a leak. `args` is host-side `structuredClone`d once
* at start so a script scribbling on it cannot mutate the caller's object —
* that is a benign-bug guard, not isolation. Realm functions (pipeline
* stages, parallel thunks) are called, not materialized — their values stay
* realm-side until they cross through a hook or the final return.
*
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
* unsupported options/schemas, tripped caps, seam start failures,
* cancellation) ALWAYS propagate through `parallel`/`pipeline` — they cross
* the realm boundary as fatal clones, recognized structurally — and the
* per-item `null` is reserved for child-run failures and ordinary in-stage
* script errors. Every hook-returned promise gets a no-op rejection consumer
* attached, so a script that drops a promise (fires an `agent()` without
* awaiting it) cannot surface an unhandled rejection when cancellation
* rejects it — the app boot layer exits the process on unhandled rejections.
* cancellation) ALWAYS propagate through `parallel`/`pipeline` — recognized
* by host `instanceof`, which a script cannot forge — and the per-item `null`
* is reserved for child-run failures and ordinary in-stage script errors.
* Every hook-returned promise gets a no-op rejection consumer attached, so a
* script that drops a promise (fires an `agent()` without awaiting it) cannot
* surface an unhandled rejection when cancellation rejects it — the app boot
* layer exits the process on unhandled rejections.
*
* @module @deepseek-ai/dsh-workflow-vm/runtime
*/
@@ -39,14 +36,14 @@ import type { Agent } from '@deepseek-ai/dsh-agent'
import type {} from '@deepseek-ai/dsh-subagent'
import { assertSupportedOutputSchema, OutputSchemaError } from '@deepseek-ai/dsh-tools'
import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
import { isFatalWorkflowError, WorkflowError } from '@deepseek-ai/dsh-workflow'
import type {
WorkflowAgentEndInfo,
WorkflowAgentInfo,
WorkflowMeta,
WorkflowResult,
} from '@deepseek-ai/dsh-workflow'
import { materializeFromRealm, MaterializeError, describeThrown, thrownRendering, isFatalWorkflowErrorClone, REALM_THROWN_RENDERER_SOURCE } from './realm.ts'
import { materializeFromRealm, MaterializeError, renderThrown } from './realm.ts'
/** The per-run knobs the engine resolves from its Config. */
export interface ExecutionLimits {
@@ -60,6 +57,8 @@ export interface ExecutionLimits {
maxItemsPerCall: number
/** vm timeout for the script's initial synchronous slice. */
syncTimeoutMs: number
/** How long after `cancel()` a still-unsettled script is abandoned (result force-settles `cancelled`). */
disposeGraceMs: number
}
/** The engine-side observers the execution reports progress through. */
@@ -124,13 +123,24 @@ export class WorkflowExecution {
private readonly controller = new AbortController()
private currentPhase: string | undefined
private readonly context: vm.Context
private readonly realmJsonParse: (text: string) => unknown
private readonly realmArrayFrom: (items: unknown[]) => unknown[]
private readonly realmPromiseResolve: (value: unknown) => Promise<unknown>
private readonly realmErrorClone: (name: string, code: string | undefined, message: string, fatal: boolean) => unknown
private readonly compiled: vm.Script
/** Every live `agent()` call promise — awaited or stray — for {@link quiesce}. */
private readonly inFlightAgents = new Set<Promise<unknown>>()
/** Fires {@link abandoned}; assigned by the promise executor at field initialization. */
private declareAbandoned!: () => void
private abandonTimer: NodeJS.Timeout | undefined
/**
* Rejects `disposeGraceMs` after {@link cancel} if the script has not
* settled by then. `drive()` races the script against it, so `result`
* ALWAYS settles within the grace of a cancellation — even when the script
* is parked on a promise no hook owns (`await new Promise(() => {})`), which
* cancellation cannot reject. Without this, a consumer awaiting `result`
* before disposing (the tool's shape) would hang forever on such a script,
* wedging its caller past any abort.
*/
private readonly abandoned = new Promise<never>((_, reject) => {
this.declareAbandoned = () => { reject(new WorkflowError('workflow script abandoned after the cancellation grace', 'CANCELLED')) }
})
constructor(
private readonly ctx: Context,
@@ -144,61 +154,34 @@ export class WorkflowExecution {
) {
// Compile FIRST: a body syntax error must throw out of the constructor
// (the engine maps it to SCRIPT_PARSE) before any realm state exists.
// The body is wrapped in a realm-side catch that pre-renders any thrown
// value to a string (see REALM_THROWN_RENDERER_SOURCE) — rendering happens
// inside the realm's own execution window, never on a host catch path.
// lineOffset compensates for the wrapper line, so stack traces carry the
// script's own line numbers (the meta statement was blanked, not removed).
try {
this.compiled = new vm.Script(
`(async () => { try {\n${body}\n} catch (e) { throw (${REALM_THROWN_RENDERER_SOURCE})(e) } })()`,
{
filename: `workflow:${meta.name}`,
lineOffset: -1,
},
)
this.compiled = new vm.Script(`(async () => {\n${body}\n})()`, {
filename: `workflow:${meta.name}`,
lineOffset: -1,
})
} catch (error: unknown) {
throw new WorkflowError(`workflow script does not parse: ${String(error)}`, 'SCRIPT_PARSE', { cause: error })
}
this.context = vm.createContext({}, { name: `workflow:${meta.name}` })
vm.runInContext(DETERMINISM_PRELUDE, this.context)
// The realm's own JSON.parse — the host→realm rebuild channel.
const realmJson = vm.runInContext('JSON', this.context) as { parse(text: string): unknown }
this.realmJsonParse = (text: string) => realmJson.parse(text)
// The realm's own Array.from / Promise.resolve / an error factory, bound
// NOW so a script reassigning its globals later cannot swap them:
// combinator results must be realm arrays, hook promises realm promises,
// and hook failures realm-built clones.
this.realmArrayFrom = vm.runInContext('Array.from.bind(Array)', this.context) as (items: unknown[]) => unknown[]
this.realmPromiseResolve = vm.runInContext('Promise.resolve.bind(Promise)', this.context) as (value: unknown) => Promise<unknown>
this.realmErrorClone = vm.runInContext(`(name, code, message, fatal) => {
const error = new Error(message)
error.name = name
if (code !== undefined) error.code = code
error.fatal = fatal
return error
}`, this.context) as (name: string, code: string | undefined, message: string, fatal: boolean) => unknown
// A run that settles without ever being abandoned leaves `abandoned`
// permanently pending or rejecting into the void — consume it so a late
// grace timer cannot surface an unhandled rejection.
void this.contain(this.abandoned)
const globals: Record<string, unknown> = {
agent: (prompt: unknown, opts?: unknown) => this.realmFacing(this.track(this.agent(prompt, opts))),
parallel: (thunks: unknown) => this.realmFacing(this.parallel(thunks)),
pipeline: (items: unknown, ...stages: unknown[]) => this.realmFacing(this.pipeline(items, stages)),
phase: (title: unknown) => {
try {
this.phase(title)
} catch (error: unknown) {
throw this.toRealmError(error)
}
},
log: (message: unknown) => {
try {
this.log(message)
} catch (error: unknown) {
throw this.toRealmError(error)
}
},
args: this.toRealm(args),
agent: (prompt: unknown, opts?: unknown) => this.contain(this.track(this.agent(prompt, opts))),
parallel: (thunks: unknown) => this.contain(this.parallel(thunks)),
pipeline: (items: unknown, ...stages: unknown[]) => this.contain(this.pipeline(items, stages)),
phase: (title: unknown) => { this.phase(title) },
log: (message: unknown) => { this.log(message) },
// Host-side clone: a script scribbling on args must not mutate the
// caller's object (a benign-bug guard; args is plain JSON by the seam
// contract, so structuredClone is total here and throws loud otherwise).
args: args === undefined ? undefined : structuredClone(args),
}
for (const [key, value] of Object.entries(globals)) {
// Data properties on the contextified global; frozen shape not required —
@@ -226,7 +209,10 @@ export class WorkflowExecution {
/**
* Cancel the run: children abort (the shared signal), waiting `agent()`
* slots reject, and every future hook call throws `CANCELLED` — the script
* dies at its next await. Idempotent; the first reason wins.
* dies at its next await. A script that STILL has not settled after
* `disposeGraceMs` (parked on a promise no hook owns) is abandoned so
* `result` settles regardless (see {@link abandoned}). Idempotent; the
* first reason wins.
*/
cancel(reason?: string): void {
if (this.cancelReason !== undefined) return
@@ -234,14 +220,18 @@ export class WorkflowExecution {
this.cancelError = new WorkflowError(`workflow run cancelled: ${this.cancelReason}`, 'CANCELLED')
this.controller.abort(this.cancelReason)
for (const waiter of this.slotWaiters.splice(0)) waiter.reject(this.cancelledError())
this.abandonTimer = setTimeout(() => { this.declareAbandoned() }, this.limits.disposeGraceMs)
// unref'd: an armed grace timer must never hold the process open.
this.abandonTimer.unref()
}
/**
* Run the script to settlement. Resolves — never rejects — with the run's
* {@link WorkflowResult}: the materialized return value on `completed`, the
* failure message on `error`, and `cancelled` when the script died of
* cancellation. After settlement, any stray children a script fired without
* awaiting are aborted (their `agent()` wrappers dispose them).
* cancellation (or outlived its post-cancel grace and was abandoned — see
* {@link abandoned}). After settlement, any stray children a script fired
* without awaiting are aborted (their `agent()` wrappers dispose them).
*/
async drive(): Promise<WorkflowResult> {
try {
@@ -249,7 +239,9 @@ export class WorkflowExecution {
// the script must not execute at all, let alone report `completed`.
if (this.isCancelled()) throw this.cancelledError()
const scriptPromise = this.compiled.runInContext(this.context, { timeout: this.limits.syncTimeoutMs }) as Promise<unknown>
const raw: unknown = await this.contain(Promise.resolve(scriptPromise))
// The race is the result-settles-after-cancel guarantee: a parked
// script loses to the abandon channel once the grace expires.
const raw: unknown = await Promise.race([this.contain(Promise.resolve(scriptPromise)), this.abandoned])
// Cancelled while the body ran: a script that settled without touching
// another hook (or without any) must still report `cancelled` — the
// holder asked for cancellation and `completed` would be a lie.
@@ -258,25 +250,23 @@ export class WorkflowExecution {
return { value, stopReason: 'completed', agentsStarted: this.started }
} catch (error: unknown) {
// Any failure after cancel() reports `cancelled` with the canonical
// reason — the reject path mirrors the resolve path's post-settle
// check, and a hook CANCELLED failure crosses the realm boundary as a
// clone that deliberately fails the host `instanceof`.
// reason — the reject path mirrors the resolve path's post-settle check.
if (this.isCancelled()) {
return { value: null, stopReason: 'cancelled', error: this.cancelledError().message, agentsStarted: this.started }
}
// Ordinary script failures arrive pre-rendered by the realm-side catch
// (thrownRendering); host-thrown errors (a vm timeout, a WorkflowError)
// and adversarial values that bypassed the wrapper (e.g. a hostile
// thenable rejection) render via the total, host-code-only
// describeThrown. Neither path can throw — drive() resolving is the
// `result` never-rejects seam contract.
return { value: null, stopReason: 'error', error: thrownRendering(error) ?? describeThrown(error), agentsStarted: this.started }
// renderThrown is total (host- and realm-thrown values alike), so this
// arm cannot throw — drive() resolving is the `result` never-rejects
// seam contract.
return { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: this.started }
} finally {
// Reap strays: a script that fired agent() calls without awaiting them
// leaves live children behind after settlement — abort them all. (The
// per-call wrappers dispose each child; the contain() consumer keeps
// their rejections from going unhandled.)
if (this.cancelReason === undefined) this.cancel('workflow settled')
// drive() settling means nothing is left to abandon — including the
// timer the self-cancel above just armed.
if (this.abandonTimer !== undefined) clearTimeout(this.abandonTimer)
}
}
@@ -291,37 +281,6 @@ export class WorkflowExecution {
return promise
}
/**
* Hand a hook's host promise to the script as a REALM promise (the realm's
* own `Promise.resolve` assimilates it) whose failure reason is a
* realm-built clone — the script must never hold host prototypes, and both
* the promise object and a caught rejection would otherwise expose them
* (module doc). The realm promise gets the same no-op rejection consumer as
* {@link contain}, since the script may drop it; the intermediate host
* promises are handled by the assimilation chain itself.
*/
private realmFacing(hostPromise: Promise<unknown>): Promise<unknown> {
const translated = hostPromise.catch((error: unknown) => {
throw this.toRealmError(error)
})
const realmPromise = this.realmPromiseResolve(translated)
realmPromise.catch(() => { /* consumed: a script-dropped realm promise must not surface an unhandled rejection (see contain) */ })
return realmPromise
}
/**
* Rebuild a host failure as a realm-built error clone: a `WorkflowError`
* keeps its name/code/message/fatal (the combinators recognize the shape
* via {@link isFatalWorkflowErrorClone}); anything else becomes a generic
* realm `Error` carrying its {@link describeThrown} rendering.
*/
private toRealmError(error: unknown): unknown {
if (error instanceof WorkflowError) {
return this.realmErrorClone('WorkflowError', error.code, error.message, error.fatal)
}
return this.realmErrorClone('Error', undefined, describeThrown(error), false)
}
/**
* Register one `agent()` call promise for {@link quiesce} tracking; the
* entry drops when the call fully settles (which is AFTER its child's
@@ -354,14 +313,6 @@ export class WorkflowExecution {
return this.cancelError ?? new WorkflowError('workflow run cancelled', 'CANCELLED')
}
/** Rebuild a host value inside the script realm (via the realm's own JSON.parse). */
private toRealm(value: unknown): unknown {
if (value === undefined) return undefined
if (value === null) return null
if (typeof value === 'string' || typeof value === 'number' || typeof value === 'boolean') return value
return this.realmJsonParse(JSON.stringify(value))
}
/** Materialize the script's return value; violations become RESULT_UNSERIALIZABLE. */
private materializeResult(raw: unknown): unknown {
try {
@@ -453,7 +404,7 @@ export class WorkflowExecution {
return null
}
this.observer.agentEnd({ ...info, outcome: 'completed' })
return this.toRealm(result.structured)
return result.structured
}
this.observer.agentEnd({ ...info, outcome: 'completed' })
return outputText(result.output)
@@ -532,20 +483,17 @@ export class WorkflowExecution {
}
return thunk as () => unknown
})
const settled = await Promise.all(thunks.map(async (thunk) => {
return Promise.all(thunks.map(async (thunk) => {
try {
return await thunk()
} catch (error: unknown) {
// Hooks translate host errors at the realm boundary, so a fatal error
// reaches a thunk catch only as a realm clone (a script forging the
// shape merely kills its own run).
if (isFatalWorkflowErrorClone(error)) throw error
// Hook failures are host WorkflowErrors; a fatal one is recognized by
// host `instanceof` — a script-built object can never pass it, so
// fatality cannot be forged (nor accidentally dissolved).
if (isFatalWorkflowError(error)) throw error
return null
}
}))
// The container must be a REALM array (module doc); the elements are
// realm values already.
return this.realmArrayFrom(settled)
}
/** The `pipeline(items, ...stages)` hook: per-item stage chains, NO cross-stage barrier. */
@@ -563,7 +511,7 @@ export class WorkflowExecution {
}
return stage as (previous: unknown, item: unknown, index: number) => unknown
})
const settled = await Promise.all(rawItems.map(async (item: unknown, index) => {
return Promise.all(rawItems.map(async (item: unknown, index) => {
let value: unknown = item
try {
for (const stage of stages) {
@@ -572,15 +520,12 @@ export class WorkflowExecution {
return value
} catch (error: unknown) {
// An ordinary stage throw drops the ITEM to null and skips its
// remaining stages; a fatal error (a realm clone — see parallel())
// kills the whole script.
if (isFatalWorkflowErrorClone(error)) throw error
// remaining stages; a fatal host WorkflowError (see parallel()) kills
// the whole script.
if (isFatalWorkflowError(error)) throw error
return null
}
}))
// The container must be a REALM array (module doc); the elements are
// realm values already.
return this.realmArrayFrom(settled)
}
private assertItemCap(length: number, hook: string): void {