Merge remote-tracking branch 'origin/master' into codex/simp-prune-workflow-worker-surface
# Conflicts: # docs/config-catalog.md
This commit is contained in:
@@ -1,25 +1,12 @@
|
||||
/**
|
||||
* The model-facing `workflow` tool: run a JavaScript orchestration script that
|
||||
* fans out subagents, and return the script's final value. Pure schema +
|
||||
* lifecycle shaping — script parsing, execution, caps, and cancellation live
|
||||
* behind `ctx.workflows` (`@deepseek-ai/dsh-workflow`), so a hardened engine
|
||||
* swaps in without touching what the model sees.
|
||||
*
|
||||
* Collection is SYNCHRONOUS this cut (like `dsh-tool-subagent`): `execute`
|
||||
* starts a run and awaits `run.result` inside a `try/finally` that always
|
||||
* disposes the run, so the script and its children are torn down on every
|
||||
* path. A non-`completed` stop reason maps to an `isError` tool result (by
|
||||
* throwing) rather than returning partial output as success. Background
|
||||
* collection is deferred to the cross-tool background redesign.
|
||||
*
|
||||
* Render intent (decided up front, per the render-intent RFC): a `generic`
|
||||
* card whose title carries the workflow's `meta.name`, read directly from the
|
||||
* call's `meta` parameter — presentation is a pure function of `args`.
|
||||
*
|
||||
* Usage policy ships with the tool as a `tool:<toolName>` system-prompt
|
||||
* section (explicit-ask-only guidance) — tool guidance lives in tool plugins,
|
||||
* never in the deployment persona.
|
||||
*
|
||||
* The model-facing `workflow` tool: run a JavaScript orchestration script that fans out
|
||||
* subagents, and return the script's final value. Pure schema + lifecycle shaping — script
|
||||
* parsing, execution, caps, and cancellation live behind `ctx.workflows`
|
||||
* (`@deepseek-ai/dsh-workflow`), so a hardened engine swaps in without touching what the model
|
||||
* sees. Execution awaits `run.result` and always disposes the run; non-completed reasons become tool
|
||||
* errors, and background collection remains deferred. Presentation is an args-only generic card
|
||||
* titled from `meta.name`. Explicit-ask usage guidance is registered as the tool's own prompt
|
||||
* section rather than deployment persona prose.
|
||||
* @module @deepseek-ai/dsh-tool-workflow
|
||||
*/
|
||||
|
||||
@@ -184,10 +171,9 @@ export function apply(ctx: Context, config: Config): void {
|
||||
...exec.signal ? { signal: exec.signal } : {},
|
||||
})
|
||||
|
||||
// Bridge the tool's abort signal to the run: if the parent step is
|
||||
// aborted while the script is in flight, cancel the whole run. The
|
||||
// engine also receives `signal` directly, but an explicit bridge keeps
|
||||
// the tool's contract local (and covers an engine that ignores it).
|
||||
// Bridge the tool's abort signal to the run: if the parent step is aborted while the
|
||||
// script is in flight, cancel the whole run. The signal also enters the engine directly, but
|
||||
// this local bridge preserves the tool contract even if an implementation ignores it.
|
||||
const onAbort = (): void => { run.cancel('parent step aborted') }
|
||||
exec.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
// `addEventListener` does NOT fire for a signal already aborted before
|
||||
|
||||
@@ -224,13 +224,8 @@ describe('dsh-tool-workflow', () => {
|
||||
|
||||
describe('composition with the REAL worker-thread engine (the mock above must stay honest)', () => {
|
||||
it('an abort releases the tool even when the script parks on a promise no hook owns', async () => {
|
||||
// Regression for the review-found turn wedge: the tool awaits
|
||||
// run.result BEFORE its disposing finally, the registry and the loop
|
||||
// await the tool — so if cancellation could not settle result (a script
|
||||
// parked on `await new Promise(() => {})`), an aborted turn stayed
|
||||
// wedged forever. The seam now guarantees result settles within the
|
||||
// grace of cancel(); this drives that guarantee through the real
|
||||
// registry + real tool + real engine.
|
||||
// The tool and loop await run.result before cleanup, so cancellation must settle a script
|
||||
// parked on an unowned promise. Exercise that guarantee through the real registry and worker.
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
|
||||
@@ -34,7 +34,7 @@ Unknown options, malformed arguments, unsupported schemas, tripped caps, provide
|
||||
|
||||
## Run sequence
|
||||
|
||||
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode uses a data-URL bootstrap that installs the TypeScript transforms inside the worker; built mode passes the sibling CommonJS bundle `lib/worker.cjs` as a filesystem string. CommonJS is required because pkg's VFS Worker hook compiles filesystem-string entries in that format; the same entry also works under ordinary Node resolution. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
|
||||
For each `agent()` call:
|
||||
|
||||
@@ -91,7 +91,7 @@ The host keeps a ledger of forwarded child starts. A graceful worker supplies th
|
||||
|
||||
### Parent tool result, indirectly
|
||||
|
||||
**What the model sees**: Through `dsh-tool-workflow`, success exposes only the materialized final JSON value and child count in that consumer's exact wrapper. An engine failure becomes exactly `Error: workflow run failed: <engine-error>`; stable engine-error shapes include `workflow script does not parse: <error>`, `invalid meta: <violations>`, `agent() requires a non-empty prompt string`, `agent() could not start a child: <error>`, `child agent run failed: <error>`, and the exact `parallel()`, `pipeline()`, `phase()`, option, schema, and JSON-boundary validation messages from this package. Intermediate child outputs are available to the script but not the parent model.
|
||||
**What the model sees**: Through [`dsh-tool-workflow`](../tool-workflow/README.md), success exposes only the materialized final JSON value and child count in that consumer's wrapper. This engine supplies stable errors including `workflow script does not parse: <error>`, `invalid meta: <violations>`, `agent() requires a non-empty prompt string`, `agent() could not start a child: <error>`, `child agent run failed: <error>`, and its exact `parallel()`, `pipeline()`, `phase()`, option, schema, and JSON-boundary validation messages. Intermediate child outputs are available to the script but not the parent model.
|
||||
|
||||
**Token effect**: Zero direct parent tokens from this engine. Final result size is capped by the tool consumer and retained until compaction.
|
||||
|
||||
|
||||
@@ -1,43 +1,8 @@
|
||||
/**
|
||||
* The host half of one worker-engine run: spawn the Worker, bridge its child
|
||||
* RPC onto the holder-bound subagent service, fan its observer messages into
|
||||
* the engine's events, and own cancellation, the settle-within-grace
|
||||
* guarantee, and child cleanup. The worker's lifetime IS the run's lifetime:
|
||||
* `dispose()` always ends with `worker.terminate()`, so no thread outlives its
|
||||
* run.
|
||||
*
|
||||
* The run's `result` promise settles exactly once, from whichever of these
|
||||
* lands first: receipt of the worker's `result` message, an unexpected worker
|
||||
* death (`error`/`messageerror`/premature `exit` → `stopReason: 'error'`, or
|
||||
* `'cancelled'` when a cancel was in flight), or the post-cancel grace timer (a
|
||||
* script that never settles is force-settled `cancelled` and its worker
|
||||
* terminated — the real kill an in-process engine could not perform). At
|
||||
* `result` receipt the host snapshots whether caller/signal/dispose
|
||||
* cancellation is already in flight: an earlier cancellation overrides a
|
||||
* non-cancelled report; otherwise the report wins before settlement-only child
|
||||
* cleanup invokes arbitrary provider callbacks. Worker death uses the same
|
||||
* boundary: it claims `error` (or a previously requested `cancelled`) before
|
||||
* reaping children, so cleanup callbacks cannot rewrite the outcome. That
|
||||
* first signal also closes inbound message admission: Node may emit `error`,
|
||||
* then deliver queued messages, then emit `exit`, but those late messages may
|
||||
* neither create work nor narrate after settlement. If Result or grace already
|
||||
* owns the outcome, death preserves it while still cleaning resources; the
|
||||
* eventual exit performs a final disposal-only sweep without repeating child
|
||||
* cancellation.
|
||||
*
|
||||
* Provider starts and published children are tracked separately. Every start
|
||||
* receives one shared per-run abort signal; the provider owns partial setup
|
||||
* until its promise fulfills. If admission closes while a start is pending,
|
||||
* the signal aborts it; a late fulfillment is disposed without publication to
|
||||
* the worker. Ready runs enter a callId registry whose memoized disposal is
|
||||
* shared by graceful worker RPC, public disposal, normal-settlement reap, and
|
||||
* worker-death cleanup. Quiescence requires both pending starts and published
|
||||
* children to drain. Lifecycle pairing is host-guaranteed independently:
|
||||
* every forwarded `agent-start` enters a ledger, and a dead or terminated
|
||||
* worker's missing `agent-end` is synthesized exactly once as cancelled. On a
|
||||
* termination path `agentsStarted` reports the host-observed child-start count;
|
||||
* calls still queued worker-side for a concurrency slot are unknowable.
|
||||
*
|
||||
* Host side of one workflow run. The first worker result, unexpected death, or
|
||||
* cancellation-grace expiry owns settlement and closes message admission.
|
||||
* Pending starts share one abort signal; published children share idempotent
|
||||
* cleanup, and quiescence waits for both while synthesizing any missing end events.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/host
|
||||
*/
|
||||
|
||||
@@ -64,29 +29,10 @@ interface ChildRecord {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the worker entry and spawn options for the current runtime shape.
|
||||
* Unbuilt (tsx demos, vitest — `import.meta.url` points into `src/`), the
|
||||
* entry is a JavaScript data-URL bootstrap. That bootstrap runs INSIDE the
|
||||
* user worker, registers tsx's ESM AND CommonJS transforms there, and only
|
||||
* then imports the TypeScript sibling. The whole mixed-module source graph
|
||||
* therefore receives TypeScript transformation and the tsconfig paths map in
|
||||
* the worker's own module-loader realm. A worker inherits no
|
||||
* transform pipeline from vitest (vite transforms in-process), and a parent
|
||||
* `--import tsx` registration is not a contract that user workers share on
|
||||
* every supported Node line. Built (`lib/index.js`), the entry is the sibling
|
||||
* bundle the package tsdown config emits and no loader is needed (`execArgv`
|
||||
* pinned empty in both shapes — hermetic, like the environment).
|
||||
*
|
||||
* Both shapes spawn with an EMPTY environment (`env: {}`): the documented vm
|
||||
* escape reaches `process`, and the harness's ambient credentials
|
||||
* (`DEEPSEEK_API_KEY` et al.) must not ride along — the same stance as
|
||||
* `dsh-code-runtime-worker`, stronger than the scrubbed env the
|
||||
* defensive-patterns rule requires for spawned commands (a shell needs PATH;
|
||||
* this worker needs nothing). Sole exception: the unbuilt shape forwards
|
||||
* `TSX_TSCONFIG_PATH` when the parent carries it (loader plumbing the paths
|
||||
* map depends on outside the repo cwd, not a secret). This closes the
|
||||
* AMBIENT channel only — an escapee still holds process-wide privileges
|
||||
* like fs access (the README's trust premise stands).
|
||||
* Resolve a built worker bundle or an unbuilt bootstrap that installs both tsx
|
||||
* transforms inside the worker. Both shapes clear `execArgv` and the ambient
|
||||
* environment; the unbuilt shape forwards only `TSX_TSCONFIG_PATH` for path
|
||||
* resolution.
|
||||
* @param init - the run payload, passed as `workerData`.
|
||||
* @returns the entry path or URL and the Worker options to spawn it with.
|
||||
*/
|
||||
@@ -95,15 +41,7 @@ function resolveWorkerSpawn(init: WorkerInit): { entry: string | URL; options: W
|
||||
if (!import.meta.url.endsWith('.ts')) {
|
||||
return { entry: fileURLToPath(new URL('./worker.cjs', import.meta.url)), options: { workerData: init, env: {}, execArgv: [] } }
|
||||
}
|
||||
// Resolve tsx lazily: only the unbuilt shape executes this arm, so a built
|
||||
// consumer never needs the dev-only loader installed. A JavaScript entry is
|
||||
// essential — it can install tsx's ESM and CommonJS hooks from INSIDE the
|
||||
// user worker before any TypeScript enters Node's native strip-only parser.
|
||||
// Both hooks are load-bearing because the source graph crosses both module
|
||||
// shapes on supported Node lines. TSX_TSCONFIG_PATH is
|
||||
// the one variable forwarded through the scrub: a parent running outside
|
||||
// the repo cwd (the ACP snapshot harness is the real case) pins the paths
|
||||
// map through it. Loader plumbing, not a secret.
|
||||
// Resolve tsx only for unbuilt consumers and install it before importing TS.
|
||||
const workerEntry = new URL('./worker.ts', import.meta.url)
|
||||
const tsxEsmApiEntry = import.meta.resolve('tsx/esm/api')
|
||||
const tsxCjsApiEntry = import.meta.resolve('tsx/cjs/api')
|
||||
|
||||
@@ -1,41 +1,8 @@
|
||||
/**
|
||||
* The `node:worker_threads` workflow engine: the {@link WorkflowService}
|
||||
* implementation. Runs each script in its OWN worker thread (one run = one
|
||||
* worker, no pooling — a run is heavyweight, so thread spin-up is noise): the
|
||||
* body executes in a vm context INSIDE the worker with the workflow hooks
|
||||
* injected, and `agent()` calls bridge back to `ctx.subagents` over the
|
||||
* message port — child agents are I/O-bound LLM loops and stay on the host
|
||||
* event loop; the thread isolates the SCRIPT, the only part that can spin
|
||||
* synchronously.
|
||||
*
|
||||
* TRUST PREMISE: scripts are MODEL-WRITTEN — the same trust level as the
|
||||
* model's existing bash access — so this engine defends against BUGGY
|
||||
* scripts, never hostile ones. A worker thread is NOT a security boundary:
|
||||
* the vm context inside it is escapable by construction, and an escapee
|
||||
* holds the same process privileges as the host (Node's permission model is
|
||||
* process-wide); genuine sandboxing (isolated-vm, a separate process) is an
|
||||
* engine swap behind the seam. What the thread buys, concretely:
|
||||
*
|
||||
* - `start()` never blocks the host: the script's initial synchronous slice
|
||||
* (and any later synchronous spin) occupies the WORKER's event loop, not
|
||||
* the harness's.
|
||||
* - Termination is REAL: a script that outlives its post-cancel grace is
|
||||
* `worker.terminate()`d — nothing of the script survives `dispose()`,
|
||||
* where an in-process engine could only abandon the spin on its own loop.
|
||||
* - The value boundary is serialization by construction: everything crossing
|
||||
* the thread is structured-clone data (and plain JSON before that, by the
|
||||
* materialization walk in ./realm.ts).
|
||||
*
|
||||
* Engine-specific limitations: worker startup (~tens of ms) is paid per run;
|
||||
* on a termination path `agentsStarted` reports the host-observed child
|
||||
* count (calls still queued worker-side for a slot are unknowable — see
|
||||
* ./host.ts); and a worker that dies unexpectedly (an OOM, a script reaching
|
||||
* `process.exit` through the documented vm escape) settles the run
|
||||
* `stopReason: 'error'` with the exit diagnostics.
|
||||
*
|
||||
* Plugin export shape: a default-exported {@link WorkflowService} subclass
|
||||
* (the class-based service form, like `dsh-bash-local`).
|
||||
*
|
||||
* Worker-thread workflow engine. Each run executes its model-written script in
|
||||
* an escapable vm context on a fresh worker and bridges `agent()` calls to host
|
||||
* subagents. The thread prevents synchronous script work from blocking the host
|
||||
* and permits forced termination, but it is containment rather than a security boundary.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread
|
||||
*/
|
||||
|
||||
|
||||
@@ -1,13 +1,8 @@
|
||||
/**
|
||||
* Meta validation: check the caller-provided {@link WorkflowMeta} DATA against
|
||||
* the shape contract and reject everything else loud, every violation named.
|
||||
* Meta arrives as plain JSON through the seam (the model-facing tool carries
|
||||
* it as a schema-validated object parameter) — the engine never evaluates
|
||||
* script text to obtain it, so no script-controlled code can run on the host
|
||||
* here (an evaluated meta literal could smuggle getters that spin the host
|
||||
* outside any vm timeout, the exact escape the worker thread exists to
|
||||
* prevent).
|
||||
*
|
||||
* Meta validation: check the caller-provided {@link WorkflowMeta} DATA against the shape
|
||||
* contract and reject everything else loud, every violation named. Meta arrives as schema-checked
|
||||
* JSON data, never evaluated script text; evaluating it on the host could run getters outside the
|
||||
* worker timeout that exists to isolate model-written code.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/meta
|
||||
*/
|
||||
|
||||
|
||||
@@ -1,18 +1,9 @@
|
||||
/**
|
||||
* The host⇄worker wire protocol: one string-valued enum of message tags per
|
||||
* direction, a payload map giving each tag its parameters (the single source
|
||||
* of truth), and the message unions derived from them. Everything in a
|
||||
* payload is plain JSON data by construction (the runtime materializes
|
||||
* script values before they reach a message; the host projects seam results
|
||||
* down to their JSON fields), so the structured-clone hop never meets a
|
||||
* value it cannot carry.
|
||||
*
|
||||
* Both directions are CLOSED (engine-owned): each side switches on `type`
|
||||
* and ends with `assertNever` — an unknown message is a protocol bug, never
|
||||
* something to skip silently. Senders go through a generic
|
||||
* `post(type, payload)` whose payload parameter is looked up from the map,
|
||||
* so a tag/payload mismatch is a compile error at the call site.
|
||||
*
|
||||
* The host⇄worker wire protocol: one string-valued enum of message tags per direction, a
|
||||
* payload map giving each tag its parameters (the single source of truth), and the message
|
||||
* unions derived from them. Payloads are plain JSON by construction for structured clone. Both
|
||||
* directions are closed engine protocols whose receivers use `assertNever`; generic typed senders
|
||||
* make tag/payload mismatches compile-time errors rather than silently skipped messages.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/protocol
|
||||
*/
|
||||
|
||||
|
||||
@@ -1,32 +1,10 @@
|
||||
/**
|
||||
* The engine's value boundary: copy script-realm values into plain JSON data
|
||||
* — loud about everything JSON cannot carry — and render thrown script
|
||||
* values to failure text. The script runs in a vm context INSIDE the worker
|
||||
* thread, so "host" here means the worker-side JavaScript around that
|
||||
* context; everything that later crosses the thread boundary is JSON by this
|
||||
* walk, which is what makes the postMessage hop total.
|
||||
*
|
||||
* TRUST PREMISE (everything in this module hangs on it): workflow scripts are
|
||||
* MODEL-WRITTEN, the same trust level as the model's existing bash access, so
|
||||
* this boundary guards against BUGGY scripts, not hostile ones. It rejects
|
||||
* loud what JSON would silently mangle — functions, symbols, bigints,
|
||||
* non-finite numbers, nested `undefined`, cycles, sparse arrays, exotic
|
||||
* prototypes — because accepted-then-ignored is this repo's banned failure
|
||||
* mode. It does NOT defend against adversarial values: the walk reads
|
||||
* properties ordinarily (a getter runs, and whatever it returns is what
|
||||
* crosses), {@link renderThrown} reads `stack`/`message`/`String()` directly,
|
||||
* and a proxy is walked through its traps. A hostile script gains nothing
|
||||
* worth defending here — the vm context inside the worker is escapable by
|
||||
* construction, so hostile-value containment would be cost without a threat
|
||||
* model (what the worker thread DOES buy is that a spin occupies the
|
||||
* worker's loop, not the host's, and termination is real).
|
||||
*
|
||||
* The host→realm direction needs no machinery at all: hooks hand the script
|
||||
* plain values of the worker realm, prototypes included — the script is
|
||||
* trusted. One consequence is documented in the engine README: an error
|
||||
* thrown by a hook is built OUTSIDE the script's vm context, so an in-script
|
||||
* `instanceof Error` check is false; read `name`/`code`/`message` instead.
|
||||
*
|
||||
* Materializes values leaving the script vm into plain JSON before they cross the worker
|
||||
* boundary, and renders thrown script values without rejecting the run. The walk rejects
|
||||
* lossy JSON shapes but trusts model-written workflow scripts: getters and proxy traps may
|
||||
* run, and the vm is not a security boundary. The worker provides host-loop isolation and
|
||||
* forced termination, not hostile-value containment. See
|
||||
* docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md for the isolation rationale.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/realm
|
||||
*/
|
||||
|
||||
@@ -74,17 +52,16 @@ function hasPlainPrototype(value: object): boolean {
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy `value` (typically from the vm realm) into plain host JSON data.
|
||||
* Throws {@link MaterializeError} naming the offending path for anything JSON
|
||||
* cannot carry losslessly. Properties are read ordinarily — a getter runs and
|
||||
* its RESULT is materialized; a read that throws surfaces as a
|
||||
* {@link MaterializeError} carrying the rendered failure. `undefined` is
|
||||
* accepted only at the ROOT (a script with no `return` value) — the caller
|
||||
* decides what it means; an `undefined` nested INSIDE a container is a
|
||||
* violation.
|
||||
* Copy `value` (typically from the vm realm) into plain host JSON data. Root `undefined` is
|
||||
* returned unchanged; nested `undefined` and values JSON cannot represent losslessly fail
|
||||
* with the offending path. Property accessors run normally, and a throwing read is wrapped
|
||||
* with its rendered failure.
|
||||
*
|
||||
* @param value - the realm value to materialize.
|
||||
* @param root - the path label for the root value (error messages).
|
||||
* @returns the host-realm copy (plain objects/arrays/scalars only).
|
||||
* @throws {@link MaterializeError} for unsupported values, cycles, sparse arrays, exotic
|
||||
* prototypes, or property reads that throw.
|
||||
*/
|
||||
export function materializeFromRealm(value: unknown, root = 'value'): unknown {
|
||||
if (value === undefined) return undefined
|
||||
|
||||
@@ -1,39 +1,14 @@
|
||||
/**
|
||||
* Per-run execution state for the engine's THREAD side: the script's vm
|
||||
* context and its injected hooks (`agent`/`parallel`/`pipeline`/`phase`/
|
||||
* `log`/`args`), the concurrency semaphore and caps, cancellation, and the
|
||||
* drive loop that turns a script settlement into a {@link WorkflowResult}.
|
||||
* Children are started by RPC to the host through a {@link ChildPort}, so
|
||||
* this module never touches a cordis context — it runs inside the worker
|
||||
* thread.
|
||||
*
|
||||
* Value boundary (the trust premise lives in ./realm.ts): values ENTERING the
|
||||
* worker-side host code from the script (hook options, schemas, the return
|
||||
* value) are materialized by `materializeFromRealm` — a plain walk that
|
||||
* rejects loud everything JSON cannot carry, which also makes every value
|
||||
* safe for the later postMessage hop. Values ENTERING the realm (`args`,
|
||||
* `agent()` results, hook promises and their failures, combinator arrays) are
|
||||
* handed over DIRECTLY as worker-realm values: the script is model-written
|
||||
* and trusted, so outer prototypes are not a leak. `args` is cloned once at
|
||||
* start so a script scribbling on it cannot mutate the session's init object
|
||||
* (a benign-bug guard; the postMessage clone already isolated the caller).
|
||||
*
|
||||
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
|
||||
* unsupported options/schemas, tripped caps, synchronous start refusal,
|
||||
* provider-start failure, ready-child result rejection, and
|
||||
* cancellation) ALWAYS propagate through
|
||||
* `parallel`/`pipeline` — recognized by `instanceof` against this realm's
|
||||
* class, which a script inside the vm context cannot forge — and the per-item
|
||||
* `null` is reserved for child-run failures and ordinary in-stage script
|
||||
* errors. Every hook-returned promise gets a no-op rejection consumer, so a
|
||||
* dropped promise cannot surface an unhandled rejection (which would kill the
|
||||
* worker and read as an engine fault).
|
||||
*
|
||||
* There is deliberately NO worker-side abandon channel: a script that never
|
||||
* settles after a cancel simply never posts a result, and the HOST enforces
|
||||
* the settles-within-grace guarantee by force-settling `cancelled` and
|
||||
* terminating the worker — the real kill an in-process engine could not have.
|
||||
* Per-run worker-side vm hooks, child RPC, concurrency/caps, cancellation, and result shaping; it
|
||||
* never touches Cordis. Script values leaving the realm are materialized as plain JSON before
|
||||
* messaging. Values entering the trusted model-written realm are passed directly; `args` alone is
|
||||
* cloned so script mutation cannot alter initialization data. See `./realm.ts` for the trust model.
|
||||
*
|
||||
* Fatal workflow errors—bad hook arguments, unsupported schemas/options, caps, start failures, and
|
||||
* cancellation—propagate through combinators. Only child failures and ordinary stage errors become
|
||||
* per-item nulls. Every returned promise has a rejection consumer so dropped script promises cannot
|
||||
* kill the worker. A cancelled script that never settles emits nothing; the host force-settles the
|
||||
* run within grace and terminates the thread.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/runtime
|
||||
*/
|
||||
|
||||
|
||||
@@ -1,19 +1,13 @@
|
||||
/**
|
||||
* The worker-side half of the engine: {@link runWorkerSession} wires one
|
||||
* MessagePort to one {@link WorkflowExecution} — hook progress and child
|
||||
* starts go out as messages, run control and child lifecycle come back in —
|
||||
* and posts the run's terminal result exactly once. Deliberately separated
|
||||
* from the thread bootstrap (./worker.ts): the whole session is drivable
|
||||
* in-process over a `MessageChannel`, which is where its unit coverage lives
|
||||
* (code inside a real Worker is invisible to the main process's coverage).
|
||||
*
|
||||
* Startup handshake: the session posts `ready` and runs the script only
|
||||
* after the host's `go` — without it, a cancellation racing the worker's
|
||||
* boot could arrive AFTER the script's initial synchronous slice already
|
||||
* ran, and a run cancelled before start must not execute the body at all.
|
||||
* A `cancel` arriving instead of `go` still releases the gate: `drive()`
|
||||
* sees the cancelled state and settles without running the body.
|
||||
* The worker-side half of the engine: {@link runWorkerSession} wires one MessagePort to one
|
||||
* {@link WorkflowExecution} — hook progress and child starts go out as messages, run control
|
||||
* and child lifecycle come back in — and posts the run's terminal result exactly once. Keeping it
|
||||
* separate from `worker.ts` lets unit tests drive the session over a MessageChannel, because main
|
||||
* process coverage cannot observe code inside a real Worker.
|
||||
*
|
||||
* The session announces ready and waits for `go`, so cancellation racing startup can prevent even
|
||||
* the script's synchronous prefix. A cancel in place of `go` releases the gate into a cancelled
|
||||
* drive without executing the body.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/session
|
||||
*/
|
||||
|
||||
@@ -137,12 +131,11 @@ export function requireParentPort(port: MessagePort | null): MessagePort {
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one workflow script to settlement against `port`, posting the terminal
|
||||
* result message exactly once; resolves after that post (stray children may
|
||||
* still be winding down through the port — the host owns their teardown and
|
||||
* ultimately terminates the thread). Never rejects: a constructor failure
|
||||
* (unparseable body — host pre-parse makes this a Node-version-skew signal)
|
||||
* is reported as an `error` result rather than dying without a result.
|
||||
* Run one workflow script to settlement against `port`, posting the terminal result message
|
||||
* exactly once; resolves after that post (stray children may still be winding down through the
|
||||
* port — the host owns their teardown and ultimately terminates the thread). It never rejects:
|
||||
* constructor failure becomes an error result. Host pre-parse makes syntax failure here a likely
|
||||
* Node-version skew, but the session still reports it instead of dying silently.
|
||||
* @param port - the channel to the host (the real `parentPort`, or one side
|
||||
* of an in-process `MessageChannel` in tests).
|
||||
* @param init - the run payload the host provided as `workerData`.
|
||||
|
||||
@@ -1,11 +1,7 @@
|
||||
/**
|
||||
* Non-protocol wire vocabulary for the worker-thread engine: the `workerData` init
|
||||
* payload and the child-port interfaces the worker-side runtime consumes.
|
||||
* The host⇄worker MESSAGE protocol lives in ./protocol.ts; everything here
|
||||
* that a message transports (`ChildStartRequest`, `ChildResult`) is plain
|
||||
* JSON data by construction, so the structured-clone hop never meets a value
|
||||
* it cannot carry. Types only, per the package convention.
|
||||
*
|
||||
* Non-protocol wire vocabulary for the worker-thread engine: the `workerData` init payload and
|
||||
* the child-port interfaces the worker-side runtime consumes. Host/worker messages are defined in
|
||||
* `./protocol.ts`; transported child requests and results are plain JSON for structured clone.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/types
|
||||
*/
|
||||
|
||||
|
||||
@@ -1,11 +1,7 @@
|
||||
/**
|
||||
* The worker-thread entry the engine spawns: bootstrap ./session.ts on the
|
||||
* real `parentPort`. Deliberately a single statement — every piece of logic
|
||||
* lives in `runWorkerSession`, which the unit suite drives in-process over a
|
||||
* `MessageChannel` (code inside a real Worker is invisible to main-process
|
||||
* coverage); loading this module on the main thread throws via
|
||||
* `requireParentPort`, which is how the suite covers the file itself.
|
||||
*
|
||||
* Single-statement worker entry that boots `runWorkerSession` on real `parentPort`. Logic remains in
|
||||
* the session module for in-process MessageChannel coverage; importing this entry on the main thread
|
||||
* exercises `requireParentPort`'s failure path.
|
||||
* @module @deepseek-ai/dsh-workflow-workerthread/worker
|
||||
*/
|
||||
|
||||
|
||||
@@ -12,17 +12,12 @@ const builtWorker = join(packageRoot, 'lib', 'worker.cjs')
|
||||
const run = promisify(execFile)
|
||||
|
||||
/**
|
||||
* The BUILT-output guard for the worker entry: every other suite runs
|
||||
* unbuilt (src/ + tsx), so nothing else proves that `lib/index.js` resolves
|
||||
* its sibling `lib/worker.cjs` and that the bundle boots a worker under plain
|
||||
* node (no tsx loader). Keyless — a zero-agent script needs no provider —
|
||||
* and self-skips until `pnpm run build` has produced the bundles.
|
||||
* Keyless built-artifact guard: plain Node loads `lib/index.js` and its sibling
|
||||
* `lib/worker.cjs` without tsx. Skips until the build produces both bundles.
|
||||
*/
|
||||
describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.cjs)', () => {
|
||||
it('the built engine spawns its built worker under plain node and completes a run', async () => {
|
||||
// ESM resolves bare specifiers from the IMPORTING FILE's location, so the
|
||||
// driver must live inside the package for its node_modules to apply — a
|
||||
// temp-named file at the package root, removed on the way out.
|
||||
// Keep the driver in-package so bare imports resolve its node_modules.
|
||||
const driver = join(packageRoot, `.built-worker-driver-${process.pid}.mjs`)
|
||||
try {
|
||||
await writeFile(driver, `
|
||||
@@ -36,7 +31,7 @@ await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
const run = ctx.workflows.start({
|
||||
script: 'return 6 * 7',
|
||||
meta: { name: 'built-smoke', description: 'built worker smoke' },
|
||||
// A zero-agent script never touches the provider, so a bare id suffices.
|
||||
// A zero-agent script never touches the provider.
|
||||
parent: { id: 'built-smoke-parent', options: {} },
|
||||
})
|
||||
const result = await run.result
|
||||
@@ -47,7 +42,6 @@ if (result.stopReason !== 'completed' || result.value !== 42) {
|
||||
}
|
||||
console.log('built-worker-smoke-ok')
|
||||
`, 'utf8')
|
||||
// Plain node — no tsx loader anywhere; the bundle must stand on its own.
|
||||
const { stdout } = await run(process.execPath, [driver], { cwd: packageRoot, timeout: 60_000 })
|
||||
expect(stdout).toContain('built-worker-smoke-ok')
|
||||
} finally {
|
||||
|
||||
@@ -437,10 +437,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
// Cancel FIRST, then the (stale) started reply: the worker processes them
|
||||
// in order, so the agent() continuation resumes already-cancelled — the
|
||||
// window the real host cannot produce (it refuses starts once cancelled)
|
||||
// but a teardown race can.
|
||||
// Simulate a teardown race by delivering cancellation before a stale start reply.
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'raced the start' })
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
const result = await host.result()
|
||||
@@ -448,7 +445,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
|
||||
await vi.waitFor(() => {
|
||||
expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId)
|
||||
})
|
||||
// The child never became an agent-start: it was wound down pre-lifecycle.
|
||||
// The unpublished child is disposed without a lifecycle announcement.
|
||||
expect(host.ofType(WorkerToHostType.AgentStart)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
@@ -17,27 +17,13 @@ function fakeParent(): Agent {
|
||||
return { id: AgentId('workflow-parent'), options: {} } as unknown as Agent
|
||||
}
|
||||
|
||||
// Worker-thread startup is CPU-bound (a fresh thread compiles the runtime on
|
||||
// every start): on a contended CI runner it regularly blows past vitest's 5s
|
||||
// default test timeout, observed repeatedly on the coverage lane.
|
||||
// Allow cold worker startup on contended CI runners.
|
||||
vi.setConfig({ testTimeout: 30_000 })
|
||||
|
||||
/**
|
||||
* `vi.waitFor` with a contention-proof default timeout: the 1s default
|
||||
* flaked repeatedly on the CI coverage lane, where worker-thread cold start
|
||||
* (CPU-bound — a fresh thread compiles the runtime) competes with three
|
||||
* sibling vitest workers for CPU. The 10s default is for exactly those
|
||||
* races — waiting for a worker to start, run its first script line, or
|
||||
* deliver an async child-registration message to the host. It is NOT for a
|
||||
* wait that asserts the HOST reacted PROMPTLY to something that already
|
||||
* happened (a settled result, an observed worker death): those keep an
|
||||
* explicit tight override below, or the generous default would silently
|
||||
* accept a multi-second regression in host-side reap latency as passing
|
||||
* (proven by injecting a 6s delay into one such reap and watching the
|
||||
* un-overridden version of this helper still pass in ~6s).
|
||||
* @param assertion - retried until it stops throwing or the timeout elapses.
|
||||
* @param timeout - override for a wait that must stay deliberately tight.
|
||||
* @returns resolves when the assertion passes.
|
||||
* Wait up to 10 seconds for CPU-bound worker startup or cross-thread delivery on contended CI.
|
||||
* Host reactions after an observed event use explicit tight overrides, so this generous startup
|
||||
* allowance cannot hide multi-second reap regressions.
|
||||
*/
|
||||
function waitFor(assertion: () => void, timeout = 10_000): Promise<void> {
|
||||
return vi.waitFor(assertion, { timeout, interval: 50 })
|
||||
|
||||
@@ -1,14 +1,9 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/**
|
||||
* The engine ships two runtime entries: the engine service (index) and the
|
||||
* worker-thread entry (worker) the engine spawns via `new Worker`. The
|
||||
* entries are JS emitted by tsc under lib/types and are bundled as two
|
||||
* single-entry passes so shared modules (realm, runtime, session) are inlined
|
||||
* into each instead of split into a hash-named chunk (the worker entry must
|
||||
* be a self-contained file the Worker constructor can load by path). The
|
||||
* worker bundle is CommonJS because pkg's VFS Worker hook compiles
|
||||
* filesystem-string entries as CommonJS.
|
||||
* Build the engine and worker separately so each inlines shared modules; a
|
||||
* multi-entry build creates an unlisted chunk. The path-loaded worker is
|
||||
* CommonJS because pkg's VFS Worker hook compiles it in that format.
|
||||
*/
|
||||
export default defineConfig([
|
||||
{
|
||||
|
||||
@@ -1,21 +1,6 @@
|
||||
/**
|
||||
* The workflow capability seam (`ctx.workflows`): an abstract service defining
|
||||
* WHAT a workflow engine does — execute a model-written orchestration script
|
||||
* that fans out subagents — without saying HOW. Implementations subclass
|
||||
* {@link WorkflowService} and register as the `workflows` service (one
|
||||
* implementation per context, cordis' standard duplicate-service behavior);
|
||||
* the implementation is `@deepseek-ai/dsh-workflow-workerthread`, which runs each
|
||||
* script in its own worker thread. Hardened engines (an isolated-vm or
|
||||
* separate-process sandbox) swap in without touching the model-facing tool
|
||||
* that consumes them (`@deepseek-ai/dsh-tool-workflow`).
|
||||
*
|
||||
* The `workflow/*` lifecycle events are OBSERVE-ONLY data: they
|
||||
* carry {@link WorkflowRunInfo} (id + meta), never the live {@link WorkflowRun}
|
||||
* — a listener must not gain `cancel`/`dispose`; control stays with the
|
||||
* `start()` caller holding the run. Same-process payloads are borrowed
|
||||
* immutable values. Every listener is independently contained, so a throw or
|
||||
* rejected promise can neither strand a run nor starve peers.
|
||||
*
|
||||
* Workflow capability seam. Implementations execute orchestration scripts;
|
||||
* observe-only lifecycle events never expose run control.
|
||||
* @module @deepseek-ai/dsh-workflow
|
||||
*/
|
||||
|
||||
@@ -117,27 +102,10 @@ export type WorkflowEventName =
|
||||
| 'workflow/end'
|
||||
|
||||
/**
|
||||
* The workflow-seam error codes. Every one of these is FATAL when it reaches
|
||||
* a script (see {@link WorkflowError.fatal}): the combinators re-throw it
|
||||
* instead of dissolving it into an ordinary per-item `null`.
|
||||
*
|
||||
* - `SCRIPT_PARSE` — the script (or its meta statement) does not parse.
|
||||
* - `META_INVALID` — the meta block evaluated but fails the shape contract.
|
||||
* - `INVALID_ARGUMENT` — a hook was called with malformed arguments.
|
||||
* - `UNSUPPORTED_OPTION` — an `agent()` option this engine does not support
|
||||
* (deferred: `effort`/`isolation`/`agentType`) or does not know.
|
||||
* - `UNSUPPORTED_SCHEMA` — an `agent()` schema outside the structured-output
|
||||
* subset (see dsh-tools).
|
||||
* - `AGENT_CAP` / `ITEM_CAP` — the run/agent caps tripped.
|
||||
* - `AGENT_START` — the provider's asynchronous start rejected before
|
||||
* cancellation took precedence.
|
||||
* - `AGENT_RESULT` — a ready run had its `result` REJECT: an infrastructure
|
||||
* fault at the subagent seam. This is distinct from a child that failed and resolved
|
||||
* (which is the per-item `null`, never an error).
|
||||
* - `RESULT_UNSERIALIZABLE` — a value crossing the script/host value boundary
|
||||
* is not plain JSON data.
|
||||
* - `CANCELLED` — the run was cancelled; pending and future hooks reject
|
||||
* with this (the script-kill mechanism).
|
||||
* Machine-routable fatal workflow failures: parse/meta/argument/schema errors,
|
||||
* resource caps, subagent infrastructure failures, unserializable boundary
|
||||
* values, and cancellation. An ordinary child failure resolves its item to
|
||||
* `null` and is not one of these fatal codes.
|
||||
*/
|
||||
export type WorkflowErrorCode =
|
||||
| 'SCRIPT_PARSE'
|
||||
@@ -182,31 +150,11 @@ export function isFatalWorkflowError(error: unknown): boolean {
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract workflow execution service. Subclass, implement {@link start}, and
|
||||
* load the subclass as a plugin — it registers as `ctx.workflows` (one
|
||||
* implementation per context; loading a second throws, cordis' standard
|
||||
* duplicate-service behavior).
|
||||
*
|
||||
* Semantics every implementation must honor:
|
||||
* - {@link start} throws synchronously for a request that cannot begin (an
|
||||
* unparseable script, an invalid meta block). Once it returns a
|
||||
* {@link WorkflowRun}, `result` NEVER rejects — every failure resolves with
|
||||
* `stopReason: 'error'` (or `'cancelled'`) — and once the run is cancelled,
|
||||
* `result` SETTLES within the implementation's bounded grace even if the
|
||||
* script itself never settles (a consumer awaiting `result` must never be
|
||||
* wedged past a cancellation).
|
||||
* - The `workflow/*` events fire through {@link emitWorkflowEvent} (borrowed
|
||||
* immutable data, per-listener containment); `workflow/end` fires exactly once
|
||||
* per started run, after `result` is settled or as it settles.
|
||||
* - `dispose()` reaches quiescence within a bounded grace: it cancels, waits
|
||||
* for the script to settle AND its started children to finish disposing,
|
||||
* and abandons whatever is left rather than hanging its caller (the engine
|
||||
* documents what abandonment leaves behind).
|
||||
* - Runs are HOLDER-OWNED: the engine hands control (`cancel`/`dispose`) to
|
||||
* the `start()` caller and does not track its live runs — disposing the
|
||||
* engine's own fiber mid-run deliberately leaves those runs to their
|
||||
* holders' teardown, so an engine reload cannot yank a run out from under
|
||||
* the consumer awaiting it.
|
||||
* Workflow execution seam. Invalid requests throw before publication; a live
|
||||
* run is holder-owned, its result never rejects, cancellation and disposal are
|
||||
* bounded, and disposal waits for child cleanup within that bound. Lifecycle
|
||||
* listener failures are contained, and `workflow/end` fires exactly once as the
|
||||
* result settles.
|
||||
*/
|
||||
export abstract class WorkflowService extends Service {
|
||||
constructor(ctx: Context) {
|
||||
@@ -222,14 +170,7 @@ export abstract class WorkflowService extends Service {
|
||||
abstract start(request: WorkflowStartRequest): WorkflowRun
|
||||
|
||||
/**
|
||||
* Emit one `workflow/*` lifecycle event with per-listener containment. Each
|
||||
* subscriber receives the same borrowed immutable payload; a throw or
|
||||
* asynchronously rejected listener is logged (never propagated — the logging
|
||||
* itself is total, even for a thrown value whose own string coercion
|
||||
* throws), so one bad subscriber can neither fail the engine mid-run,
|
||||
* surface as an unhandled rejection on a detached settle hook, nor starve
|
||||
* the listeners registered after it (cordis `emit` halts on the first throw
|
||||
* — same guarantee as the subagent seam's lifecycle emits).
|
||||
* Emit a lifecycle event while containing and logging each listener failure.
|
||||
* @param name - the `workflow/*` event to dispatch.
|
||||
* @param args - the event's payload, matching its declared signature.
|
||||
*/
|
||||
@@ -248,10 +189,7 @@ export abstract class WorkflowService extends Service {
|
||||
}
|
||||
|
||||
/**
|
||||
* Total renderer for a listener-thrown value: the containment catch must never
|
||||
* itself throw, and `String(error)` does when the value's own `toString` /
|
||||
* `Symbol.toPrimitive` throws. Local rather than an engine package's renderer
|
||||
* — the seam sits below every engine and cannot import one.
|
||||
* Render any thrown value without violating listener containment.
|
||||
* @param error - any thrown value.
|
||||
* @returns `String(error)`, or a fixed label when even coercion throws.
|
||||
*/
|
||||
@@ -259,8 +197,7 @@ function renderListenerError(error: unknown): string {
|
||||
try {
|
||||
return String(error)
|
||||
} catch {
|
||||
// Only a throwing toString/Symbol.toPrimitive lands here; the fixed label
|
||||
// keeps the containment guarantee total.
|
||||
// String coercion itself may throw.
|
||||
return '[unrenderable thrown value]'
|
||||
}
|
||||
}
|
||||
|
||||
@@ -106,16 +106,10 @@ export interface WorkflowResult {
|
||||
}
|
||||
|
||||
/**
|
||||
* The handle the consumer holds while a script executes. The consumer awaits
|
||||
* `result`, may `cancel` mid-flight, and MUST `dispose` on every path.
|
||||
* `result` does NOT reject — a script failure resolves with `stopReason:
|
||||
* 'error'` — and once the run is cancelled it SETTLES within the engine's
|
||||
* bounded grace even if the script itself never settles (the engine
|
||||
* force-settles `cancelled`; what becomes of the script is engine-documented
|
||||
* — the worker-thread engine terminates its worker), so a consumer awaiting
|
||||
* `result` is never wedged past a cancellation. `dispose()` = cancel + that
|
||||
* bounded settle + child quiescence; it never hangs on a stuck script and is
|
||||
* safe to call on every path (idempotent).
|
||||
* Holder-owned live workflow. `result` never rejects and settles within the
|
||||
* engine's cancellation grace; failures resolve through `stopReason`. Consumers
|
||||
* may cancel and must call idempotent `dispose()` on every path to await bounded
|
||||
* script settlement and child quiescence.
|
||||
*/
|
||||
export interface WorkflowRun {
|
||||
readonly id: WorkflowRunId
|
||||
|
||||
Reference in New Issue
Block a user