deepseek-harness/docs/rfc/implemented/feature/2026-07-05-dynamic-workflows.md

RFC: Dynamic workflows — a script-driven multi-agent orchestration seam

Status: implemented

Problem

The harness can delegate ONE task to ONE child (dsh-tool-subagent), but work that fans out across many independent pieces — an audit over many files, a migration, multi-angle research, adversarial verification of findings — forces the model to orchestrate turn by turn: every intermediate result lands in the parent context, the plan lives nowhere durable, and coordination costs a model round-trip per step. Claude Code ships this capability as dynamic workflows: the model writes a JavaScript orchestration script, a runtime executes it, and the script — not the conversation — holds the loop, the branching, and the intermediate results.

Decision

A workflow capability family at packages/workflow/ in the bash seam shape (interface / implementation / consumer), plus the structured-output foundation it needs on the subagent seam.

The script contract (Claude Code-compatible)

A workflow call is two parts: a meta JSON parameter (the identity block — name, description, optional whenToUse/phases; the field vocabulary matches Claude Code's meta block) and a script — a plain-JS body with top-level await, ending in return <json-value>. Meta is DATA, never code: the engine shape-validates it and evaluates no script text to obtain it (a body still opening with a CC-style export const meta statement is rejected with a pointed message). The body sees exactly: agent(prompt, {label, phase, schema, model}), parallel(thunks), pipeline(items, ...stages) (NO cross-stage barrier; (prev, item, index) callbacks), phase(title), log(message), and args. CC semantics are preserved where they matter to script authors: a failed child resolves null (scripts .filter(Boolean)); an ordinary stage throw nulls the ITEM and skips its remaining stages. CC's determinism bans (Date.now()/Math.random()/argless new Date() throwing) are NOT enforced — they exist for CC's journaling/resume, which this cut defers — so a CC-authored BODY runs unchanged (its meta header moves into the parameter) while scripts written here may freely read the clock.

One deliberate strictness DIVERGENCE from CC: hook misuse — unknown or deferred options (effort/isolation/agentType), malformed arguments, schemas outside the supported subset, tripped caps, seam start failures — throws a WorkflowError with fatal: true, and the combinators RE-THROW fatal errors instead of nulling the item. Without this, a typo'd option dissolves into a null indistinguishable from a child failure — the accepted-then-ignored failure mode this repo bans. One addition: the tool's args parameter is a JSON OBJECT (a bare list is wrapped as a field) so the wire schema stays honest.

The seam (dsh-workflow)

ctx.workflows is an abstract WorkflowService in the bash shape — one engine per context, no named-provider registry (engines are deployment swaps, not co-residents). start(request) throws synchronously for a script that cannot begin; a returned WorkflowRun's result NEVER rejects (failures resolve as stopReason: 'error' | 'cancelled'). The workflow/* events are observe-only emits carrying DATA SNAPSHOTS (id + meta; workflow/end omits the result value), per-listener contained, mirroring subagent/start/subagent/end — control stays with the run's holder. Vocabulary details: core-data-structures/workflow.md.

The engine (dsh-workflow-workerthread): one worker thread per run

Trust premise (governs every engine decision below): workflow scripts are MODEL-WRITTEN — the same trust level as the model's existing bash access — so the engine defends against BUGGY scripts, never hostile ones. In scope: result never rejects, no unhandled rejections from dropped hook promises, loud rejection of values JSON cannot carry, fatal-vs-null hook discipline, cancellation that always frees the caller. Out of scope, deliberately: adversarial values (throwing/spinning accessors, proxies with hostile traps, prototype forgery, prepareStackTrace hijack) AND Node-API escape from the script's context — the vm context shares object machinery with its surrounding realm, so a script can reach the Function constructor (globalThis.constructor.constructor) and from it process and every Node builtin; the absent globals are API surface, not containment, and a worker thread is NOT a security boundary (an escapee holds process-wide privileges — Node's permission model is per-process). Worker-side code MAY run script code while reading script values, and that is accepted: a synchronous spin costs the script its OWN thread (terminated at the post-cancel grace), never the host loop, so containing error VALUES would be cost without a threat model. Genuine sandboxing (isolated-vm, a separate process) remains an engine swap behind the seam, not incremental defenses here.

Why node:worker_threads: one run uses one unpooled worker because a workflow run is already heavyweight relative to thread startup. The script runs in a vm context inside the worker, keeping the script-visible surface to the hook contract instead of exposing a bare worker realm, while agent() bridges by message-port RPC to I/O-bound child loops on the host. This keeps start() from blocking the host on the script's synchronous slice, makes the post-cancel deadline end in a real worker.terminate(), and gives cross-thread values a serialization boundary by construction. isolated-vm was rejected for its maintenance state, required --no-node-snapshot consumer flag on Node ≥ 20, and node-gyp fallback.

Host-side meta validation and body pre-parsing preserve the seam's synchronous errors, and private enum-keyed payload maps define the wire protocol. Pending async starts, published child records, one host cancellation signal, worker-death reaping, result precedence, and disposal quiescence preserve the subagent run contract across that wire; the agent-scope runtime-design RFC owns those race algorithms. Coverage uses an in-process MessageChannel for worker-side logic that main-process V8 coverage cannot see and separately proves the built lib/worker.js—a second tsdown entry sanctioned by the "./worker" subpath export—under plain Node in the built-bin smoke gate.

Meta as data, never evaluated: the meta block reaches the seam as a plain JSON request field (the tool's schema-validated meta parameter) and the engine only shape-validates it, every violation named. This is a host-isolation invariant, not a convenience: evaluating a meta literal host-side — even one contractually "pure", in an empty timed vm context — hands script-controlled getters a host stack with no timeout the moment the result is READ, defeating the exact spin isolation the worker thread buys.

Value boundary: values leaving the script (meta, hook options, schemas, the return value) go through materializeFromRealm — a plain recursive walk that rejects loud everything JSON cannot carry (exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, nested undefined), copying via Object.defineProperty so a "__proto__" key becomes a data property, never a prototype mutation; getters are read ordinarily and their RESULT crosses (a throwing read fails loud) — which is also what makes every later postMessage hop total. Values entering the realm (args, agent() results, hook promises and failures, combinator arrays) are handed over directly as worker-realm values — the script is trusted, so outer prototypes are not a leak; args rides the workerData structured clone (the caller-isolation copy) and is cloned once more so a script scribbling on it cannot mutate the session's init object. Hook failures are WorkflowErrors built OUTSIDE the script's context: the combinators recognize fatality by instanceof against the engine's own class (unforgeable from the script), and the script-visible consequence — in-script instanceof Error is false for hook errors; branch on e.name/e.code — is documented in the engine README. Realm functions (stages, thunks) are called, never materialized. Thrown script values are rendered by a total renderer (stack → message → String(), fixed label if rendering throws), so result cannot reject. Caps (maxConcurrentAgents auto = min(16, max(1, availableParallelism() - 2)), maxTotalAgents 1000, maxItemsPerCall 4096) and timeouts are validated Config, not literals.

The consumer (dsh-tool-workflow)

A workflow tool mirroring dsh-tool-subagent's synchronous shape: start, await, try/finally dispose, abort-bridge exec.signal, non-completed → isError. Render intent: a generic card titled by the call's meta.name parameter (presentation is a pure function of args). The tool description IS the model-facing authoring spec. The usage policy ships with the tool as its own tool:<toolName> prompt section (explicit-ask-only guidance — tool guidance lives in tool plugins, never in the deployment persona); the harness has no ultracode-style effort gate.

The foundation: structured output on the subagent seam

SubagentStartRequest.outputSchema is implemented by dsh-subagent-inprocess for both in-process backends. Each structured child receives its own scoped capture tool, instruction, and enforcement registrations on child.ctx; concurrent children can use different schemas without sharing mutable policy, and disposing the child removes the entire attachment.

An output schema makes a schema-valid committed capture mandatory for successful child completion. The scoped runtime presents the capture tool and instruction, commits only a successful final outcome—including the enclosing run_code outcome for an SDK call—denies later side effects after capture becomes pending, and stops the child without another model step after commit. A validation failure remains a retryable tool error; clean completion without a committed capture settles as an error.

StructuredOutputSchema is the raw enforceable JSON-Schema subset in dsh-tools (single-string type, properties/required/additionalProperties, items, scalar enum/const), and unsupported keywords fail loudly because that wire data becomes the capture tool's parameters verbatim. The agent-scope runtime-design RFC owns the assembly, commit, guard, and terminal-stop correctness algorithms.

Deferred (documented non-goals of this cut)

• Background collection (start tool → run id → completion notice → collect), designed alongside bash/subagent background unification.
• Journaling + resume (resumeFromRunId, cached agent() prefixes) — implementing it reintroduces CC's determinism bans as a script-contract tightening (scripts may read the clock today).
• Saved/bundled workflows (a .deepseek/workflows/ registry, slash-command surface) and script persistence to a run directory (the tool-call event already records the script durably).
• Nested workflow(), token budget, and the effort/isolation/agentType agent options (each rejects loud with a message naming it deferred).
• An overall run wall-clock timeout — cancellation always frees the caller (result settles within the grace), so a cap on total run time is a policy knob for the background redesign, not a correctness need here.
• Engine hardening beyond worker threads: an isolated-vm or separate-process engine behind the same seam (actual sandboxing; memory limits).
• ACP progress UI over the workflow/* events (a /workflows-style view); the events exist for it.
• ACP-backend structured output and toolFilter (both still capability-gated false).

Alternatives considered

• Hostile-value containment in the host (trap-free proxy rejection, accessor-never-invoked descriptor walks, realm-side pre-rendering of thrown values, realm-built promises/arrays/error clones with structural fatal recognition): rejected because every defense targets an author the trust premise accepts, while the thread's serialization boundary already makes cross-realm values total by construction.
• In-process node:vm execution: mechanically simplest — no RPC, no thread — but start() blocks the caller for the script's initial synchronous slice, a synchronous spin past the first await cannot be killed in-process (the vm timeout covers only that first slice), and dispose() could only abandon an unsettling script on the host loop. The worker-thread engine keeps the same vm-context script surface while unblocking the host and making termination real.
• Background execution as the default (CC's shape): deferred; foreground-synchronous matches dsh-tool-subagent's cut, and background semantics should be designed ONCE across bash/subagent/workflow rather than per-tool.
• Workflow-layer JSON parsing for agent({schema}): duplicating a seam concern at one consumer while the seam's capability flag stayed dishonestly false.
• Meta embedded in the script as export const meta = {...} (CC's exact format): keeps scripts self-contained and CC scripts drop-in, but obtaining meta requires evaluating model-written text on the host. Even an empty timed vm context cannot bound script-controlled getters when the host reads the resulting object. A JSON parameter removes the scanner, evaluation, and host-spin hole; the cost is that a CC script's meta header must move into the parameter (the body stays drop-in).
• SchemaSpec as the outputSchema type: the author-facing DSL cannot express what arrives as data and cannot be validated against without conversion loss.
• A schema-object library (zod, or the repo's schemastery) for the structured-output subset: the schema is wire data — plain JSON that crosses the vm realm boundary in agent({schema}) and lands verbatim in the forced tool's parameters — exactly where live schema objects cannot sit; consuming raw JSON Schema at runtime would need a third-party converter on top (zod core only emits JSON Schema, not the reverse), and it would put a second schema language beside schemastery's config role.
• ajv for value validation: it validates FULL JSON Schema, so the subset gate — the module's actual point, since every accepted keyword must be one the harness enforces — would remain hand-written regardless; it compiles validators through new Function; and it would be dsh-tools' first runtime dependency, all to replace the ~70-line value walker while the path-qualified, every-violation error reporting stays custom either way.
• Provider JSON mode (response_format: {type: json_object}) instead of the forced capture tool: the official API guarantees valid JSON, not schema-conforming JSON (no json_schema type; the docs' own guidance is to validate client-side, with the schema riding in the prompt), so both walkers survive untouched and only the capture-tool mechanics could go — at the cost of tools during a structured child's run (whether response_format composes with tool calling is undocumented), the in-turn validation retry (ToolArgsError keeps recovery inside the turn; a JSON-mode empty body — a documented failure mode — ends the turn, and the only recovery is the re-prompt loop this design rejects), and a new per-adapter LlmCallConfig surface. The accepted upgrade path is strict TOOL schemas (provider-side constrained decoding on tool parameters) when available: the same forced tool and subset gate, with the gate narrowed to the provider's strict subset.

Consequences

The harness gains CC-compatible script orchestration: fan-out plans live in a rerunnable artifact instead of the parent context, and outputSchema yields an authoritative structured child result across native and Code Mode presentation. The cost, bounded by the trust premise, is a worker thread per run (~tens-of-ms spin-up), every hook crossing a message port as RPC, and a termination-path agentsStarted that degrades to the host-observed count; in exchange start() never blocks the host, a post-cancel grace ends in a real worker.terminate(), and the value boundary is serialization by construction. A worker thread is still not a security boundary — scripts share the model's trust level, and actual sandboxing requires an isolated-vm/separate-process engine behind the seam. The fatal-vs-null strictness divergence from CC means a CC-authored script that relies on option typos dissolving to null behaves differently, preserving the repo's no-accepted-then-ignored rule. Consumers must hold the run handle for control (cancel/dispose); observers get data snapshots only, so no listener can extend a run's lifetime or corrupt another's view.