refactor: apply repository naming contract
Apply the accepted pre-release package, service, type, directory, and role renames as one repository-wide change.
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/workflow/workflow-worker-thread/README.md
|
||||
README.md: ba32c9dab4e870dd52c8f8acba3cfa627fa78000
|
||||
README.zh.md: 99107aec40a45c8460f10082f2b75b908e3e8692
|
||||
124
packages/workflow/workflow-worker-thread/README.md
Normal file
124
packages/workflow/workflow-worker-thread/README.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# @deepseek-ai/dsh-workflow-worker-thread
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
This package implements `WorkflowEngine` with one Node worker thread per run. The worker executes the orchestration script; child agents remain on the host and are reached through `ctx.subagents` over a typed host/worker protocol.
|
||||
|
||||
The package root exports the default engine plugin and its `Config`; the worker protocol, runtime, and session modules stay private to the implementation. The operational `./worker` entry remains the engine's spawn target.
|
||||
|
||||
The split has one primary purpose: a synchronous script loop cannot block the harness event loop, and a script that ignores cancellation can be terminated with its worker. It is not a security sandbox.
|
||||
|
||||
## Trust and isolation boundary
|
||||
|
||||
Workflow scripts are model-written and have the same trust premise as the model's existing bash access. `node:vm` inside a worker is an API-shaping mechanism, not a security boundary: an escaped script can recover Node capabilities with the host process's privileges.
|
||||
|
||||
The worker still provides useful containment:
|
||||
|
||||
- Script CPU work and synchronous spins stay off the host event loop.
|
||||
- `worker.terminate()` gives disposal a real final stop.
|
||||
- The worker starts with an empty environment, except unbuilt loader plumbing, so ambient credentials do not cross through `process.env`.
|
||||
- Host/worker messages use structured-clone data, with plain-JSON validation at the script boundary.
|
||||
|
||||
A genuinely untrusted-script sandbox would require a different engine behind the same workflow seam.
|
||||
|
||||
## Script contract
|
||||
|
||||
The workflow's `meta` is host-provided data, not evaluated script text. The engine validates its required `name` and `description`, rejects unknown fields, and parse-checks the body before returning a run.
|
||||
|
||||
Inside the worker, the script receives `args` and these hooks:
|
||||
|
||||
- `agent(prompt, { label, phase, schema, model })` starts one host-side subagent. With a schema it returns the structured value; otherwise it returns final text. An ordinary failed child yields `null`.
|
||||
- `parallel(thunks)` runs thunks under the configured concurrency limit.
|
||||
- `pipeline(items, ...stages)` passes `(previous, item, index)` without a cross-stage barrier.
|
||||
- `phase(title)` and `log(message)` emit observer narration.
|
||||
|
||||
Unknown options, malformed arguments, unsupported schemas, tripped caps, provider-start failures, and infrastructure result failures are fatal workflow errors. No timers, filesystem API, or Node globals are intentionally injected, though the trust caveat above still applies.
|
||||
|
||||
## Run sequence
|
||||
|
||||
`start()` validates meta, parses the body, resolves a registered normalized provider route, and resolves any per-run total-child cap before creating a worker or publishing `workflow/start`. A requested `maxTotalAgents` must be a positive safe integer no greater than the engine's configured deployment ceiling. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
|
||||
For each `agent()` call:
|
||||
|
||||
1. The worker sends `child-start` with a plain-data prompt and options.
|
||||
2. The host calls the start request's provider override, or otherwise the configured provider, through async `SubagentRuntime.start`, passing the workflow's parent and one canonical per-run abort signal. Provider choice applies to every child in that run and is not visible to the script.
|
||||
3. If start rejects, the host sends `child-start-error`; provider startup has already reached quiescence and no child lifecycle event is emitted.
|
||||
4. If start fulfills while the workflow still admits work, the host records the run, observes `result`, then sends `child-started`. Even an already-settled result is forwarded afterward, preserving start-before-result order.
|
||||
5. The worker emits paired `workflow/agent-start` and `workflow/agent-end` narration and requests child disposal after collection.
|
||||
|
||||
Provider starts are tracked separately from published children. If cancellation, worker death, or normal workflow settlement closes admission while a start is pending, the shared signal aborts it. A provider that nevertheless fulfills after closure is disposed by the host and never announced to the worker.
|
||||
|
||||
## Value boundary
|
||||
|
||||
Values leaving the script pass through `materializeFromRealm`, which accepts plain, lossless JSON data and rejects exotic prototypes, functions, symbols, cycles, sparse arrays, non-finite numbers, and nested `undefined`. The walk runs in the worker, and defines object keys as data properties so `__proto__` cannot mutate a prototype.
|
||||
|
||||
Child results are projected and snapshotted before crossing from the host to the worker. This is a real process-like serialization boundary; it is deliberately different from trusted same-process workflow and subagent event payloads, which are borrowed immutable values.
|
||||
|
||||
## Cancellation and disposal
|
||||
|
||||
`WorkflowRun.cancel()` records the first reason, tells the worker to cancel, aborts the one signal shared by every pending and published child, and arms the `disposeGraceMs` timer. Worker hooks then throw `CANCELLED` at their next await. If the run remains unsettled at the deadline, the host resolves it as cancelled, pairs stranded child lifecycle events, and terminates the worker.
|
||||
|
||||
The subagent seam has one cancellation channel: the request signal. There is no separate child-cancel RPC. Published child teardown uses `run.dispose()`; pending provider starts remain provider-owned until their promise rejects or fulfills.
|
||||
|
||||
Normal settlement also aborts pending starts and begins disposing any published fire-and-forget children before the result becomes externally settled. The host's quiescence condition includes both pending starts and published child disposals, so cleanup does not forget an async startup transaction.
|
||||
|
||||
`dispose()` is idempotent. It cancels the run, starts host-driven disposal immediately, waits for result plus child quiescence up to the same grace, terminates the worker unconditionally, and performs a final survivor sweep. Per-child disposal is memoized so worker RPC, host cancellation, death cleanup, and public disposal all join one operation.
|
||||
|
||||
## Outcome and event guarantees
|
||||
|
||||
Terminal outcome is first-wins at host claim points. An accepted external cancellation overrides a later non-cancelled worker result; a result or worker death that claims first cannot be rewritten by reentrant cleanup callbacks.
|
||||
|
||||
Worker error, message failure, or premature exit closes message admission before cleanup, then resolves `error` unless cancellation already owns the run. Late queued messages cannot create children or narrate after that logical boundary.
|
||||
|
||||
The host keeps a ledger of forwarded child starts. A graceful worker supplies their ends; death or force termination synthesizes any missing end as cancelled. Every forwarded `workflow/agent-start` is therefore paired exactly once, although cleanup after an already-arrived workflow result may complete afterward.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `provider` | `spawn` | Host-side subagent provider used by `agent()`. |
|
||||
| `maxConcurrentAgents` | `0` | Concurrent `agent()` ceiling; `0` resolves from available CPU parallelism. |
|
||||
| `maxTotalAgents` | `1000` | Total `agent()` calls in one run. |
|
||||
| `maxItemsPerCall` | `4096` | Items accepted by one `parallel()` or `pipeline()` call. |
|
||||
| `syncTimeoutMs` | `5000` | VM timeout for the script's initial synchronous slice. |
|
||||
| `disposeGraceMs` | `5000` | Bound before force-settlement/termination and for public disposal. |
|
||||
|
||||
An owning consumer may set `WorkflowStartRequest.subagentProvider` and `WorkflowStartRequest.maxTotalAgents` for one run. These are engine-level policy, not script hooks or model-facing options; the ordinary `workflow` tool leaves both unset. A per-run total-child cap may lower but never raise the configured `maxTotalAgents` ceiling.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Child-agent requests
|
||||
|
||||
#### What the model sees
|
||||
|
||||
Every script `agent()` call sends its prompt verbatim and optional model or structured-output schema to a subagent provider. Each child sees that provider's own context; phase and log narration stays on observer events.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Potentially many independent child contexts are paid, bounded by `maxConcurrentAgents`, `maxTotalAgents`, and `maxItemsPerCall`; they never join the parent history directly.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Independent of the parent request cache and of sibling children. Each child can reuse only a byte-identical prefix under its own provider, model, prompt, and schema; its later history grows append-only.
|
||||
|
||||
### Parent tool result, indirectly
|
||||
|
||||
#### What the model sees
|
||||
|
||||
Through [`dsh-tool-workflow`](../tool-workflow/README.md), success exposes only the materialized final JSON value and child count in that consumer's wrapper. This engine supplies stable errors including `workflow script does not parse: <error>`, `invalid meta: <violations>`, `agent() requires a non-empty prompt string`, `agent() could not start a child: <error>`, `child agent run failed: <error>`, and its exact `parallel()`, `pipeline()`, `phase()`, option, schema, and JSON-boundary validation messages. Intermediate child outputs are available to the script but not the parent model.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Zero direct parent tokens from this engine. Final result size is capped by the tool consumer and retained until compaction.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Append-only; newly visible content follows the reusable request prefix and does not invalidate existing KV-cache entries.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The worker/vm is not a security boundary** — model-written code can escape `node:vm` and reach the worker's process authority; a hostile-code deployment needs a separate-process or container engine.
|
||||
- **One worker thread is paid per run** — there is no pool, warm runtime, or cross-run script cache.
|
||||
- **No ambient timers, filesystem, or network are injected, but escaped code can still reach Node** — the missing globals are portability API, not containment.
|
||||
- **Termination can only report host-observed starts** — `agentsStarted` excludes worker-side calls still queued behind concurrency when a forced termination makes them unknowable.
|
||||
- **Cross-realm errors fail `instanceof Error` inside scripts** — workflow authors must branch on stable fields such as `name` and `code`.
|
||||
124
packages/workflow/workflow-worker-thread/README.zh.md
Normal file
124
packages/workflow/workflow-worker-thread/README.zh.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# @deepseek-ai/dsh-workflow-worker-thread
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
本包为 `WorkflowEngine` 提供实现,每次运行使用一个 Node worker thread。worker 执行编排脚本;子 agent(智能体)留在宿主上,脚本通过带类型的宿主/worker 协议经由 `ctx.subagents` 访问它们。
|
||||
|
||||
包根目录默认导出引擎插件及其 `Config`;worker 协议、运行时和会话模块均为实现私有。操作入口 `./worker` 仍是引擎的 spawn 目标。
|
||||
|
||||
这种拆分只有一个主要目的:同步脚本循环不能阻塞 harness 事件循环,忽略取消的脚本可以连同其 worker 一起终止。它不是安全沙箱。
|
||||
|
||||
## 信任与隔离边界
|
||||
|
||||
工作流脚本由模型编写,信任前提与模型已有的 bash 访问相同。worker 内的 `node:vm` 是塑造 API 的机制,不是安全边界:逃逸的脚本可以用宿主进程权限重新取得 Node 能力。
|
||||
|
||||
worker 仍提供实用的隔离:
|
||||
|
||||
- 脚本 CPU 工作和同步自旋不会占用宿主事件循环;
|
||||
- `worker.terminate()` 为 dispose(资源释放)提供真实的最终停止手段;
|
||||
- 除未构建 loader 所需的衔接配置外,worker 以空环境启动,因此环境凭据不会通过 `process.env` 跨越边界;
|
||||
- 宿主/worker 消息使用结构化克隆数据,并在脚本边界执行普通 JSON 校验。
|
||||
|
||||
真正的不可信脚本沙箱需要在同一工作流 seam 背后采用不同引擎。
|
||||
|
||||
## 脚本约定
|
||||
|
||||
工作流的 `meta` 是宿主提供的数据,而不是待求值的脚本文本。引擎会校验必需的 `name` 和 `description`、拒绝未知字段,并在返回运行前检查脚本正文能否解析。
|
||||
|
||||
在 worker 内,脚本会收到 `args` 以及以下钩子:
|
||||
|
||||
- `agent(prompt, { label, phase, schema, model })` 启动一个宿主侧 subagent。提供 schema 时返回结构化值,否则返回最终文本。普通子 agent 失败会产生 `null`;
|
||||
- `parallel(thunks)` 在已配置的并发限制下运行 thunk;
|
||||
- `pipeline(items, ...stages)` 在没有跨阶段屏障的情况下传递 `(previous, item, index)`;
|
||||
- `phase(title)` 和 `log(message)` 发出观察器叙述。
|
||||
|
||||
未知选项、格式错误的参数、不支持的 schema、超出上限、提供方启动失败和基础设施结果失败都属于致命工作流错误。有意不注入 timer、文件系统 API 或 Node 全局变量,但上述信任注意事项仍然适用。
|
||||
|
||||
## 运行顺序
|
||||
|
||||
`start()` 会校验 meta、解析脚本正文、解析一个已注册且规范化的提供方路由,并解析每次运行的子 agent 总数上限,然后才创建 worker 或发布 `workflow/start`。请求的 `maxTotalAgents` 必须是正安全整数,且不能超过引擎配置的部署上限。源代码模式通过 data URL bootstrap 安装 TypeScript 转换;构建模式把同级 `lib/worker.cjs` 作为文件系统路径传入,因为 pkg 的虚拟文件系统(VFS)钩子要求 CommonJS。两者都能在普通 Node 下运行。ready/go 握手可以避免启动信号取消与 worker 启动发生竞态,导致脚本最初的同步片段被执行。
|
||||
|
||||
对于每次 `agent()` 调用:
|
||||
|
||||
1. worker 发送 `child-start`,其中包含普通数据提示词和选项。
|
||||
2. 宿主通过异步 `SubagentRuntime.start` 调用启动请求中指定的提供方,否则调用已配置的提供方;调用会传入工作流父级和该次运行共用的唯一中止信号。提供方选择应用于该次运行的每个子 agent,对脚本不可见。
|
||||
3. 如果启动被拒绝,宿主会发送 `child-start-error`;提供方启动已经完全停稳,不会发出子 agent 生命周期事件。
|
||||
4. 如果启动兑现时工作流仍接纳工作,宿主会记录该运行、观察 `result`,然后发送 `child-started`。即使结果已经结算,也只会随后转发,以保持先启动、后结果的顺序。
|
||||
5. worker 发出成对的 `workflow/agent-start` 和 `workflow/agent-end` 叙述,并在收集后请求 dispose 子 agent。
|
||||
|
||||
提供方启动与已发布子 agent 分开跟踪。如果启动仍在等待,而取消、worker 死亡或正常工作流结算关闭了接纳,共享信号会中止该启动。即便提供方随后兑现,宿主也会 dispose 它,且绝不向 worker 通知。
|
||||
|
||||
## 值边界
|
||||
|
||||
离开脚本的值会经过 `materializeFromRealm`;该函数接受普通的无损 JSON 数据,并拒绝特殊原型、函数、symbol、循环、稀疏数组、非有限数和嵌套 `undefined`。遍历在 worker 内执行,并把对象键定义为数据属性,使 `__proto__` 无法改变原型。
|
||||
|
||||
子 agent 结果从宿主跨越到 worker 之前,会先投影并制作快照。这是真正近似进程的序列化边界;它有意不同于可信的同进程工作流和 subagent 事件 payload,后者以不可变方式借用值。
|
||||
|
||||
## 取消与 dispose
|
||||
|
||||
`WorkflowRun.cancel()` 会记录第一个原因、通知 worker 取消、中止每个待处理及已发布子 agent 共享的唯一信号,并启动 `disposeGraceMs` 定时器。worker 钩子会在下次 await 时抛出 `CANCELLED`。如果运行到期限仍未结算,宿主会将其以已取消状态兑现、为悬空的子 agent 生命周期事件配对,并终止 worker。
|
||||
|
||||
subagent seam 只有一个取消通道:请求信号。不存在单独的子 agent 取消 RPC。已发布子 agent 使用 `run.dispose()` 清理;待处理的提供方启动在其 promise 拒绝或兑现前仍由提供方负责。
|
||||
|
||||
正常结算也会中止待处理启动,并在结果对外结算前开始 dispose 所有已发布但无需等待的子 agent。宿主的完全停稳条件同时包括待处理启动和已发布子 agent 的 dispose,因此清理不会遗漏异步启动事务。
|
||||
|
||||
`dispose()` 是幂等的。它会取消运行、立即启动宿主驱动的 dispose、在同一宽限时间内等待结果和子 agent 完全停稳、无条件终止 worker,并执行最后一次幸存项扫描。每个子 agent 的 dispose 都会记忆化,使 worker RPC、宿主取消、死亡清理和公开 dispose 都汇入同一操作。
|
||||
|
||||
## 结果与事件保证
|
||||
|
||||
在宿主的结果确认点,终态结果遵循先到者胜。已接受的外部取消会覆盖后到的非取消 worker 结果;先完成确认的结果或 worker 死亡不能被可重入清理回调改写。
|
||||
|
||||
worker 错误、消息失败或提前退出会在清理前关闭消息接纳,然后以 `error` 兑现;如果取消已经接管该运行,则不覆盖取消。后到的排队消息无法在该逻辑边界后创建子 agent 或发出叙述。
|
||||
|
||||
宿主会维护已转发子 agent 启动的台账。优雅退出的 worker 会提供对应的结束事件;死亡或强制终止会把缺失的结束事件合成为已取消。因此,每个已转发的 `workflow/agent-start` 都会且只会配对一次,不过已经到达的工作流结果之后的清理可能稍后才完成。
|
||||
|
||||
## 配置
|
||||
|
||||
| 键 | 默认值 | 含义 |
|
||||
|---|---|---|
|
||||
| `provider` | `spawn` | `agent()` 使用的宿主侧 subagent 提供方。 |
|
||||
| `maxConcurrentAgents` | `0` | 并发 `agent()` 上限;`0` 会根据可用 CPU 并行度解析。 |
|
||||
| `maxTotalAgents` | `1000` | 一次运行中的 `agent()` 调用总数。 |
|
||||
| `maxItemsPerCall` | `4096` | 一次 `parallel()` 或 `pipeline()` 调用接受的条目数。 |
|
||||
| `syncTimeoutMs` | `5000` | 脚本最初同步片段的 VM 超时时间。 |
|
||||
| `disposeGraceMs` | `5000` | 强制结算/终止之前的期限,也是公开 dispose 的期限。 |
|
||||
|
||||
负责该引擎的消费方可以为一次运行设置 `WorkflowStartRequest.subagentProvider` 和 `WorkflowStartRequest.maxTotalAgents`。它们属于引擎级策略,不是脚本钩子或面向模型的选项;普通 `workflow` 工具不会设置两者。每次运行的子 agent 总数上限可以降低、但绝不能提高已配置的 `maxTotalAgents` 上限。
|
||||
|
||||
## 模型体验
|
||||
|
||||
### 子 agent 请求
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
脚本每次调用 `agent()`,都会把提示词原样发送给 subagent 提供方,并附带可选模型或结构化输出 schema。每个子 agent 看到该提供方自己的上下文;phase 和 log 叙述只留在观察器事件中。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
可能需要为许多独立子 agent 上下文支付 token 成本,数量受 `maxConcurrentAgents`、`maxTotalAgents` 和 `maxItemsPerCall` 限制;这些上下文绝不会直接加入父级历史。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
与父级请求缓存和同级子 agent 缓存相互独立。每个子 agent 只能在其自身提供方、模型、提示词和 schema 下复用逐字节相同的前缀;其后续历史仅追加增长。
|
||||
|
||||
### 父级工具结果(间接)
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
通过 [`dsh-tool-workflow`](../tool-workflow/README.md),成功结果只会在该消费方的包装层中公开实体化的最终 JSON 值和子 agent 数量。本引擎提供稳定错误,包括 `workflow script does not parse: <error>`、`invalid meta: <violations>`、`agent() requires a non-empty prompt string`、`agent() could not start a child: <error>`、`child agent run failed: <error>`,以及其精确的 `parallel()`、`pipeline()`、`phase()`、选项、schema 和 JSON 边界校验消息。中间子 agent 输出可供脚本使用,但不提供给父模型。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
本引擎不会直接向父级添加 token。最终结果大小由工具消费方限制,并保留到压缩(compaction)为止。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
仅追加;新增可见内容位于可复用请求前缀之后,不会使现有 KV Cache 条目失效。
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **worker/vm 不是安全边界**:模型编写的代码可以逃逸 `node:vm` 并取得 worker 的进程权限;不可信代码部署需要独立进程或容器引擎。
|
||||
- **每次运行都要支付一个 worker thread 的成本**:没有池、预热运行时或跨运行脚本缓存。
|
||||
- **不注入默认可用的定时器、文件系统或网络,但逃逸代码仍可访问 Node**:这些缺失的全局变量属于可移植性 API 设计,而非隔离措施。
|
||||
- **终止只能报告宿主观察到的启动**:`agentsStarted` 不包括因并发限制仍在 worker 侧排队、且在强制终止后无法得知的调用。
|
||||
- **跨 realm 错误在脚本内无法通过 `instanceof Error`**:工作流作者必须根据 `name` 和 `code` 等稳定字段分支。
|
||||
69
packages/workflow/workflow-worker-thread/package.json
Normal file
69
packages/workflow/workflow-worker-thread/package.json
Normal file
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-workflow-worker-thread",
|
||||
"description": "worker-thread workflow engine: executes model-written orchestration scripts off the host event loop, bridging agent() calls back to ctx.subagents",
|
||||
"version": "0.0.1-rc.2",
|
||||
"publishConfig": {
|
||||
"access": "restricted"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/deepseek-ai/deepseek-harness.git",
|
||||
"directory": "packages/workflow/workflow-worker-thread"
|
||||
},
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./worker": {
|
||||
"types": "./lib/types/worker.d.ts",
|
||||
"default": "./lib/worker.cjs"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/worker.cjs",
|
||||
"lib/types/**/*.d.ts"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^"
|
||||
},
|
||||
"dependencies": {
|
||||
"@deepseek-ai/schemastery": "workspace:^"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop-testkit": "workspace:^",
|
||||
"@deepseek-ai/dsh-brand": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent-spawn-in-process": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow": "workspace:^",
|
||||
"@deepseek-ai/cordis": "workspace:^",
|
||||
"tsx": "^4.19.2"
|
||||
}
|
||||
}
|
||||
594
packages/workflow/workflow-worker-thread/src/host.ts
Normal file
594
packages/workflow/workflow-worker-thread/src/host.ts
Normal file
@@ -0,0 +1,594 @@
|
||||
/**
|
||||
* Host side of one workflow run. The first worker result, unexpected death, or
|
||||
* cancellation-grace expiry owns settlement and closes message admission.
|
||||
* Pending starts share one abort signal; published children share idempotent
|
||||
* cleanup, and quiescence waits for both while synthesizing any missing end events.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/host
|
||||
*/
|
||||
|
||||
import { Worker } from 'node:worker_threads'
|
||||
import type { WorkerOptions } from 'node:worker_threads'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import { snapshotJsonValue } from '@deepseek-ai/dsh-session'
|
||||
import type SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentRun } from '@deepseek-ai/dsh-subagent'
|
||||
import type { WorkflowAgentEndInfo, WorkflowAgentInfo, WorkflowMeta, WorkflowResult, WorkflowRun, WorkflowRunId } from '@deepseek-ai/dsh-workflow'
|
||||
import { renderThrown } from './realm.ts'
|
||||
import type { ExecutionObserver } from './runtime.ts'
|
||||
import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
|
||||
import type { HostToWorkerPayloads, WorkerToHostMessage } from './protocol.ts'
|
||||
import type { ChildResult, ChildStartRequest, WorkerInit } from './types.ts'
|
||||
|
||||
/** One published child and its shared quiescent-disposal transaction. */
|
||||
interface ChildRecord {
|
||||
readonly run: SubagentRun
|
||||
disposal?: Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a built worker bundle or an unbuilt bootstrap that installs both tsx
|
||||
* transforms inside the worker. Both shapes clear `execArgv` and the ambient
|
||||
* environment; the unbuilt shape forwards only `TSX_TSCONFIG_PATH` for path
|
||||
* resolution.
|
||||
* @param init - the run payload, passed as `workerData`.
|
||||
* @returns the entry path or URL and the Worker options to spawn it with.
|
||||
*/
|
||||
function resolveWorkerSpawn(init: WorkerInit): { entry: string | URL; options: WorkerOptions } {
|
||||
/* v8 ignore next 3 -- the built-output arm: tests always run unbuilt (src/); the built-worker e2e exercises this shape for real */
|
||||
if (!import.meta.url.endsWith('.ts')) {
|
||||
return { entry: fileURLToPath(new URL('./worker.cjs', import.meta.url)), options: { workerData: init, env: {}, execArgv: [] } }
|
||||
}
|
||||
// Resolve tsx only for unbuilt consumers and install it before importing TS.
|
||||
const workerEntry = new URL('./worker.ts', import.meta.url)
|
||||
const tsxEsmApiEntry = import.meta.resolve('tsx/esm/api')
|
||||
const tsxCjsApiEntry = import.meta.resolve('tsx/cjs/api')
|
||||
const bootstrap = [
|
||||
`import { register as registerEsm } from ${JSON.stringify(tsxEsmApiEntry)}`,
|
||||
`import { register as registerCjs } from ${JSON.stringify(tsxCjsApiEntry)}`,
|
||||
'registerCjs()',
|
||||
'registerEsm()',
|
||||
`await import(${JSON.stringify(workerEntry.href)})`,
|
||||
].join('\n')
|
||||
return {
|
||||
entry: new URL(`data:text/javascript,${encodeURIComponent(bootstrap)}`),
|
||||
options: {
|
||||
workerData: init,
|
||||
env: process.env.TSX_TSCONFIG_PATH === undefined ? {} : { TSX_TSCONFIG_PATH: process.env.TSX_TSCONFIG_PATH },
|
||||
execArgv: [],
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* One live worker-engine run — the seam's {@link WorkflowRun}, returned by
|
||||
* `start()` directly. Owns the Worker, the child registry, and the result
|
||||
* settlement; `result` never rejects. `meta` is trusted same-process data
|
||||
* borrowed as immutable by the handle and lifecycle events. The holder-bound
|
||||
* SubagentRuntime handle is captured before the
|
||||
* engine returns this run, so unloading the engine removes only the ability to
|
||||
* start another workflow; this run can still start and clean up its children.
|
||||
*/
|
||||
export class WorkerRun implements WorkflowRun {
|
||||
/** Settles exactly once with the run's outcome; never rejects. */
|
||||
readonly result: Promise<WorkflowResult>
|
||||
private settleResolve!: (result: WorkflowResult) => void
|
||||
private settled = false
|
||||
/** A Result/death/grace outcome atomically won before teardown callbacks. */
|
||||
private terminalClaimed = false
|
||||
/** The first death signal closes worker-message admission and owns failure-time cleanup. */
|
||||
private workerDeathObserved = false
|
||||
private cancelReason: string | undefined
|
||||
private graceTimer: NodeJS.Timeout | undefined
|
||||
private readonly worker: Worker
|
||||
/** Set on `exit`: the thread is gone, so posting has nowhere to go. */
|
||||
private workerGone = false
|
||||
/** Accepted `child-start` messages — the terminate-path `agentsStarted` (see module doc). */
|
||||
private hostStarted = 0
|
||||
/** Published children by callId; an entry leaves only after disposal settles. */
|
||||
private readonly children = new Map<number, ChildRecord>()
|
||||
/** Provider starts that have not yet fulfilled or rejected. */
|
||||
private readonly pendingStarts = new Set<Promise<void>>()
|
||||
/** Started-but-not-ended agents by seq — the pairing ledger the HOST guarantees (see {@link endAgent}). */
|
||||
private readonly liveAgents = new Map<number, WorkflowAgentInfo>()
|
||||
private readonly quiescenceWaiters: (() => void)[] = []
|
||||
/** The per-run abort fanout every child start request carries. */
|
||||
private readonly controller = new AbortController()
|
||||
/** External start signal and the exact callback installed on it, retained only until first settle/teardown. */
|
||||
private inputSignal: AbortSignal | undefined
|
||||
private inputSignalAbort: (() => void) | undefined
|
||||
private disposed: Promise<void> | undefined
|
||||
|
||||
constructor(
|
||||
private readonly ctx: Context,
|
||||
private readonly subagents: SubagentRuntime,
|
||||
readonly id: WorkflowRunId,
|
||||
readonly meta: WorkflowMeta,
|
||||
private readonly parent: Agent,
|
||||
init: WorkerInit,
|
||||
private readonly provider: string,
|
||||
private readonly disposeGraceMs: number,
|
||||
private readonly observer: ExecutionObserver,
|
||||
signal: AbortSignal | undefined,
|
||||
) {
|
||||
this.result = new Promise<WorkflowResult>((resolve) => { this.settleResolve = resolve })
|
||||
// workerData rides the structured clone: args are plain JSON by the seam
|
||||
// contract, so the clone is total and doubles as the caller-isolation
|
||||
// copy (a clone failure throws loud out of start()).
|
||||
const { entry, options } = resolveWorkerSpawn(init)
|
||||
this.worker = new Worker(entry, options)
|
||||
this.worker.on('message', (message: WorkerToHostMessage) => { this.onMessage(message) })
|
||||
this.worker.on('error', (error) => { this.onWorkerDeath(`workflow worker failed: ${renderThrown(error)}`, false) })
|
||||
/* v8 ignore next -- messageerror: not constructible from the engine's own protocol (every payload is JSON data) */
|
||||
this.worker.on('messageerror', (error) => { this.onWorkerDeath(`workflow worker message failed to deserialize: ${renderThrown(error)}`, false) })
|
||||
this.worker.on('exit', (code) => {
|
||||
this.workerGone = true
|
||||
this.onWorkerDeath(`workflow worker exited before the run settled (exit code ${code})`, true)
|
||||
})
|
||||
if (signal?.aborted) {
|
||||
this.cancel('workflow start signal already aborted')
|
||||
} else if (signal !== undefined) {
|
||||
const onAbort = (): void => {
|
||||
this.detachInputSignal()
|
||||
this.cancel('workflow signal aborted')
|
||||
}
|
||||
this.inputSignal = signal
|
||||
this.inputSignalAbort = onAbort
|
||||
signal.addEventListener('abort', onAbort, { once: true })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel the run: the worker is told (its hooks start throwing and the
|
||||
* script dies at its next await), the required signal shared by every child
|
||||
* start is aborted, and the grace timer
|
||||
* arms: a run still unsettled `disposeGraceMs` later force-settles
|
||||
* `cancelled` and its worker is TERMINATED. Idempotent; the first reason
|
||||
* wins.
|
||||
* @param reason - human-readable cause (default `'workflow cancelled'`).
|
||||
*/
|
||||
cancel(reason?: string): void {
|
||||
// A settled run has nothing left to cancel, and a terminal source claimed
|
||||
// before its cleanup callbacks must exclude cancellation reentered by one
|
||||
// of those callbacks. Without the settled guard the
|
||||
// ordinary consumer path (await result, then dispose -> cancel) would arm
|
||||
// a grace timer nothing ever clears, pinning the run and its Worker
|
||||
// closure until the grace expires - a bounded leak per completed run.
|
||||
if (this.settled || this.terminalClaimed || this.cancelReason !== undefined) return
|
||||
this.cancelReason = reason ?? 'workflow cancelled'
|
||||
this.post(HostToWorkerType.Cancel, { reason: this.cancelReason })
|
||||
this.abortChildren(this.cancelReason)
|
||||
this.graceTimer = setTimeout(() => {
|
||||
// Cancellation already owns the race through cancelReason; close the
|
||||
// terminal boundary explicitly before observer teardown callbacks.
|
||||
this.terminalClaimed = true
|
||||
// The worker may no longer speak (it is about to be terminated): pair
|
||||
// every stranded start before the run settles, so ends precede
|
||||
// workflow/end.
|
||||
this.endStrandedAgents()
|
||||
this.settleResult(this.cancelledResult(this.hostStarted))
|
||||
void this.worker.terminate()
|
||||
}, this.disposeGraceMs)
|
||||
// unref'd: an armed grace timer must never hold the process open.
|
||||
this.graceTimer.unref()
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel + bounded settle + termination. Host-drives every registered
|
||||
* child's disposal IMMEDIATELY — a wedged worker can relay no dispose RPC,
|
||||
* and deferring child teardown to the post-terminate reap would spend the
|
||||
* whole grace waiting for a quiescence that cannot start, then return with
|
||||
* the disposals still in flight — so child disposal overlaps the same
|
||||
* grace the worker gets to settle (the worker's own dispose RPCs join the
|
||||
* shared per-child disposal). Waits (at most the grace) for the result and
|
||||
* child quiescence, then terminates the worker unconditionally — the
|
||||
* thread never outlives its run — and reaps whatever children remain
|
||||
* (their disposal is contained, not awaited past the grace, the same
|
||||
* abandonment the seam documents for a slow-disposing child). Idempotent;
|
||||
* safe on every path.
|
||||
* @returns resolves when the run's resources are released or abandoned.
|
||||
*/
|
||||
dispose(): Promise<void> {
|
||||
if (this.disposed !== undefined) return this.disposed
|
||||
// Claim the public transaction BEFORE its body invokes child/provider
|
||||
// disposal. A raw provider callback can reenter handle.dispose(); it must
|
||||
// join this promise rather than start a second traversal.
|
||||
const claimed = Promise.withResolvers<undefined>()
|
||||
this.disposed = claimed.promise
|
||||
void (async () => {
|
||||
this.detachInputSignal()
|
||||
this.cancel('workflow disposed')
|
||||
// cancel() deliberately becomes a no-op after terminal settlement, but
|
||||
// disposal still owns every registered child. Reap independently so an
|
||||
// already-settled workflow cannot wait on child quiescence before it has
|
||||
// started the surviving children's disposals. On an unsettled run this
|
||||
// joins the cancel path through the per-call cancellation/disposal gates.
|
||||
this.reapChildren('workflow disposed')
|
||||
await Promise.race([
|
||||
(async () => {
|
||||
await this.result
|
||||
await this.childQuiescence()
|
||||
})(),
|
||||
sleep(this.disposeGraceMs),
|
||||
])
|
||||
await this.worker.terminate()
|
||||
this.reapChildren('workflow disposed')
|
||||
})().then(
|
||||
() => { claimed.resolve(undefined) },
|
||||
/* v8 ignore next -- result/quiescence never reject and Worker.terminate is the only external promise */
|
||||
(error: unknown) => { claimed.reject(error) },
|
||||
)
|
||||
return this.disposed
|
||||
}
|
||||
|
||||
/** Post one message to the worker (payload looked up from the tag's map entry), tolerating a thread that is already gone. */
|
||||
private post<T extends HostToWorkerType>(type: T, payload: HostToWorkerPayloads[T]): void {
|
||||
if (this.workerGone || this.workerDeathObserved) return
|
||||
try {
|
||||
this.worker.postMessage({ type, ...payload })
|
||||
} catch (error: unknown) {
|
||||
// Only a teardown race can land here (every engine message is JSON
|
||||
// data, so serialization cannot fail); there is nothing left to
|
||||
// deliver to — log and move on.
|
||||
/* v8 ignore next -- postMessage teardown race (a throw between exit and its event): not constructible in-process */
|
||||
this.ctx.logger.warn(`workflow-worker-thread: postMessage failed: ${renderThrown(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
private onMessage(message: WorkerToHostMessage): void {
|
||||
// Node may emit `error`, then deliver an already-queued `message`, then
|
||||
// emit `exit`. The first death signal is the host's logical delivery
|
||||
// barrier: nothing arriving afterward may create a child, narrate after
|
||||
// workflow/end, or compete with the chosen outcome.
|
||||
if (this.workerDeathObserved) return
|
||||
switch (message.type) {
|
||||
case WorkerToHostType.Ready:
|
||||
this.post(HostToWorkerType.Go, {})
|
||||
break
|
||||
case WorkerToHostType.Phase:
|
||||
// Post-cancel narration is suppressed host-side: worker-side the
|
||||
// hooks throw once the cancel message is PROCESSED, but narration
|
||||
// already in flight (or emitted while the cancel crossed the
|
||||
// boundary) must not reach observers — nothing is emitted after
|
||||
// cancel() returns.
|
||||
if (this.cancelReason === undefined) this.observer.phase(message.title)
|
||||
break
|
||||
case WorkerToHostType.Log:
|
||||
if (this.cancelReason === undefined) this.observer.log(message.message)
|
||||
break
|
||||
case WorkerToHostType.AgentStart:
|
||||
this.liveAgents.set(message.info.seq, message.info)
|
||||
this.observer.agentStart(message.info)
|
||||
break
|
||||
case WorkerToHostType.AgentEnd:
|
||||
// NOT suppressed on cancel: cancelled children report their paired
|
||||
// agent-end with outcome 'cancelled'. The gate (with the termination
|
||||
// paths' synthesis) is what makes the one-pair-per-started-child
|
||||
// contract hold on every stop path.
|
||||
this.endAgent(message.info)
|
||||
break
|
||||
case WorkerToHostType.ChildStart:
|
||||
this.onChildStart(message.callId, message.request)
|
||||
break
|
||||
case WorkerToHostType.ChildDispose:
|
||||
this.onChildDispose(message.callId)
|
||||
break
|
||||
case WorkerToHostType.Result:
|
||||
this.onResult(message.result)
|
||||
break
|
||||
/* v8 ignore next 2 -- closed engine-owned union; the arm only makes adding a message type a compile error */
|
||||
default:
|
||||
assertNever(message, 'worker-to-host message')
|
||||
}
|
||||
}
|
||||
|
||||
/** Why a ready provider result may no longer be admitted to the worker. */
|
||||
private childAdmissionFailure(): { reason: string; rendered: string } | undefined {
|
||||
if (this.cancelReason !== undefined) {
|
||||
return { reason: this.cancelReason, rendered: `workflow run cancelled: ${this.cancelReason}` }
|
||||
}
|
||||
if (this.workerDeathObserved) {
|
||||
return { reason: 'workflow worker gone', rendered: 'workflow worker is no longer available' }
|
||||
}
|
||||
if (this.terminalClaimed) {
|
||||
return { reason: 'workflow settled', rendered: 'workflow run already settled' }
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
private onChildStart(callId: number, request: ChildStartRequest): void {
|
||||
const initialFailure = this.childAdmissionFailure()
|
||||
if (initialFailure !== undefined) {
|
||||
// Refuse after a terminal boundary: a child must never start on an
|
||||
// already-aborted signal (a provider subscribing only to future abort
|
||||
// events would never observe it).
|
||||
this.post(HostToWorkerType.ChildStartError, { callId, rendered: initialFailure.rendered })
|
||||
return
|
||||
}
|
||||
this.hostStarted += 1
|
||||
const task = this.startChild(callId, request)
|
||||
this.pendingStarts.add(task)
|
||||
void task.then(
|
||||
() => { this.finishPendingStart(task) },
|
||||
/* v8 ignore next -- startChild contains provider and cleanup failures */
|
||||
() => { this.finishPendingStart(task) },
|
||||
)
|
||||
}
|
||||
|
||||
/** Await one provider-owned startup transaction and publish only while admitted. */
|
||||
private async startChild(callId: number, request: ChildStartRequest): Promise<void> {
|
||||
let run: SubagentRun
|
||||
try {
|
||||
run = await this.subagents.start(this.provider, {
|
||||
prompt: [{ type: 'text', text: request.prompt }],
|
||||
parent: this.parent,
|
||||
signal: this.controller.signal,
|
||||
...request.schema !== undefined ? { outputSchema: request.schema } : {},
|
||||
...request.provider !== undefined || request.model !== undefined
|
||||
? {
|
||||
agentOptions: {
|
||||
...request.provider !== undefined ? { provider: request.provider } : {},
|
||||
...request.model !== undefined ? { model: request.model } : {},
|
||||
},
|
||||
}
|
||||
: {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
const failure = this.childAdmissionFailure()
|
||||
this.post(HostToWorkerType.ChildStartError, {
|
||||
callId,
|
||||
rendered: failure?.rendered ?? renderThrown(error),
|
||||
})
|
||||
return
|
||||
}
|
||||
const failure = this.childAdmissionFailure()
|
||||
if (failure !== undefined) {
|
||||
this.post(HostToWorkerType.ChildStartError, { callId, rendered: failure.rendered })
|
||||
try {
|
||||
await run.dispose()
|
||||
} catch (error: unknown) {
|
||||
this.ctx.logger.warn(`workflow-worker-thread: refused child dispose failed: ${renderThrown(error)}`)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
const record: ChildRecord = { run }
|
||||
this.children.set(callId, record)
|
||||
// Attach result forwarding before publishing the child handle. Because the
|
||||
// callback itself runs in a later microtask, ChildStarted is still posted
|
||||
// first even for an already-settled scripted provider.
|
||||
const forwardResult = run.result.then<() => void, () => void>(
|
||||
(result) => {
|
||||
try {
|
||||
const snapshot = snapshotJsonValue<ChildResult>({
|
||||
output: result.output,
|
||||
...result.structured !== undefined ? { structured: result.structured } : {},
|
||||
stopReason: result.stopReason,
|
||||
})
|
||||
if (snapshot === undefined) throw new TypeError('child result is not losslessly JSON-serializable')
|
||||
return () => { this.post(HostToWorkerType.ChildSettled, { callId, result: snapshot }) }
|
||||
} catch (error: unknown) {
|
||||
const rendered = `workflow child result could not cross the worker boundary: ${renderThrown(error)}`
|
||||
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
|
||||
}
|
||||
},
|
||||
(error: unknown) => {
|
||||
const rendered = renderThrown(error)
|
||||
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
|
||||
},
|
||||
)
|
||||
this.post(HostToWorkerType.ChildStarted, { callId, childId: run.id })
|
||||
void forwardResult.then((forward) => { forward() })
|
||||
}
|
||||
|
||||
private onChildDispose(callId: number): void {
|
||||
const record = this.children.get(callId)
|
||||
if (record === undefined) {
|
||||
// Already disposed host-side (a dispose() drive or a death reap beat
|
||||
// the RPC) — the ack is still owed (the worker-side wrapper awaits it).
|
||||
this.post(HostToWorkerType.ChildDisposed, { callId })
|
||||
return
|
||||
}
|
||||
// disposeChild never rejects (containment is inside), so the ack always follows.
|
||||
void this.disposeChild(callId, record).then(() => { this.post(HostToWorkerType.ChildDisposed, { callId }) })
|
||||
}
|
||||
|
||||
/**
|
||||
* Start (or join) one registered child's disposal; the registry entry
|
||||
* leaves when it settles. Memoized per callId: the worker's dispose RPC,
|
||||
* the dispose() host drive, and the reap can all land on the same child —
|
||||
* the child's `dispose()` runs once and every caller awaits that one
|
||||
* settlement. A rejection is contained (the subagent seam's dispose() is
|
||||
* not supposed to reject, but a backend that does anyway must not break
|
||||
* quiescence): logged, and the child still leaves the registry.
|
||||
* @param callId - the child's registry key.
|
||||
* @param record - the registered child (the caller looked it up).
|
||||
* @returns resolves when the disposal settled either way; never rejects.
|
||||
*/
|
||||
private disposeChild(callId: number, record: ChildRecord): Promise<void> {
|
||||
if (record.disposal !== undefined) return record.disposal
|
||||
record.disposal = Promise.resolve()
|
||||
.then(() => record.run.dispose())
|
||||
.catch((error: unknown) => {
|
||||
this.ctx.logger.warn(`workflow-worker-thread: child dispose failed: ${renderThrown(error)}`)
|
||||
})
|
||||
.then(() => { this.finishChild(callId) })
|
||||
return record.disposal
|
||||
}
|
||||
|
||||
/** Drop a child record and release quiescence waiters when all work ends. */
|
||||
private finishChild(callId: number): void {
|
||||
this.children.delete(callId)
|
||||
this.notifyChildQuiescence()
|
||||
}
|
||||
|
||||
/** Retire one provider startup transaction. */
|
||||
private finishPendingStart(task: Promise<void>): void {
|
||||
this.pendingStarts.delete(task)
|
||||
this.notifyChildQuiescence()
|
||||
}
|
||||
|
||||
/** Release waiters only after both pending starts and published children end. */
|
||||
private notifyChildQuiescence(): void {
|
||||
if (this.children.size !== 0 || this.pendingStarts.size !== 0) return
|
||||
for (const waiter of this.quiescenceWaiters.splice(0)) waiter()
|
||||
}
|
||||
|
||||
/** Resolves once every pending start and published child has reached quiescence. */
|
||||
private childQuiescence(): Promise<void> {
|
||||
if (this.children.size === 0 && this.pendingStarts.size === 0) return Promise.resolve()
|
||||
return new Promise((resolve) => { this.quiescenceWaiters.push(resolve) })
|
||||
}
|
||||
|
||||
/** Abort + dispose every registered child (worker death / final teardown); disposal is contained, not awaited. */
|
||||
private reapChildren(reason: string): void {
|
||||
this.abortChildren(this.cancelReason ?? reason)
|
||||
for (const [callId, record] of [...this.children]) {
|
||||
void this.disposeChild(callId, record)
|
||||
}
|
||||
}
|
||||
|
||||
/** Abort the one canonical signal shared by pending and published children. */
|
||||
private abortChildren(reason: string): void {
|
||||
if (!this.controller.signal.aborted) this.controller.abort(reason)
|
||||
}
|
||||
|
||||
private onResult(result: WorkflowResult): void {
|
||||
// The owned worker session sends one Result. Keep a late duplicate or a
|
||||
// Result queued behind another terminal source completely side-effect-free.
|
||||
if (this.terminalClaimed) return
|
||||
// First-wins is decided when the Result message reaches the host. If no
|
||||
// external cancellation was already in flight, this result won. Reaping a
|
||||
// stray child below may synchronously reenter cancel() through provider
|
||||
// callbacks, but that internal post-result cleanup must not retroactively
|
||||
// rewrite the worker result that arrived first.
|
||||
const cancellationWasRequested = this.cancelReason !== undefined
|
||||
// Claim before settlement cleanup invokes provider disposal. Once Result
|
||||
// won, a later cancellation cannot rewrite it.
|
||||
this.terminalClaimed = true
|
||||
// Abort pending starts and begin disposing published children before the
|
||||
// workflow becomes externally settled. Cleanup remains independently
|
||||
// tracked by childQuiescence and the holder's dispose().
|
||||
this.reapChildren('workflow settled')
|
||||
if (!cancellationWasRequested) {
|
||||
this.settleResult(result)
|
||||
return
|
||||
}
|
||||
if (result.stopReason !== 'cancelled') {
|
||||
// The script settled while our cancel was crossing the thread boundary
|
||||
// — the seam-visible result had NOT settled when cancellation was
|
||||
// requested, so report cancelled (the vm drive()'s post-settle check,
|
||||
// relocated to the receiving side of the race).
|
||||
this.settleResult(this.cancelledResult(result.agentsStarted))
|
||||
return
|
||||
}
|
||||
this.settleResult(result)
|
||||
}
|
||||
|
||||
/** Process an error/messageerror/exit signal; `exit` also performs the final disposal sweep. */
|
||||
private onWorkerDeath(message: string, isExit: boolean): void {
|
||||
if (!this.workerDeathObserved) {
|
||||
// Close message admission BEFORE cleanup callbacks: Node can deliver a
|
||||
// message queued before the crash after its `error` event. Treating the
|
||||
// first death signal as a logical barrier prevents that late message
|
||||
// from creating work or narrating after workflow/end.
|
||||
this.workerDeathObserved = true
|
||||
const outcomeWasClaimed = this.terminalClaimed
|
||||
const cancellationWasRequested = this.cancelReason !== undefined
|
||||
// When death is itself the terminal source, claim BEFORE child reap or
|
||||
// synthesized observer callbacks. Either can reenter cancel(); a death
|
||||
// that arrived first remains an error, while a cancellation already
|
||||
// accepted before death remains cancelled. If Result/grace already won,
|
||||
// preserve it while still performing prompt failure-time cleanup.
|
||||
if (!outcomeWasClaimed) this.terminalClaimed = true
|
||||
if (this.children.size > 0 || this.pendingStarts.size > 0) this.reapChildren('workflow worker gone')
|
||||
this.endStrandedAgents()
|
||||
if (!outcomeWasClaimed) {
|
||||
if (cancellationWasRequested) {
|
||||
this.settleResult(this.cancelledResult(this.hostStarted))
|
||||
} else {
|
||||
this.settleResult({ value: null, stopReason: 'error', error: message, agentsStarted: this.hostStarted })
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!isExit) return
|
||||
// `error` is not Node's physical delivery barrier: a queued message may
|
||||
// precede `exit`. Admission is already closed, so this final sweep only
|
||||
// joins/starts disposal for registry survivors; it deliberately does not
|
||||
// repeat explicit provider cancellation.
|
||||
for (const [callId, record] of [...this.children]) void this.disposeChild(callId, record)
|
||||
this.endStrandedAgents()
|
||||
}
|
||||
|
||||
/**
|
||||
* The single agent-end emission gate: forwards `end` iff its start is still
|
||||
* unpaired in the ledger, so every forwarded `workflow/agent-start` gets
|
||||
* EXACTLY one `workflow/agent-end` — the worker's own report where it can
|
||||
* speak, a host-synthesized one where it cannot ({@link endStrandedAgents}).
|
||||
* @param end - the settlement to emit (worker-reported or synthesized).
|
||||
*/
|
||||
private endAgent(end: WorkflowAgentEndInfo): void {
|
||||
/* v8 ignore next -- a real end still in flight across the grace force-settle: not orderable in-process */
|
||||
if (!this.liveAgents.delete(end.seq)) return
|
||||
this.observer.agentEnd(end)
|
||||
}
|
||||
|
||||
/**
|
||||
* Synthesize the missing `agent-end` for every started-but-unpaired agent,
|
||||
* outcome `'cancelled'`: the reap cancels every child, and a real
|
||||
* settlement racing the force-settle loses to that already-started external
|
||||
* cancellation. The atomic terminal boundaries in {@link onResult} and
|
||||
* {@link onWorkerDeath} deliberately exclude teardown callbacks as contenders.
|
||||
* Called where the worker can no longer speak (the grace force-settle,
|
||||
* worker death, physical exit). When grace/death is the terminal source it
|
||||
* runs before settleResult, so already-known pairs precede `workflow/end`;
|
||||
* after an earlier Result, exit cleanup may close a survivor afterward.
|
||||
* The ledger preserves exactly-once pairing in both orders.
|
||||
*/
|
||||
private endStrandedAgents(): void {
|
||||
for (const info of [...this.liveAgents.values()]) {
|
||||
this.endAgent({ ...info, outcome: 'cancelled' })
|
||||
}
|
||||
}
|
||||
|
||||
private cancelledResult(agentsStarted: number): WorkflowResult {
|
||||
// cancel() is the only writer of cancelReason and every caller checks it
|
||||
// first; the fallback guards the type, not a reachable path.
|
||||
/* v8 ignore next */
|
||||
const reason = this.cancelReason ?? 'workflow cancelled'
|
||||
return { value: null, stopReason: 'cancelled', error: `workflow run cancelled: ${reason}`, agentsStarted }
|
||||
}
|
||||
|
||||
/** Remove the exact abort callback installed on the caller's start signal. */
|
||||
private detachInputSignal(): void {
|
||||
const signal = this.inputSignal
|
||||
const onAbort = this.inputSignalAbort
|
||||
if (signal === undefined || onAbort === undefined) return
|
||||
this.inputSignal = undefined
|
||||
this.inputSignalAbort = undefined
|
||||
signal.removeEventListener('abort', onAbort)
|
||||
}
|
||||
|
||||
/** First settle wins; disarms the grace timer and releases the caller signal. */
|
||||
private settleResult(result: WorkflowResult): void {
|
||||
// Every current terminal source claims ownership before calling here; keep
|
||||
// the fallback local so a future caller cannot resolve twice.
|
||||
/* v8 ignore next -- defensive fallback outside the claimed state machine */
|
||||
if (this.settled) return
|
||||
this.terminalClaimed = true
|
||||
this.settled = true
|
||||
this.detachInputSignal()
|
||||
clearTimeout(this.graceTimer)
|
||||
this.settleResolve(result)
|
||||
}
|
||||
}
|
||||
|
||||
/** A plain timer sleep (the dispose grace); unref'd so it never holds the process open. */
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const timer = setTimeout(resolve, ms)
|
||||
timer.unref()
|
||||
})
|
||||
}
|
||||
205
packages/workflow/workflow-worker-thread/src/index.ts
Normal file
205
packages/workflow/workflow-worker-thread/src/index.ts
Normal file
@@ -0,0 +1,205 @@
|
||||
/**
|
||||
* Worker-thread workflow engine. Each run executes its model-written script in
|
||||
* an escapable vm context on a fresh worker and bridges `agent()` calls to host
|
||||
* subagents. The thread prevents synchronous script work from blocking the host
|
||||
* and permits forced termination, but it is containment rather than a security boundary.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { availableParallelism } from 'node:os'
|
||||
import * as vm from 'node:vm'
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import WorkflowEngine, { WorkflowError, WorkflowRunId } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowRun, WorkflowRunInfo, WorkflowStartRequest } from '@deepseek-ai/dsh-workflow'
|
||||
import { WorkerRun } from './host.ts'
|
||||
import { validateMeta } from './meta.ts'
|
||||
import type { WorkerInit, WorkerLimits } from './types.ts'
|
||||
|
||||
export { validateMeta } from './meta.ts'
|
||||
export { materializeFromRealm, MaterializeError } from './realm.ts'
|
||||
export type {
|
||||
ChildHandle,
|
||||
ChildPort,
|
||||
ChildResult,
|
||||
ChildStartRequest,
|
||||
WorkerInit,
|
||||
WorkerLimits,
|
||||
} from './types.ts'
|
||||
|
||||
/** Plugin config (all optional — `static Config` supplies the defaults). */
|
||||
export interface Config {
|
||||
/** The `ctx.subagents` provider children run on (default `spawn`). */
|
||||
provider?: string
|
||||
/** Concurrent `agent()` ceiling; `0` (the default) auto-resolves to `min(16, max(1, cores - 2))`. */
|
||||
maxConcurrentAgents?: number
|
||||
/** Total `agent()` calls one run may start — the runaway-loop backstop (default 1000). */
|
||||
maxTotalAgents?: number
|
||||
/** Items accepted by a single `parallel()`/`pipeline()` call (default 4096). */
|
||||
maxItemsPerCall?: number
|
||||
/** vm timeout for the script's initial synchronous slice, inside the worker (default 5000 ms). */
|
||||
syncTimeoutMs?: number
|
||||
/**
|
||||
* How long after a cancellation an unsettled script may keep running before
|
||||
* the run force-settles `cancelled` and its worker is TERMINATED (default
|
||||
* 5000 ms); also bounds `dispose()`.
|
||||
*/
|
||||
disposeGraceMs?: number
|
||||
}
|
||||
|
||||
type ResolvedConfig = Required<Config>
|
||||
|
||||
/** A body that still carries the Claude Code-style meta header (meta rides the seam as data here). */
|
||||
const META_STATEMENT = /^\s*export\s+const\s+meta\b/
|
||||
|
||||
/**
|
||||
* Parse-check the body with the SAME wrapper the worker-side runtime
|
||||
* compiles, so `start()` keeps the seam's synchronous `SCRIPT_PARSE` throw
|
||||
* (the worker's own compile happens a thread away, after `start()` returned).
|
||||
* One redundant parse per run, bought deliberately for the contract. A body
|
||||
* opening with `export const meta` gets a pointed message instead of the
|
||||
* wrapper's bare SyntaxError — the model's likeliest authoring slip.
|
||||
*/
|
||||
function assertBodyParses(body: string, name: string): void {
|
||||
if (META_STATEMENT.test(body)) {
|
||||
throw new WorkflowError('workflow meta rides the `meta` request field, not the script: remove the `export const meta = {...}` statement from the body', 'SCRIPT_PARSE')
|
||||
}
|
||||
try {
|
||||
// Parse only — the script object is discarded, nothing executes.
|
||||
void new vm.Script(`(async () => {\n${body}\n})()`, { filename: `workflow:${name}`, lineOffset: -1 })
|
||||
} catch (error: unknown) {
|
||||
throw new WorkflowError(`workflow script does not parse: ${String(error)}`, 'SCRIPT_PARSE', { cause: error })
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolve one run's provider route before publishing work. */
|
||||
function resolveSubagentProvider(ctx: Context, configured: string, override: string | undefined): string {
|
||||
const provider = override ?? configured
|
||||
if (provider.length === 0 || provider !== provider.trim()) {
|
||||
throw new WorkflowError(
|
||||
'workflow subagentProvider must be a non-empty normalized string',
|
||||
'INVALID_ARGUMENT',
|
||||
)
|
||||
}
|
||||
if (ctx.subagents.getProvider(provider) === undefined) {
|
||||
throw new WorkflowError(`no subagent provider registered for "${provider}"`, 'AGENT_START')
|
||||
}
|
||||
return provider
|
||||
}
|
||||
|
||||
/** Resolve one run's total-child cap against the engine deployment ceiling. */
|
||||
function resolveMaxTotalAgents(requested: number | undefined, ceiling: number): number {
|
||||
if (requested === undefined) return ceiling
|
||||
if (!Number.isSafeInteger(requested) || requested < 1) {
|
||||
throw new WorkflowError('workflow maxTotalAgents must be a positive safe integer', 'INVALID_ARGUMENT')
|
||||
}
|
||||
if (requested > ceiling) {
|
||||
throw new WorkflowError(
|
||||
`workflow maxTotalAgents ${requested} exceeds the engine ceiling ${ceiling}`,
|
||||
'INVALID_ARGUMENT',
|
||||
)
|
||||
}
|
||||
return requested
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-thread engine service. `start()` validates the script up front
|
||||
* (meta + a host-side body parse) and returns a {@link WorkflowRun} whose
|
||||
* `result` never rejects; the `workflow/*` events fire around the run per
|
||||
* the seam contract.
|
||||
*/
|
||||
class WorkerThreadWorkflowEngine extends WorkflowEngine {
|
||||
static inject = ['subagents']
|
||||
|
||||
static Config: z<Config> = z.object({
|
||||
provider: z.string().default('spawn'),
|
||||
maxConcurrentAgents: z.natural().default(0),
|
||||
maxTotalAgents: z.natural().min(1).default(1000),
|
||||
maxItemsPerCall: z.natural().min(1).default(4096),
|
||||
syncTimeoutMs: z.natural().min(1).default(5000),
|
||||
disposeGraceMs: z.natural().default(5000),
|
||||
})
|
||||
|
||||
private readonly config: ResolvedConfig
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
// schemastery (static Config) has already filled the defaulted fields;
|
||||
// the assertion records that resolution, not a hidden fallback.
|
||||
this.config = config as ResolvedConfig
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and execute a workflow script in a fresh worker thread. Throws
|
||||
* {@link WorkflowError} synchronously (`META_INVALID` for a malformed meta
|
||||
* block, `SCRIPT_PARSE` for a body that does not compile) for a request
|
||||
* that cannot begin; once a run is returned, every failure resolves through
|
||||
* `result.stopReason` instead.
|
||||
* @param request - the script body, its meta data and `args`, the parent
|
||||
* agent, and an optional cancel signal.
|
||||
* @returns the live run (its `result` resolves when the script settles).
|
||||
*/
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
const meta = validateMeta(request.meta)
|
||||
assertBodyParses(request.script, meta.name)
|
||||
const subagentProvider = resolveSubagentProvider(this.ctx, this.config.provider, request.subagentProvider)
|
||||
const maxTotalAgents = resolveMaxTotalAgents(request.maxTotalAgents, this.config.maxTotalAgents)
|
||||
const id = WorkflowRunId(randomUUID())
|
||||
const info: WorkflowRunInfo = { id, meta }
|
||||
const limits: WorkerLimits = {
|
||||
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
|
||||
? Math.min(16, Math.max(1, availableParallelism() - 2))
|
||||
: this.config.maxConcurrentAgents,
|
||||
maxTotalAgents,
|
||||
maxItemsPerCall: this.config.maxItemsPerCall,
|
||||
syncTimeoutMs: this.config.syncTimeoutMs,
|
||||
}
|
||||
const init: WorkerInit = {
|
||||
meta,
|
||||
body: request.script,
|
||||
...request.args !== undefined ? { args: request.args } : {},
|
||||
limits,
|
||||
}
|
||||
// Capture the dependency while this service call is still traced through
|
||||
// the start() holder. Cordis strips the engine-provider shadow when it
|
||||
// returns the SubagentRuntime handle, so an already-returned run can keep
|
||||
// starting children after an engine HMR unload removes ctx.workflowEngine.
|
||||
// Re-resolving `this.ctx.subagents` later from WorkerRun would instead walk
|
||||
// the now-inactive engine fiber and break the seam's holder-owned lifetime.
|
||||
const runCtx = this.ctx
|
||||
const subagents = runCtx.subagents
|
||||
const workerRun = new WorkerRun(
|
||||
runCtx,
|
||||
subagents,
|
||||
id,
|
||||
meta,
|
||||
request.parent,
|
||||
init,
|
||||
subagentProvider,
|
||||
this.config.disposeGraceMs,
|
||||
{
|
||||
phase: (title) => { this.emitWorkflowEvent('workflow/phase', info, title) },
|
||||
log: (message) => { this.emitWorkflowEvent('workflow/log', info, message) },
|
||||
agentStart: (agent) => { this.emitWorkflowEvent('workflow/agent-start', info, agent) },
|
||||
agentEnd: (agent) => { this.emitWorkflowEvent('workflow/agent-end', info, agent) },
|
||||
},
|
||||
request.signal,
|
||||
)
|
||||
|
||||
this.emitWorkflowEvent('workflow/start', info)
|
||||
// `workflow/end` fires as the (never-rejecting) result settles, with the
|
||||
// outcome DATA only — the value stays with the run's holder.
|
||||
void workerRun.result.then((settled) => {
|
||||
this.emitWorkflowEvent('workflow/end', info, {
|
||||
stopReason: settled.stopReason,
|
||||
...settled.error !== undefined ? { error: settled.error } : {},
|
||||
agentsStarted: settled.agentsStarted,
|
||||
})
|
||||
})
|
||||
|
||||
return workerRun
|
||||
}
|
||||
}
|
||||
|
||||
export default WorkerThreadWorkflowEngine
|
||||
30
packages/workflow/workflow-worker-thread/src/invariant.ts
Normal file
30
packages/workflow/workflow-worker-thread/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-workflow-worker-thread`.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from '@deepseek-ai/cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-workflow-worker-thread'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'workflow-worker-thread-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this process-boundary implementation exposes no same-process event relation;
|
||||
* worker protocol and built-worker tests cover it.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
82
packages/workflow/workflow-worker-thread/src/meta.ts
Normal file
82
packages/workflow/workflow-worker-thread/src/meta.ts
Normal file
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* Meta validation checks caller-provided DATA against the {@link WorkflowMeta}
|
||||
* contract and rejects every violation by name. Meta arrives as schema-checked
|
||||
* JSON data, never evaluated script text; evaluating it on the host could run getters outside the
|
||||
* worker timeout that exists to isolate model-written code.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/meta
|
||||
*/
|
||||
|
||||
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowMeta, WorkflowPhase } from '@deepseek-ai/dsh-workflow'
|
||||
|
||||
/** Collect shape violations for a meta value (plain JSON data by the seam contract). */
|
||||
function validateMetaShape(meta: unknown): { meta?: WorkflowMeta; violations: string[] } {
|
||||
const violations: string[] = []
|
||||
if (typeof meta !== 'object' || meta === null || Array.isArray(meta)) {
|
||||
return { violations: ['meta must be an object'] }
|
||||
}
|
||||
const record = meta as Record<string, unknown>
|
||||
const known = new Set(['name', 'description', 'whenToUse', 'phases'])
|
||||
for (const key of Object.keys(record)) {
|
||||
if (!known.has(key)) violations.push(`meta.${key} is not a recognized field (name/description/whenToUse/phases)`)
|
||||
}
|
||||
if (typeof record.name !== 'string' || record.name.length === 0) violations.push('meta.name must be a non-empty string')
|
||||
if (typeof record.description !== 'string' || record.description.length === 0) violations.push('meta.description must be a non-empty string')
|
||||
if (record.whenToUse !== undefined && typeof record.whenToUse !== 'string') violations.push('meta.whenToUse must be a string')
|
||||
const phases: WorkflowPhase[] = []
|
||||
if (record.phases !== undefined) {
|
||||
if (!Array.isArray(record.phases)) {
|
||||
violations.push('meta.phases must be an array')
|
||||
} else {
|
||||
record.phases.forEach((phase, index) => {
|
||||
if (typeof phase !== 'object' || phase === null || Array.isArray(phase)) {
|
||||
violations.push(`meta.phases[${index}] must be an object`)
|
||||
return
|
||||
}
|
||||
const entry = phase as Record<string, unknown>
|
||||
for (const key of Object.keys(entry)) {
|
||||
if (!['title', 'detail', 'provider', 'model'].includes(key)) violations.push(`meta.phases[${index}].${key} is not a recognized field`)
|
||||
}
|
||||
if (typeof entry.title !== 'string' || entry.title.length === 0) violations.push(`meta.phases[${index}].title must be a non-empty string`)
|
||||
if (entry.detail !== undefined && typeof entry.detail !== 'string') violations.push(`meta.phases[${index}].detail must be a string`)
|
||||
if (entry.provider !== undefined && typeof entry.provider !== 'string') violations.push(`meta.phases[${index}].provider must be a string`)
|
||||
if (entry.model !== undefined && typeof entry.model !== 'string') violations.push(`meta.phases[${index}].model must be a string`)
|
||||
if (violations.length === 0) {
|
||||
phases.push({
|
||||
title: entry.title as string,
|
||||
...entry.detail !== undefined ? { detail: entry.detail as string } : {},
|
||||
...entry.provider !== undefined ? { provider: entry.provider as string } : {},
|
||||
...entry.model !== undefined ? { model: entry.model as string } : {},
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
if (violations.length > 0) return { violations }
|
||||
return {
|
||||
violations,
|
||||
meta: {
|
||||
name: record.name as string,
|
||||
description: record.description as string,
|
||||
...record.whenToUse !== undefined ? { whenToUse: record.whenToUse as string } : {},
|
||||
...record.phases !== undefined ? { phases } : {},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a caller-provided meta value against the {@link WorkflowMeta}
|
||||
* contract. Throws `META_INVALID` naming every violation (unknown fields,
|
||||
* missing/mistyped `name`/`description`, malformed `phases`); the returned
|
||||
* meta is a NORMALIZED copy built from the validated fields, so the engine
|
||||
* never aliases the caller's object.
|
||||
* @param value - the meta data from the start request (plain JSON by the seam contract).
|
||||
* @returns the validated, normalized meta block.
|
||||
*/
|
||||
export function validateMeta(value: unknown): WorkflowMeta {
|
||||
const { meta, violations } = validateMetaShape(value)
|
||||
if (meta === undefined) {
|
||||
throw new WorkflowError(`invalid meta: ${violations.join('; ')}`, 'META_INVALID')
|
||||
}
|
||||
return meta
|
||||
}
|
||||
101
packages/workflow/workflow-worker-thread/src/protocol.ts
Normal file
101
packages/workflow/workflow-worker-thread/src/protocol.ts
Normal file
@@ -0,0 +1,101 @@
|
||||
/**
|
||||
* The host⇄worker wire protocol: one string-valued enum of message tags per direction, a
|
||||
* payload map giving each tag its parameters (the single source of truth), and the message
|
||||
* unions derived from them. Payloads are plain JSON by construction for structured clone. Both
|
||||
* directions are closed engine protocols whose receivers use `assertNever`; generic typed senders
|
||||
* make tag/payload mismatches compile-time errors rather than silently skipped messages.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/protocol
|
||||
*/
|
||||
|
||||
import type { WorkflowAgentEndInfo, WorkflowAgentInfo, WorkflowResult } from '@deepseek-ai/dsh-workflow'
|
||||
import type { ChildResult, ChildStartRequest } from './types.ts'
|
||||
|
||||
/** Message tags the worker sends the host (the wire values are the tag strings). */
|
||||
export enum WorkerToHostType {
|
||||
/** The startup handshake: the session is listening and awaits {@link HostToWorkerType.Go}. */
|
||||
Ready = 'ready',
|
||||
/** Observer narration: a `phase(title)` call. */
|
||||
Phase = 'phase',
|
||||
/** Observer narration: a `log(message)` call. */
|
||||
Log = 'log',
|
||||
/** Observer lifecycle: one `agent()` call started a child. */
|
||||
AgentStart = 'agent-start',
|
||||
/** Observer lifecycle: one `agent()` call settled. */
|
||||
AgentEnd = 'agent-end',
|
||||
/** Child RPC: start a child on the host (answered by ChildStarted or ChildStartError). */
|
||||
ChildStart = 'child-start',
|
||||
/** Child RPC: dispose a started child (answered by ChildDisposed). */
|
||||
ChildDispose = 'child-dispose',
|
||||
/** The run's single terminal result. */
|
||||
Result = 'result',
|
||||
}
|
||||
|
||||
/** The payload each worker→host tag carries. */
|
||||
export interface WorkerToHostPayloads {
|
||||
/** Ready carries nothing. */
|
||||
[WorkerToHostType.Ready]: Record<never, never>
|
||||
/** The phase title, verbatim. */
|
||||
[WorkerToHostType.Phase]: { title: string }
|
||||
/** The logged message, verbatim. */
|
||||
[WorkerToHostType.Log]: { message: string }
|
||||
/** The call's sequence number, label, phase, and child id. */
|
||||
[WorkerToHostType.AgentStart]: { info: WorkflowAgentInfo }
|
||||
/** The call identity plus its outcome. */
|
||||
[WorkerToHostType.AgentEnd]: { info: WorkflowAgentEndInfo }
|
||||
/** The RPC correlation id and the prompt plus validated options. */
|
||||
[WorkerToHostType.ChildStart]: { callId: number; request: ChildStartRequest }
|
||||
/** The RPC correlation id of the child to dispose. */
|
||||
[WorkerToHostType.ChildDispose]: { callId: number }
|
||||
/** The run's terminal outcome. */
|
||||
[WorkerToHostType.Result]: { result: WorkflowResult }
|
||||
}
|
||||
|
||||
/** Message tags the host sends the worker (the wire values are the tag strings). */
|
||||
export enum HostToWorkerType {
|
||||
/** Releases the startup gate: run the script body. */
|
||||
Go = 'go',
|
||||
/** Cancel the run: hooks start throwing and the script dies at its next await. */
|
||||
Cancel = 'cancel',
|
||||
/** Child RPC reply: the provider fulfilled with a published run (exactly one start reply per ChildStart). */
|
||||
ChildStarted = 'child-started',
|
||||
/** Child RPC reply: the provider's asynchronous start failed. */
|
||||
ChildStartError = 'child-start-error',
|
||||
/** Child RPC: a started child's result RESOLVED (its JSON projection). */
|
||||
ChildSettled = 'child-settled',
|
||||
/** Child RPC: a started child's result REJECTED (an infrastructure fault, rendered). */
|
||||
ChildFailed = 'child-failed',
|
||||
/** Child RPC reply: a requested disposal completed. */
|
||||
ChildDisposed = 'child-disposed',
|
||||
}
|
||||
|
||||
/** The payload each host→worker tag carries. */
|
||||
export interface HostToWorkerPayloads {
|
||||
/** Go carries nothing. */
|
||||
[HostToWorkerType.Go]: Record<never, never>
|
||||
/** The cancel reason, canonical for the whole run. */
|
||||
[HostToWorkerType.Cancel]: { reason: string }
|
||||
/** The RPC correlation id and the child agent's id (minted by the subagent seam). */
|
||||
[HostToWorkerType.ChildStarted]: { callId: number; childId: string }
|
||||
/** The RPC correlation id and the rendered start failure. */
|
||||
[HostToWorkerType.ChildStartError]: { callId: number; rendered: string }
|
||||
/** The RPC correlation id and the child's terminal result projection. */
|
||||
[HostToWorkerType.ChildSettled]: { callId: number; result: ChildResult }
|
||||
/** The RPC correlation id and the rendered infrastructure fault. */
|
||||
[HostToWorkerType.ChildFailed]: { callId: number; rendered: string }
|
||||
/** The RPC correlation id of the completed disposal. */
|
||||
[HostToWorkerType.ChildDisposed]: { callId: number }
|
||||
}
|
||||
|
||||
/**
|
||||
* One worker→host message of tag `T`; unparameterized, the closed union over
|
||||
* every tag (a discriminated union — `switch` on `type` narrows).
|
||||
*/
|
||||
export type WorkerToHostMessage<T extends WorkerToHostType = WorkerToHostType> =
|
||||
{ [K in T]: { type: K } & WorkerToHostPayloads[K] }[T]
|
||||
|
||||
/**
|
||||
* One host→worker message of tag `T`; unparameterized, the closed union over
|
||||
* every tag (a discriminated union — `switch` on `type` narrows).
|
||||
*/
|
||||
export type HostToWorkerMessage<T extends HostToWorkerType = HostToWorkerType> =
|
||||
{ [K in T]: { type: K } & HostToWorkerPayloads[K] }[T]
|
||||
151
packages/workflow/workflow-worker-thread/src/realm.ts
Normal file
151
packages/workflow/workflow-worker-thread/src/realm.ts
Normal file
@@ -0,0 +1,151 @@
|
||||
/**
|
||||
* Materializes values leaving the script vm into plain JSON before they cross the worker
|
||||
* boundary, and renders thrown script values without rejecting the run. The walk rejects
|
||||
* values that JSON cannot preserve but trusts model-written workflow scripts: getters and proxy traps may
|
||||
* run, and the vm is not a security boundary. The worker provides host-loop isolation and
|
||||
* forced termination, not hostile-value containment. See
|
||||
* .agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md for the isolation rationale.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/realm
|
||||
*/
|
||||
|
||||
/** Thrown by {@link materializeFromRealm}; the caller wraps it into the right `WorkflowError` code. */
|
||||
export class MaterializeError extends Error {
|
||||
constructor(public readonly path: string, public readonly reason: string) {
|
||||
super(`${path}: ${reason}`)
|
||||
this.name = 'MaterializeError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render a thrown value to failure text without ever throwing: prefer the
|
||||
* `stack` (host or realm — a realm error's `stack` is a plain string read),
|
||||
* fall back to `message`, then `String()`. Reading those properties MAY run
|
||||
* script code (a getter, `toString`) — accepted under the module's trust
|
||||
* premise; if that code itself throws, a fixed label is returned instead.
|
||||
* @param error - any value thrown in the host or worker realm.
|
||||
* @returns human-readable text for the failure report; prefers the stack.
|
||||
*/
|
||||
export function renderThrown(error: unknown): string {
|
||||
try {
|
||||
const stack = (error as { stack?: unknown } | null | undefined)?.stack
|
||||
if (typeof stack === 'string' && stack.length > 0) return stack
|
||||
const message = (error as { message?: unknown } | null | undefined)?.message
|
||||
if (typeof message === 'string' && message.length > 0) return message
|
||||
return String(error)
|
||||
} catch {
|
||||
// A throwing accessor/toString on the thrown value — rendering must be
|
||||
// total (drive()'s never-reject contract), so fall back to a fixed label.
|
||||
return '[unrenderable thrown value]'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether an object's prototype chain represents a plain data object: `null`, or a prototype
|
||||
* whose own prototype is `null` (the realm's `Object.prototype` — which we
|
||||
* cannot compare by identity across realms). A `Date`/`Map`/class instance
|
||||
* has a longer chain and is rejected.
|
||||
*/
|
||||
function hasPlainPrototype(value: object): boolean {
|
||||
const proto: unknown = Object.getPrototypeOf(value)
|
||||
if (proto === null) return true
|
||||
return Object.getPrototypeOf(proto) === null
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy `value` (typically from the vm realm) into plain host JSON data. Root `undefined` is
|
||||
* returned unchanged; nested `undefined` and values JSON cannot represent losslessly fail
|
||||
* with the offending path. Property accessors run normally, and a throwing read is wrapped
|
||||
* with its rendered failure.
|
||||
*
|
||||
* @param value - the realm value to materialize.
|
||||
* @param root - the path label for the root value (error messages).
|
||||
* @returns the host-realm copy (plain objects/arrays/scalars only).
|
||||
* @throws {@link MaterializeError} for unsupported values, cycles, sparse arrays, exotic
|
||||
* prototypes, or property reads that throw.
|
||||
*/
|
||||
export function materializeFromRealm(value: unknown, root = 'value'): unknown {
|
||||
if (value === undefined) return undefined
|
||||
try {
|
||||
return materialize(value, root, new Set())
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof MaterializeError) throw error
|
||||
// A property read ran script code that threw; total-ize it so callers can
|
||||
// keep the narrow MaterializeError contract.
|
||||
throw new MaterializeError(root, `reading the value threw: ${renderThrown(error)}`)
|
||||
}
|
||||
}
|
||||
|
||||
function materialize(value: unknown, path: string, seen: Set<object>): unknown {
|
||||
switch (typeof value) {
|
||||
case 'boolean':
|
||||
case 'string':
|
||||
return value
|
||||
case 'number': {
|
||||
if (!Number.isFinite(value)) throw new MaterializeError(path, 'non-finite numbers are not JSON data')
|
||||
return value
|
||||
}
|
||||
case 'bigint':
|
||||
throw new MaterializeError(path, 'bigints are not JSON data')
|
||||
case 'function':
|
||||
throw new MaterializeError(path, 'functions are not plain JSON data')
|
||||
case 'symbol':
|
||||
throw new MaterializeError(path, 'symbols are not plain JSON data')
|
||||
case 'undefined':
|
||||
throw new MaterializeError(path, 'undefined is not JSON data')
|
||||
case 'object':
|
||||
break
|
||||
}
|
||||
if (value === null) return null
|
||||
const objectValue: object = value
|
||||
if (seen.has(objectValue)) throw new MaterializeError(path, 'circular references are not JSON data')
|
||||
seen.add(objectValue)
|
||||
try {
|
||||
if (Array.isArray(objectValue)) return materializeArray(objectValue, path, seen)
|
||||
return materializeObject(objectValue, path, seen)
|
||||
} finally {
|
||||
seen.delete(objectValue)
|
||||
}
|
||||
}
|
||||
|
||||
function materializeArray(value: unknown[], path: string, seen: Set<object>): unknown[] {
|
||||
const out: unknown[] = []
|
||||
for (let index = 0; index < value.length; index++) {
|
||||
if (!(index in value)) throw new MaterializeError(`${path}[${index}]`, 'sparse arrays are not JSON data')
|
||||
out.push(materialize(value[index], `${path}[${index}]`, seen))
|
||||
}
|
||||
// Own enumerable props beyond the indices (e.g. `arr.total = 3`) would be
|
||||
// silently dropped by JSON — reject them instead.
|
||||
for (const key of Object.keys(value)) {
|
||||
const index = Number(key)
|
||||
if (!Number.isInteger(index) || index < 0 || index >= value.length) {
|
||||
throw new MaterializeError(`${path}.${key}`, 'arrays with non-index properties are not JSON data')
|
||||
}
|
||||
}
|
||||
if (Object.getOwnPropertySymbols(value).length > 0) {
|
||||
throw new MaterializeError(path, 'symbol-keyed properties are not plain JSON data')
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function materializeObject(value: object, path: string, seen: Set<object>): Record<string, unknown> {
|
||||
if (!hasPlainPrototype(value)) {
|
||||
throw new MaterializeError(path, 'only plain objects and arrays are JSON data (exotic prototype)')
|
||||
}
|
||||
if (Object.getOwnPropertySymbols(value).length > 0) {
|
||||
throw new MaterializeError(path, 'symbol-keyed properties are not plain JSON data')
|
||||
}
|
||||
const out: Record<string, unknown> = {}
|
||||
// Object.keys = own enumerable string keys, matching JSON.stringify's
|
||||
// property selection exactly (non-enumerable props never reach JSON output).
|
||||
for (const key of Object.keys(value)) {
|
||||
// defineProperty, never assignment: a "__proto__" key must become an OWN
|
||||
// data property of the copy, not a prototype mutation.
|
||||
Object.defineProperty(out, key, {
|
||||
value: materialize((value as Record<string, unknown>)[key], `${path}.${key}`, seen),
|
||||
enumerable: true,
|
||||
writable: true,
|
||||
configurable: true,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
487
packages/workflow/workflow-worker-thread/src/runtime.ts
Normal file
487
packages/workflow/workflow-worker-thread/src/runtime.ts
Normal file
@@ -0,0 +1,487 @@
|
||||
/**
|
||||
* Per-run worker-side vm hooks, child RPC, concurrency/caps, cancellation, and result serialization; it
|
||||
* never touches Cordis. Script values leaving the realm are materialized as plain JSON before
|
||||
* messaging. Values entering the trusted model-written realm are passed directly; `args` alone is
|
||||
* cloned so script mutation cannot alter initialization data. See `./realm.ts` for the trust model.
|
||||
*
|
||||
* Fatal workflow errors—bad hook arguments, unsupported schemas/options, caps, start failures, and
|
||||
* cancellation—propagate through combinators. Only child failures and ordinary stage errors become
|
||||
* per-item nulls. Every returned promise has a rejection consumer so dropped script promises cannot
|
||||
* kill the worker. A cancelled script that never settles emits nothing; the host force-settles the
|
||||
* run within grace and terminates the thread.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/runtime
|
||||
*/
|
||||
|
||||
import * as vm from 'node:vm'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import { assertObjectJsonSchema, JsonSchemaError } from '@deepseek-ai/dsh-tools'
|
||||
import type { ObjectJsonSchema } from '@deepseek-ai/dsh-tools'
|
||||
import { isFatalWorkflowError, WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import type {
|
||||
WorkflowAgentEndInfo,
|
||||
WorkflowAgentInfo,
|
||||
WorkflowMeta,
|
||||
WorkflowResult,
|
||||
} from '@deepseek-ai/dsh-workflow'
|
||||
import { materializeFromRealm, MaterializeError, renderThrown } from './realm.ts'
|
||||
import type { ChildHandle, ChildPort, WorkerLimits } from './types.ts'
|
||||
|
||||
/** The observers the execution reports progress through (the session posts them to the host). */
|
||||
export interface ExecutionObserver {
|
||||
phase(title: string): void
|
||||
log(message: string): void
|
||||
agentStart(info: WorkflowAgentInfo): void
|
||||
agentEnd(info: WorkflowAgentEndInfo): void
|
||||
}
|
||||
|
||||
/** The `agent()` options the script may pass; everything else rejects loud. */
|
||||
const SUPPORTED_AGENT_OPTIONS = new Set(['label', 'phase', 'schema', 'provider', 'model'])
|
||||
/** Deferred Claude Code options we name explicitly in the rejection message. */
|
||||
const DEFERRED_AGENT_OPTIONS = new Set(['effort', 'isolation', 'agentType'])
|
||||
|
||||
/** Flatten a child's final output blocks to text (the non-schema `agent()` result). */
|
||||
function outputText(blocks: ContentBlock[]): string {
|
||||
return blocks
|
||||
.filter((block): block is Extract<ContentBlock, { type: 'text' }> => block.type === 'text')
|
||||
.map(block => block.text)
|
||||
.join('')
|
||||
}
|
||||
|
||||
/** A short display label derived from the prompt when the script passes none. */
|
||||
function defaultLabel(prompt: string): string {
|
||||
const newline = prompt.indexOf('\n')
|
||||
const line = newline === -1 ? prompt : prompt.slice(0, newline)
|
||||
return line.length <= 48 ? line : `${line.slice(0, 47)}…`
|
||||
}
|
||||
|
||||
/**
|
||||
* One live script execution inside the worker. Constructed per run by the
|
||||
* session; `drive()` is called exactly once and NEVER rejects — every failure
|
||||
* becomes a {@link WorkflowResult} with a non-`completed` stop reason. The
|
||||
* host owns cancellation and cleanup of any dropped child work.
|
||||
*/
|
||||
export class WorkflowExecution {
|
||||
/** 1-based count of `agent()` calls started (the `agentsStarted` result field). */
|
||||
private started = 0
|
||||
private activeSlots = 0
|
||||
private readonly slotWaiters: { resolve(): void; reject(error: unknown): void }[] = []
|
||||
private cancelReason: string | undefined
|
||||
private cancelError: WorkflowError | undefined
|
||||
private currentPhase: string | undefined
|
||||
private readonly context: vm.Context
|
||||
private readonly compiled: vm.Script
|
||||
|
||||
constructor(
|
||||
meta: WorkflowMeta,
|
||||
body: string,
|
||||
args: unknown,
|
||||
private readonly limits: WorkerLimits,
|
||||
private readonly observer: ExecutionObserver,
|
||||
private readonly children: ChildPort,
|
||||
) {
|
||||
// Compile FIRST: a body syntax error must throw out of the constructor
|
||||
// before any realm state exists. The host pre-parses the identical
|
||||
// wrapper, so under one Node version this throw is unreachable in
|
||||
// production — the session still maps it to an error result defensively.
|
||||
// lineOffset compensates for the wrapper line, so stack traces carry the
|
||||
// script's own line numbers.
|
||||
try {
|
||||
this.compiled = new vm.Script(`(async () => {\n${body}\n})()`, {
|
||||
filename: `workflow:${meta.name}`,
|
||||
lineOffset: -1,
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
throw new WorkflowError(`workflow script does not parse: ${String(error)}`, 'SCRIPT_PARSE', { cause: error })
|
||||
}
|
||||
|
||||
this.context = vm.createContext({}, { name: `workflow:${meta.name}` })
|
||||
|
||||
const globals: Record<string, unknown> = {
|
||||
agent: (prompt: unknown, opts?: unknown) => this.contain(this.agent(prompt, opts)),
|
||||
parallel: (thunks: unknown) => this.contain(this.parallel(thunks)),
|
||||
pipeline: (items: unknown, ...stages: unknown[]) => this.contain(this.pipeline(items, stages)),
|
||||
phase: (title: unknown) => { this.phase(title) },
|
||||
log: (message: unknown) => { this.log(message) },
|
||||
// workerData already performed the real cross-thread structured clone.
|
||||
args,
|
||||
}
|
||||
for (const [key, value] of Object.entries(globals)) {
|
||||
// Data properties on the contextified global; frozen shape not required —
|
||||
// a script overwriting its own hooks only sabotages itself.
|
||||
;(this.context as Record<string, unknown>)[key] = typeof value === 'function' ? Object.freeze(value) : value
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the run has been cancelled. A METHOD, not an inline property
|
||||
* read: `cancel()` mutates `cancelReason` concurrently (the session's
|
||||
* message handler), and an inline read after an `await` gets narrowed by
|
||||
* control flow into an always-false comparison.
|
||||
*/
|
||||
private isCancelled(): boolean {
|
||||
return this.cancelReason !== undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared hook entry guard: after {@link cancel}, EVERY hook throws
|
||||
* `CANCELLED` at its next call — cancellation is the next HOOK boundary,
|
||||
* not just the next `agent()`, so a script that caught one cancelled
|
||||
* rejection cannot keep emitting progress through `phase`/`log` or enter a
|
||||
* combinator.
|
||||
*/
|
||||
private throwIfCancelled(): void {
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
}
|
||||
|
||||
/**
|
||||
* Cancel the run: waiting `agent()` slots reject and every future hook call
|
||||
* throws `CANCELLED` — the script dies at its next await. A script that
|
||||
* never settles anyway (parked on a promise no hook owns) is the HOST's
|
||||
* problem: its grace timer force-settles the run and terminates the
|
||||
* worker. Idempotent; the first reason wins.
|
||||
* @param reason - human-readable cause carried on the CANCELLED error. The
|
||||
* host independently aborts the required signal shared by every child.
|
||||
*/
|
||||
cancel(reason: string): void {
|
||||
if (this.cancelReason !== undefined) return
|
||||
this.cancelReason = reason
|
||||
this.cancelError = new WorkflowError(`workflow run cancelled: ${this.cancelReason}`, 'CANCELLED')
|
||||
for (const waiter of this.slotWaiters.splice(0)) waiter.reject(this.cancelledError())
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the script to settlement. Resolves — never rejects — with the run's
|
||||
* {@link WorkflowResult}: the materialized return value on `completed`, the
|
||||
* failure message on `error`, and `cancelled` when the script died of
|
||||
* cancellation. This method only chooses the result; the session publishes
|
||||
* it and the host owns terminal child cancellation.
|
||||
* @returns the settled outcome — this promise NEVER rejects (the seam's
|
||||
* `result`-never-rejects contract); every failure maps to a variant.
|
||||
*/
|
||||
async drive(): Promise<WorkflowResult> {
|
||||
try {
|
||||
// Cancelled before the body ever ran (an already-aborted start signal,
|
||||
// relayed by the host before its `go`): the script must not execute at
|
||||
// all, let alone report `completed`.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
const scriptPromise = this.compiled.runInContext(this.context, { timeout: this.limits.syncTimeoutMs }) as Promise<unknown>
|
||||
const raw: unknown = await this.contain(Promise.resolve(scriptPromise))
|
||||
// Cancelled while the body ran: a script that settled without touching
|
||||
// another hook (or without any) must still report `cancelled` — the
|
||||
// holder asked for cancellation and `completed` would be a lie.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
const value = raw === undefined ? null : this.materializeResult(raw)
|
||||
return { value, stopReason: 'completed', agentsStarted: this.started }
|
||||
} catch (error: unknown) {
|
||||
// Any failure after cancel() reports `cancelled` with the canonical
|
||||
// reason — the reject path mirrors the resolve path's post-settle check.
|
||||
if (this.isCancelled()) {
|
||||
return { value: null, stopReason: 'cancelled', error: this.cancelledError().message, agentsStarted: this.started }
|
||||
}
|
||||
// renderThrown is total (thrown values of any realm), so this arm
|
||||
// cannot throw — drive() resolving is the `result` never-rejects contract
|
||||
// contract.
|
||||
return { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: this.started }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach a no-op rejection consumer WITHOUT changing what the caller
|
||||
* receives: if the script drops the promise (no await), cancellation cannot
|
||||
* become an unhandled rejection (which would kill the worker thread); if
|
||||
* the script does await it, it still observes the rejection.
|
||||
*/
|
||||
private contain<T>(promise: Promise<T>): Promise<T> {
|
||||
promise.catch(() => { /* consumed: see method contract — a dropped hook promise must not surface an unhandled rejection */ })
|
||||
return promise
|
||||
}
|
||||
|
||||
private cancelledError(): WorkflowError {
|
||||
// cancel() arms cancelError before any caller can observe isCancelled()
|
||||
// === true; the fallback guards the type, not a reachable path.
|
||||
/* v8 ignore next */
|
||||
return this.cancelError ?? new WorkflowError('workflow run cancelled', 'CANCELLED')
|
||||
}
|
||||
|
||||
/** Materialize the script's return value; violations become RESULT_UNSERIALIZABLE. */
|
||||
private materializeResult(raw: unknown): unknown {
|
||||
try {
|
||||
return materializeFromRealm(raw, 'workflow result')
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: materializeFromRealm only throws MaterializeError */
|
||||
if (!(error instanceof MaterializeError)) throw error
|
||||
throw new WorkflowError(
|
||||
`the workflow's return value is not plain JSON data — ${error.message}. Return only JSON-serializable objects/arrays/scalars.`,
|
||||
'RESULT_UNSERIALIZABLE',
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire one concurrency slot (FIFO). Cancellation rejects QUEUED waiters
|
||||
* (see {@link cancel}); the callers guard their own entry and post-acquire
|
||||
* windows, so no cancelled-precheck is duplicated here.
|
||||
*/
|
||||
private acquireSlot(): Promise<void> {
|
||||
if (this.activeSlots < this.limits.maxConcurrentAgents) {
|
||||
this.activeSlots += 1
|
||||
return Promise.resolve()
|
||||
}
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
this.slotWaiters.push({
|
||||
resolve: () => {
|
||||
this.activeSlots += 1
|
||||
resolve()
|
||||
},
|
||||
reject,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
private releaseSlot(): void {
|
||||
this.activeSlots -= 1
|
||||
const next = this.slotWaiters.shift()
|
||||
if (next) next.resolve()
|
||||
}
|
||||
|
||||
/** The `agent(prompt, opts)` hook. */
|
||||
private async agent(rawPrompt: unknown, rawOpts: unknown): Promise<unknown> {
|
||||
this.throwIfCancelled()
|
||||
if (typeof rawPrompt !== 'string' || rawPrompt.length === 0) {
|
||||
throw new WorkflowError('agent() requires a non-empty prompt string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const opts = this.readAgentOptions(rawOpts)
|
||||
if (this.started >= this.limits.maxTotalAgents) {
|
||||
throw new WorkflowError(
|
||||
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise the applicable maxTotalAgents limit if the scale is intentional`,
|
||||
'AGENT_CAP',
|
||||
)
|
||||
}
|
||||
this.started += 1
|
||||
const seq = this.started
|
||||
const label = opts.label ?? defaultLabel(rawPrompt)
|
||||
const phase = opts.phase ?? this.currentPhase
|
||||
|
||||
await this.acquireSlot()
|
||||
try {
|
||||
// Re-check after the acquire: the await yields at least one microtask
|
||||
// tick even when a slot is free, and a queued waiter resumes a tick
|
||||
// after its release — a cancel() landing in either window must not
|
||||
// reach the host (which would refuse anyway, but the refusal reads as
|
||||
// a start failure rather than the cancellation it is).
|
||||
this.throwIfCancelled()
|
||||
let run: ChildHandle
|
||||
try {
|
||||
run = await this.children.startAgent({
|
||||
prompt: rawPrompt,
|
||||
...opts.schema !== undefined ? { schema: opts.schema } : {},
|
||||
...opts.provider !== undefined ? { provider: opts.provider } : {},
|
||||
...opts.model !== undefined ? { model: opts.model } : {},
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
// The host refuses starts once the run is cancelled — a refusal that
|
||||
// races our own cancel state must read as the cancellation it is,
|
||||
// not as a broken contract.
|
||||
if (this.isCancelled()) throw this.cancelledError()
|
||||
throw new WorkflowError(`agent() could not start a child: ${renderThrown(error)}`, 'AGENT_START', { cause: error })
|
||||
}
|
||||
// The start round-trip yields to the event loop, so a cancel CAN land
|
||||
// between the host starting the child and this continuation running —
|
||||
// wind the fresh child down instead of leaving it live behind a dead
|
||||
// script.
|
||||
if (this.isCancelled()) {
|
||||
await run.dispose()
|
||||
throw this.cancelledError()
|
||||
}
|
||||
const info: WorkflowAgentInfo = { seq, label, ...phase !== undefined ? { phase } : {}, childId: SessionId(run.id) }
|
||||
this.observer.agentStart(info)
|
||||
try {
|
||||
let result
|
||||
try {
|
||||
result = await run.result
|
||||
} catch (error: unknown) {
|
||||
// A rejected child result is an INFRASTRUCTURE fault relayed by the
|
||||
// host — distinct from a child that failed and resolved. Pair the
|
||||
// lifecycle before propagating, and propagate FATAL: an ordinary
|
||||
// throw would dissolve to a per-item null inside the combinators,
|
||||
// and a broken provider must not read as a failed child.
|
||||
if (this.isCancelled()) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'cancelled' })
|
||||
throw this.cancelledError()
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
throw new WorkflowError(`child agent run failed: ${renderThrown(error)}`, 'AGENT_RESULT', { cause: error })
|
||||
}
|
||||
if (result.stopReason === 'completed') {
|
||||
if (opts.schema !== undefined) {
|
||||
// The provider honored outputSchema (capability-gated at start), so
|
||||
// a completed run without a structured value is a child failure.
|
||||
if (result.structured === undefined) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
return null
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'completed' })
|
||||
return result.structured
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'completed' })
|
||||
return outputText(result.output)
|
||||
}
|
||||
// A cancelled RUN kills the script; a child that failed for its own
|
||||
// reasons resolves null (scripts .filter(Boolean) per the CC contract).
|
||||
if (this.isCancelled()) {
|
||||
this.observer.agentEnd({ ...info, outcome: 'cancelled' })
|
||||
throw this.cancelledError()
|
||||
}
|
||||
this.observer.agentEnd({ ...info, outcome: 'failed' })
|
||||
return null
|
||||
} finally {
|
||||
await run.dispose()
|
||||
}
|
||||
} finally {
|
||||
this.releaseSlot()
|
||||
}
|
||||
}
|
||||
|
||||
/** Materialize + validate the `agent()` options bag from the realm. */
|
||||
private readAgentOptions(rawOpts: unknown): {
|
||||
label?: string
|
||||
phase?: string
|
||||
provider?: string
|
||||
model?: string
|
||||
schema?: ObjectJsonSchema
|
||||
} {
|
||||
if (rawOpts === undefined) return {}
|
||||
let opts: unknown
|
||||
try {
|
||||
opts = materializeFromRealm(rawOpts, 'agent() options')
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: materializeFromRealm only throws MaterializeError */
|
||||
if (!(error instanceof MaterializeError)) throw error
|
||||
throw new WorkflowError(`agent() options must be plain JSON data — ${error.message}`, 'INVALID_ARGUMENT', { cause: error })
|
||||
}
|
||||
if (typeof opts !== 'object' || opts === null || Array.isArray(opts)) {
|
||||
throw new WorkflowError('agent() options must be an object', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const record = opts as Record<string, unknown>
|
||||
for (const key of Object.keys(record)) {
|
||||
if (SUPPORTED_AGENT_OPTIONS.has(key)) continue
|
||||
if (DEFERRED_AGENT_OPTIONS.has(key)) {
|
||||
throw new WorkflowError(`agent() option "${key}" is deferred and not supported by this engine (supported: label, phase, schema, provider, model)`, 'UNSUPPORTED_OPTION')
|
||||
}
|
||||
throw new WorkflowError(`agent() option "${key}" is not recognized (supported: label, phase, schema, provider, model)`, 'UNSUPPORTED_OPTION')
|
||||
}
|
||||
for (const key of ['label', 'phase', 'provider', 'model'] as const) {
|
||||
if (record[key] !== undefined && typeof record[key] !== 'string') {
|
||||
throw new WorkflowError(`agent() option "${key}" must be a string`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
}
|
||||
let schema: ObjectJsonSchema | undefined
|
||||
if (record.schema !== undefined) {
|
||||
try {
|
||||
assertObjectJsonSchema(record.schema)
|
||||
schema = record.schema
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- defensive rethrow arm: assertObjectJsonSchema only throws JsonSchemaError */
|
||||
if (!(error instanceof JsonSchemaError)) throw error
|
||||
throw new WorkflowError(`agent() schema is outside the supported subset — ${error.message}`, 'UNSUPPORTED_SCHEMA', { cause: error })
|
||||
}
|
||||
}
|
||||
return {
|
||||
...record.label !== undefined ? { label: record.label as string } : {},
|
||||
...record.phase !== undefined ? { phase: record.phase as string } : {},
|
||||
...record.provider !== undefined ? { provider: record.provider as string } : {},
|
||||
...record.model !== undefined ? { model: record.model as string } : {},
|
||||
...schema !== undefined ? { schema } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/** The `parallel(thunks)` hook: each thunk caught → `null`; fatal errors propagate. */
|
||||
private async parallel(rawThunks: unknown): Promise<unknown[]> {
|
||||
this.throwIfCancelled()
|
||||
if (!Array.isArray(rawThunks)) {
|
||||
throw new WorkflowError('parallel() requires an array of zero-argument functions', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.assertItemCap(rawThunks.length, 'parallel()')
|
||||
const thunks = rawThunks.map((thunk, index) => {
|
||||
if (typeof thunk !== 'function') {
|
||||
throw new WorkflowError(`parallel() item ${index} is not a function`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
return thunk as () => unknown
|
||||
})
|
||||
return Promise.all(thunks.map(async (thunk) => {
|
||||
try {
|
||||
return await thunk()
|
||||
} catch (error: unknown) {
|
||||
// Hook failures are WorkflowErrors built OUTSIDE the script's realm;
|
||||
// fatality is recognized by `instanceof` against this realm's class —
|
||||
// a script-built object can never pass it, so fatality cannot be
|
||||
// forged (nor accidentally dissolved).
|
||||
if (isFatalWorkflowError(error)) throw error
|
||||
return null
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
/** The `pipeline(items, ...stages)` hook: per-item stage chains, NO cross-stage barrier. */
|
||||
private async pipeline(rawItems: unknown, rawStages: unknown[]): Promise<unknown[]> {
|
||||
this.throwIfCancelled()
|
||||
if (!Array.isArray(rawItems)) {
|
||||
throw new WorkflowError('pipeline() requires an items array', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.assertItemCap(rawItems.length, 'pipeline()')
|
||||
if (rawStages.length === 0) {
|
||||
throw new WorkflowError('pipeline() requires at least one stage function', 'INVALID_ARGUMENT')
|
||||
}
|
||||
const stages = rawStages.map((stage, index) => {
|
||||
if (typeof stage !== 'function') {
|
||||
throw new WorkflowError(`pipeline() stage ${index} is not a function`, 'INVALID_ARGUMENT')
|
||||
}
|
||||
return stage as (previous: unknown, item: unknown, index: number) => unknown
|
||||
})
|
||||
return Promise.all(rawItems.map(async (item: unknown, index) => {
|
||||
let value: unknown = item
|
||||
try {
|
||||
for (const stage of stages) {
|
||||
value = await stage(value, item, index)
|
||||
}
|
||||
return value
|
||||
} catch (error: unknown) {
|
||||
// An ordinary stage throw drops the ITEM to null and skips its
|
||||
// remaining stages; a fatal WorkflowError (see parallel()) kills the
|
||||
// whole script.
|
||||
if (isFatalWorkflowError(error)) throw error
|
||||
return null
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
private assertItemCap(length: number, hook: string): void {
|
||||
if (length > this.limits.maxItemsPerCall) {
|
||||
throw new WorkflowError(
|
||||
`${hook} received ${length} items — over the per-call cap (${this.limits.maxItemsPerCall}); split the work or raise maxItemsPerCall in the engine config`,
|
||||
'ITEM_CAP',
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/** The `phase(title)` hook: sets the current label for subsequent `agent()` calls and notifies observers. */
|
||||
private phase(title: unknown): void {
|
||||
this.throwIfCancelled()
|
||||
if (typeof title !== 'string' || title.length === 0) {
|
||||
throw new WorkflowError('phase() requires a non-empty title string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.currentPhase = title
|
||||
this.observer.phase(title)
|
||||
}
|
||||
|
||||
/** The `log(message)` hook: narration to observers. */
|
||||
private log(message: unknown): void {
|
||||
this.throwIfCancelled()
|
||||
if (typeof message !== 'string') {
|
||||
throw new WorkflowError('log() requires a message string', 'INVALID_ARGUMENT')
|
||||
}
|
||||
this.observer.log(message)
|
||||
}
|
||||
}
|
||||
201
packages/workflow/workflow-worker-thread/src/session.ts
Normal file
201
packages/workflow/workflow-worker-thread/src/session.ts
Normal file
@@ -0,0 +1,201 @@
|
||||
/**
|
||||
* The worker-side half of the engine: {@link runWorkerSession} wires one MessagePort to one
|
||||
* {@link WorkflowExecution} — hook progress and child starts go out as messages, run control
|
||||
* and child lifecycle come back in — and posts the run's terminal result exactly once. Keeping it
|
||||
* separate from `worker.ts` lets unit tests drive the session over a MessageChannel, because main
|
||||
* process coverage cannot observe code inside a real Worker.
|
||||
*
|
||||
* The session announces ready and waits for `go`, so cancellation racing startup can prevent even
|
||||
* the script's synchronous prefix. A cancel in place of `go` releases the gate into a cancelled
|
||||
* drive without executing the body.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/session
|
||||
*/
|
||||
|
||||
import type { MessagePort } from 'node:worker_threads'
|
||||
import { assertNever } from '@deepseek-ai/dsh-llm'
|
||||
import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
|
||||
import type { HostToWorkerMessage, WorkerToHostPayloads } from './protocol.ts'
|
||||
import { renderThrown } from './realm.ts'
|
||||
import { WorkflowExecution } from './runtime.ts'
|
||||
import type { ExecutionObserver } from './runtime.ts'
|
||||
import type {
|
||||
ChildHandle,
|
||||
ChildPort,
|
||||
ChildResult,
|
||||
ChildStartRequest,
|
||||
WorkerInit,
|
||||
} from './types.ts'
|
||||
|
||||
/** The book-keeping for one in-flight child RPC (keyed by callId). */
|
||||
interface PendingChild {
|
||||
started: PromiseWithResolvers<string>
|
||||
settled: PromiseWithResolvers<ChildResult>
|
||||
disposed: PromiseWithResolvers<void>
|
||||
}
|
||||
|
||||
/** The typed post half of the port: each tag pairs with ITS payload from the map (a mismatch is a compile error at the call site). */
|
||||
type Post = <T extends WorkerToHostType>(type: T, payload: WorkerToHostPayloads[T]) => void
|
||||
|
||||
/**
|
||||
* The worker-side handle for one started child agent ({@link ChildHandle}):
|
||||
* every member is an RPC to the host keyed by this call's `callId`, resolved
|
||||
* by the session's message handler through the bridge's pending entry.
|
||||
*/
|
||||
class RpcChildHandle implements ChildHandle {
|
||||
readonly result: Promise<ChildResult>
|
||||
|
||||
constructor(
|
||||
private readonly post: Post,
|
||||
private readonly callId: number,
|
||||
private readonly entry: PendingChild,
|
||||
readonly id: string,
|
||||
) {
|
||||
this.result = entry.settled.promise
|
||||
}
|
||||
|
||||
dispose(): Promise<void> {
|
||||
this.post(WorkerToHostType.ChildDispose, { callId: this.callId })
|
||||
return this.entry.disposed.promise
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side child-RPC bridge ({@link ChildPort}): allocates callIds,
|
||||
* posts the start/dispose RPCs, and owns the per-call pending
|
||||
* book-keeping the session's message handler settles via the `onChild*`
|
||||
* entry points.
|
||||
*/
|
||||
class ChildRpcBridge implements ChildPort {
|
||||
private nextCallId = 0
|
||||
private readonly pending = new Map<number, PendingChild>()
|
||||
|
||||
constructor(private readonly post: Post) {}
|
||||
|
||||
async startAgent(request: ChildStartRequest): Promise<ChildHandle> {
|
||||
this.nextCallId += 1
|
||||
const callId = this.nextCallId
|
||||
const entry: PendingChild = {
|
||||
started: Promise.withResolvers<string>(),
|
||||
settled: Promise.withResolvers<ChildResult>(),
|
||||
disposed: Promise.withResolvers<void>(),
|
||||
}
|
||||
// Containment: when asynchronous provider start fails (or
|
||||
// the run is torn down), the settled promise may never gain a consumer —
|
||||
// it must not surface as an unhandled rejection and kill the worker.
|
||||
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start */ })
|
||||
this.pending.set(callId, entry)
|
||||
this.post(WorkerToHostType.ChildStart, { callId, request })
|
||||
const childId = await entry.started.promise
|
||||
return new RpcChildHandle(this.post, callId, entry, childId)
|
||||
}
|
||||
|
||||
/** The host established a published child; releases the `startAgent` await. */
|
||||
onChildStarted(callId: number, childId: string): void {
|
||||
this.pending.get(callId)?.started.resolve(childId)
|
||||
}
|
||||
|
||||
/** Asynchronous provider start failed; reject and retire the pending RPC. */
|
||||
onChildStartError(callId: number, rendered: string): void {
|
||||
const entry = this.pending.get(callId)
|
||||
this.pending.delete(callId)
|
||||
entry?.started.reject(new Error(rendered))
|
||||
}
|
||||
|
||||
/** The child's terminal result arrived. */
|
||||
onChildSettled(callId: number, result: ChildResult): void {
|
||||
this.pending.get(callId)?.settled.resolve(result)
|
||||
}
|
||||
|
||||
/** The child's `result` rejected host-side (an infrastructure fault, relayed as fatal). */
|
||||
onChildFailed(callId: number, rendered: string): void {
|
||||
this.pending.get(callId)?.settled.reject(new Error(rendered))
|
||||
}
|
||||
|
||||
/** The host acked the dispose; the call's book-keeping is complete. */
|
||||
onChildDisposed(callId: number): void {
|
||||
const entry = this.pending.get(callId)
|
||||
this.pending.delete(callId)
|
||||
entry?.disposed.resolve()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow the nullable `parentPort` the bootstrap reads from
|
||||
* `node:worker_threads`.
|
||||
* @param port - `parentPort` as imported (null on the main thread).
|
||||
* @returns the port, non-null.
|
||||
*/
|
||||
export function requireParentPort(port: MessagePort | null): MessagePort {
|
||||
if (port === null) throw new Error('the workflow worker entry must be loaded inside a worker thread (no parentPort)')
|
||||
return port
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one workflow script to settlement against `port`, posting the terminal result message
|
||||
* exactly once; resolves after that post (stray children may still be winding down through the
|
||||
* port — the host owns their teardown and ultimately terminates the thread). It never rejects:
|
||||
* constructor failure becomes an error result. Host pre-parse makes syntax failure here a likely
|
||||
* Node-version skew, but the session still reports it instead of dying silently.
|
||||
* @param port - the channel to the host (the real `parentPort`, or one side
|
||||
* of an in-process `MessageChannel` in tests).
|
||||
* @param init - the run payload the host provided as `workerData`.
|
||||
*/
|
||||
export async function runWorkerSession(port: MessagePort, init: WorkerInit): Promise<void> {
|
||||
const post: Post = (type, payload) => {
|
||||
port.postMessage({ type, ...payload })
|
||||
}
|
||||
const children = new ChildRpcBridge(post)
|
||||
|
||||
const observer: ExecutionObserver = {
|
||||
phase: (title) => { post(WorkerToHostType.Phase, { title }) },
|
||||
log: (message) => { post(WorkerToHostType.Log, { message }) },
|
||||
agentStart: (info) => { post(WorkerToHostType.AgentStart, { info }) },
|
||||
agentEnd: (info) => { post(WorkerToHostType.AgentEnd, { info }) },
|
||||
}
|
||||
|
||||
let execution: WorkflowExecution
|
||||
try {
|
||||
execution = new WorkflowExecution(init.meta, init.body, init.args, init.limits, observer, children)
|
||||
} catch (error: unknown) {
|
||||
post(WorkerToHostType.Result, { result: { value: null, stopReason: 'error', error: renderThrown(error), agentsStarted: 0 } })
|
||||
return
|
||||
}
|
||||
|
||||
const gate = Promise.withResolvers<void>()
|
||||
port.on('message', (message: HostToWorkerMessage) => {
|
||||
switch (message.type) {
|
||||
case HostToWorkerType.Go:
|
||||
gate.resolve()
|
||||
break
|
||||
case HostToWorkerType.Cancel:
|
||||
execution.cancel(message.reason)
|
||||
// A cancel doubles as the gate release: drive() checks the cancelled
|
||||
// state before running the body, so the script never executes.
|
||||
gate.resolve()
|
||||
break
|
||||
case HostToWorkerType.ChildStarted:
|
||||
children.onChildStarted(message.callId, message.childId)
|
||||
break
|
||||
case HostToWorkerType.ChildStartError:
|
||||
children.onChildStartError(message.callId, message.rendered)
|
||||
break
|
||||
case HostToWorkerType.ChildSettled:
|
||||
children.onChildSettled(message.callId, message.result)
|
||||
break
|
||||
case HostToWorkerType.ChildFailed:
|
||||
children.onChildFailed(message.callId, message.rendered)
|
||||
break
|
||||
case HostToWorkerType.ChildDisposed:
|
||||
children.onChildDisposed(message.callId)
|
||||
break
|
||||
/* v8 ignore next 2 -- closed engine-owned union; the arm only makes adding a message type a compile error */
|
||||
default:
|
||||
assertNever(message, 'host-to-worker message')
|
||||
}
|
||||
})
|
||||
|
||||
post(WorkerToHostType.Ready, {})
|
||||
await gate.promise
|
||||
const result = await execution.drive()
|
||||
post(WorkerToHostType.Result, { result })
|
||||
}
|
||||
94
packages/workflow/workflow-worker-thread/src/types.ts
Normal file
94
packages/workflow/workflow-worker-thread/src/types.ts
Normal file
@@ -0,0 +1,94 @@
|
||||
/**
|
||||
* Non-protocol wire vocabulary for the worker-thread engine: the `workerData` init payload and
|
||||
* the child-port interfaces the worker-side runtime consumes. Host/worker messages are defined in
|
||||
* `./protocol.ts`; transported child requests and results are plain JSON for structured clone.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/types
|
||||
*/
|
||||
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { ObjectJsonSchema } from '@deepseek-ai/dsh-tools'
|
||||
import type { WorkflowMeta } from '@deepseek-ai/dsh-workflow'
|
||||
|
||||
/**
|
||||
* The per-run limits the worker-side runtime enforces. The host keeps the
|
||||
* knobs only it can act on (`provider`, `disposeGraceMs`).
|
||||
*/
|
||||
export interface WorkerLimits {
|
||||
/** Concurrent `agent()` ceiling (already auto-resolved; ≥ 1). */
|
||||
maxConcurrentAgents: number
|
||||
/** Total `agent()` calls per run (the runaway-loop backstop). */
|
||||
maxTotalAgents: number
|
||||
/** Items accepted by one `parallel()`/`pipeline()` call. */
|
||||
maxItemsPerCall: number
|
||||
/** vm timeout for the script's initial synchronous slice (inside the worker). */
|
||||
syncTimeoutMs: number
|
||||
}
|
||||
|
||||
/** The `workerData` payload one run is initialized with (host → worker, once, at spawn). */
|
||||
export interface WorkerInit {
|
||||
/** The validated meta block (plain data off the start request, validated host-side). */
|
||||
meta: WorkflowMeta
|
||||
/** The plain-JS script body, exactly as the start request carried it. */
|
||||
body: string
|
||||
/** The run's `args` value; the workerData structured clone is the copy that isolates the caller. */
|
||||
args?: unknown
|
||||
/** The worker-enforced limits. */
|
||||
limits: WorkerLimits
|
||||
}
|
||||
|
||||
/** What the worker asks the host to start for one `agent()` call (options already validated worker-side). */
|
||||
export interface ChildStartRequest {
|
||||
/** The child's prompt text. */
|
||||
prompt: string
|
||||
/** The structured-output schema, if the call passed one (already subset-checked). */
|
||||
schema?: ObjectJsonSchema
|
||||
/** The per-child provider override, if the call passed one. */
|
||||
provider?: string
|
||||
/** The per-child model override, if the call passed one. */
|
||||
model?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The JSON projection of a child's `SubagentResult` crossing the port. The
|
||||
* seam's `stopReason` union is merge-extensible, so it degrades to `string`
|
||||
* on the wire — the runtime only ever branches on `'completed'`.
|
||||
*/
|
||||
export interface ChildResult {
|
||||
/** The child's final assistant output blocks. */
|
||||
output: ContentBlock[]
|
||||
/** The structured value, present iff the request carried a schema AND the provider honored it. */
|
||||
structured?: unknown
|
||||
/** Why the child run ended (`'completed'` is the only value the runtime branches on). */
|
||||
stopReason: string
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side handle for one started child — the RPC mirror of the
|
||||
* subagent seam's run handle, reduced to what the runtime consumes.
|
||||
*/
|
||||
export interface ChildHandle {
|
||||
/** The child agent's id (minted host-side by the subagent seam). */
|
||||
readonly id: string
|
||||
/**
|
||||
* Resolves with the child's terminal {@link ChildResult}; REJECTS only when
|
||||
* the host reports an infrastructure fault (`child-failed`) — a child that
|
||||
* failed for its own reasons resolves with a non-`completed` stop reason.
|
||||
*/
|
||||
readonly result: Promise<ChildResult>
|
||||
/** Ask the host to dispose the child; resolves on the host's ack. */
|
||||
dispose(): Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-side port the runtime starts child agents through — the seam
|
||||
* that lets the execution core stay ignorant of the thread boundary.
|
||||
*/
|
||||
export interface ChildPort {
|
||||
/**
|
||||
* Start one child agent on the host (the `agent()` hook's start half).
|
||||
* @param request - the prompt and validated options.
|
||||
* @returns the published child handle; rejects when synchronous start or the
|
||||
* provider's asynchronous start fails.
|
||||
*/
|
||||
startAgent(request: ChildStartRequest): Promise<ChildHandle>
|
||||
}
|
||||
14
packages/workflow/workflow-worker-thread/src/worker.ts
Normal file
14
packages/workflow/workflow-worker-thread/src/worker.ts
Normal file
@@ -0,0 +1,14 @@
|
||||
/**
|
||||
* Single-statement worker entry that boots `runWorkerSession` on real `parentPort`. Logic remains in
|
||||
* the session module for in-process MessageChannel coverage; importing this entry on the main thread
|
||||
* exercises `requireParentPort`'s failure path.
|
||||
* @module @deepseek-ai/dsh-workflow-worker-thread/worker
|
||||
*/
|
||||
|
||||
import { parentPort, workerData } from 'node:worker_threads'
|
||||
import { requireParentPort, runWorkerSession } from './session.ts'
|
||||
import type { WorkerInit } from './types.ts'
|
||||
|
||||
// workerData is `any` at the node:worker_threads boundary; the engine is the
|
||||
// only spawner and always provides a WorkerInit.
|
||||
void runWorkerSession(requireParentPort(parentPort), workerData as WorkerInit)
|
||||
@@ -0,0 +1,65 @@
|
||||
import { existsSync } from 'node:fs'
|
||||
import { rm, writeFile } from 'node:fs/promises'
|
||||
import { join } from 'node:path'
|
||||
import { execFile } from 'node:child_process'
|
||||
import { promisify } from 'node:util'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
const packageRoot = fileURLToPath(new URL('..', import.meta.url))
|
||||
const builtIndex = join(packageRoot, 'lib', 'index.js')
|
||||
const builtWorker = join(packageRoot, 'lib', 'worker.cjs')
|
||||
const run = promisify(execFile)
|
||||
|
||||
/**
|
||||
* Keyless built-artifact guard: plain Node loads `lib/index.js` and its sibling
|
||||
* `lib/worker.cjs` without tsx. Skips until the build produces both bundles.
|
||||
*/
|
||||
describe.skipIf(!existsSync(builtIndex) || !existsSync(builtWorker))('built worker entry (lib/worker.cjs)', () => {
|
||||
it('the built engine spawns its built worker under plain node and completes a run', async () => {
|
||||
// Keep the driver in-package so bare imports resolve its node_modules.
|
||||
const driver = join(packageRoot, `.built-worker-driver-${process.pid}.mjs`)
|
||||
try {
|
||||
await writeFile(driver, `
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import WorkerThreadWorkflowEngine from '@deepseek-ai/dsh-workflow-worker-thread'
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentRuntime)
|
||||
let selectedStarts = 0
|
||||
ctx.subagents.registerProvider({
|
||||
name: 'built-selected',
|
||||
capabilities: { outputSchema: true, depthLimit: false, toolFilter: false, persona: false },
|
||||
inheritsParentContext: false,
|
||||
async start() {
|
||||
selectedStarts += 1
|
||||
return {
|
||||
id: 'built-child',
|
||||
result: Promise.resolve({ output: [], structured: { answer: 42 }, stopReason: 'completed' }),
|
||||
dispose: () => Promise.resolve(),
|
||||
}
|
||||
},
|
||||
})
|
||||
await ctx.plugin(WorkerThreadWorkflowEngine, { provider: 'must-not-be-used' })
|
||||
const run = ctx.workflowEngine.start({
|
||||
script: "const value = await agent('answer', { schema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] } }); return value.answer",
|
||||
meta: { name: 'built-smoke', description: 'built worker smoke' },
|
||||
subagentProvider: 'built-selected',
|
||||
parent: { id: 'built-smoke-parent', options: {} },
|
||||
})
|
||||
const result = await run.result
|
||||
await run.dispose()
|
||||
if (result.stopReason !== 'completed' || result.value !== 42 || selectedStarts !== 1) {
|
||||
console.error('unexpected result: ' + JSON.stringify(result))
|
||||
process.exit(1)
|
||||
}
|
||||
console.log('built-worker-smoke-ok')
|
||||
`, 'utf8')
|
||||
const { stdout } = await run(process.execPath, [driver], { cwd: packageRoot, timeout: 60_000 })
|
||||
expect(stdout).toContain('built-worker-smoke-ok')
|
||||
} finally {
|
||||
await rm(driver, { force: true })
|
||||
}
|
||||
}, 120_000)
|
||||
})
|
||||
@@ -0,0 +1,99 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit'
|
||||
import InvariantRegistry from '@deepseek-ai/dsh-invariants'
|
||||
import * as SessionInvariant from '@deepseek-ai/dsh-session/invariant'
|
||||
import * as AgentInvariant from '@deepseek-ai/dsh-agent/invariant'
|
||||
import * as AgentLoopInvariant from '@deepseek-ai/dsh-agent-loop/invariant'
|
||||
import SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import * as spawn from '@deepseek-ai/dsh-subagent-spawn-in-process'
|
||||
import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-in-process-driver'
|
||||
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
import WorkerThreadWorkflowEngine from '../src/index.ts'
|
||||
|
||||
type Script = ConstructorParameters<typeof MockAdapter>[0]
|
||||
|
||||
async function mountInvariants(ctx: Context): Promise<void> {
|
||||
await ctx.plugin(InvariantRegistry)
|
||||
await ctx.plugin(SessionInvariant)
|
||||
await ctx.plugin(AgentInvariant)
|
||||
await ctx.plugin(AgentLoopInvariant)
|
||||
}
|
||||
|
||||
/**
|
||||
* The whole in-process stack, keyless, with the script in a REAL worker
|
||||
* thread: the engine drives the REAL spawn backend (with its
|
||||
* structured runtime) on a real agent loop; the scripted mock MODEL is the
|
||||
* only mocked boundary. This is the guard the unit suites structurally
|
||||
* cannot give — the MessageChannel suite fakes the host, and the host suite
|
||||
* stubs the subagent seam.
|
||||
*/
|
||||
async function setup(script: Script) {
|
||||
const ctx = new Context()
|
||||
const adapter = new MockAdapter(script)
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await mountInvariants(ctx)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(SubagentRuntime)
|
||||
await ctx.plugin(spawn, { providerName: 'spawn' })
|
||||
await ctx.plugin(WorkerThreadWorkflowEngine, {})
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const parent = ctx.agentLoop.create(SessionId('parent'), { provider: 'mock', model: 'mock' })
|
||||
return { ctx, parent, adapter }
|
||||
}
|
||||
|
||||
describe('dsh-workflow-worker-thread over the real in-process stack', () => {
|
||||
it('runs a two-stage workflow: a plain child, then a schema child through the structured runtime', async () => {
|
||||
const { ctx, parent } = await setup([
|
||||
textResponse('the file list is a.ts'),
|
||||
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { verdict: 'real', confidence: 0.9 }),
|
||||
])
|
||||
const childIds: string[] = []
|
||||
ctx.on('workflow/agent-start', (_info, agent) => {
|
||||
// The workflow bridge must await asynchronous provider start: an observer
|
||||
// sees the real spawn child already published, never a reserved id.
|
||||
expect(ctx.agents.get(agent.childId)).toBeDefined()
|
||||
childIds.push(agent.childId)
|
||||
})
|
||||
const run = ctx.workflowEngine.start({
|
||||
meta: { name: 'integration', description: 'plain + structured children' },
|
||||
script: `phase('Read')
|
||||
const prose = await agent('read the repo')
|
||||
phase('Judge')
|
||||
const judged = await agent('judge: ' + prose, {
|
||||
schema: { type: 'object', properties: { verdict: { type: 'string', enum: ['real', 'bogus'] }, confidence: { type: 'number' } }, required: ['verdict'] },
|
||||
})
|
||||
return { prose, verdict: judged.verdict, confidence: judged.confidence }`,
|
||||
parent,
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ prose: 'the file list is a.ts', verdict: 'real', confidence: 0.9 })
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
await run.dispose()
|
||||
// Both children were disposed to quiescence — no live child agents remain.
|
||||
expect(childIds.length).toBe(2)
|
||||
for (const childId of childIds) {
|
||||
expect(ctx.agents.get(SessionId(childId))).toBeUndefined()
|
||||
}
|
||||
})
|
||||
|
||||
it('a child that fails against its schema (nudges exhausted) reaches the script as null', async () => {
|
||||
const { ctx, parent } = await setup([
|
||||
textResponse('prose only'),
|
||||
textResponse('still prose after the nudge'),
|
||||
])
|
||||
const run = ctx.workflowEngine.start({
|
||||
meta: { name: 'null-path', description: 'schema failure maps to null' },
|
||||
script: `const judged = await agent('judge it', { schema: { type: 'object', properties: { v: { type: 'string' } } } })
|
||||
return { got: judged === null ? 'null' : 'value' }`,
|
||||
parent,
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({ got: 'null' })
|
||||
await run.dispose()
|
||||
})
|
||||
})
|
||||
89
packages/workflow/workflow-worker-thread/tests/meta.spec.ts
Normal file
89
packages/workflow/workflow-worker-thread/tests/meta.spec.ts
Normal file
@@ -0,0 +1,89 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { WorkflowError } from '@deepseek-ai/dsh-workflow'
|
||||
import { validateMeta } from '../src/meta.ts'
|
||||
|
||||
/** Assert a META_INVALID throw whose message matches every given fragment. */
|
||||
function expectInvalid(value: unknown, ...fragments: string[]): void {
|
||||
let thrown: unknown
|
||||
try {
|
||||
validateMeta(value)
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
expect(thrown).toBeInstanceOf(WorkflowError)
|
||||
expect((thrown as WorkflowError).code).toBe('META_INVALID')
|
||||
for (const fragment of fragments) {
|
||||
expect((thrown as WorkflowError).message).toContain(fragment)
|
||||
}
|
||||
}
|
||||
|
||||
describe('validateMeta', () => {
|
||||
it('accepts a minimal meta and returns a normalized copy (no aliasing of the input)', () => {
|
||||
const input = { name: 'audit', description: 'audit the repo' }
|
||||
const meta = validateMeta(input)
|
||||
expect(meta).toEqual({ name: 'audit', description: 'audit the repo' })
|
||||
expect(meta).not.toBe(input)
|
||||
input.name = 'mutated'
|
||||
expect(meta.name).toBe('audit')
|
||||
})
|
||||
|
||||
it('accepts the full shape and rebuilds phases entry by entry', () => {
|
||||
const meta = validateMeta({
|
||||
name: 'migrate',
|
||||
description: 'migrate call sites',
|
||||
whenToUse: 'large mechanical sweeps',
|
||||
phases: [
|
||||
{ title: 'Discover', provider: 'openai' },
|
||||
{ title: 'Transform', detail: 'one agent per file', model: 'deepseek-v4-pro' },
|
||||
],
|
||||
})
|
||||
expect(meta).toEqual({
|
||||
name: 'migrate',
|
||||
description: 'migrate call sites',
|
||||
whenToUse: 'large mechanical sweeps',
|
||||
phases: [
|
||||
{ title: 'Discover', provider: 'openai' },
|
||||
{ title: 'Transform', detail: 'one agent per file', model: 'deepseek-v4-pro' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects non-object values loud', () => {
|
||||
expectInvalid(undefined, 'meta must be an object')
|
||||
expectInvalid('a string', 'meta must be an object')
|
||||
expectInvalid(null, 'meta must be an object')
|
||||
expectInvalid([{ name: 'x', description: 'd' }], 'meta must be an object')
|
||||
})
|
||||
|
||||
it('rejects unknown fields by name (accepted-then-ignored is banned)', () => {
|
||||
expectInvalid({ name: 'x', description: 'd', color: 'red' }, 'meta.color is not a recognized field')
|
||||
})
|
||||
|
||||
it('rejects missing or mistyped name/description/whenToUse', () => {
|
||||
expectInvalid({ description: 'd' }, 'meta.name must be a non-empty string')
|
||||
expectInvalid({ name: '', description: 'd' }, 'meta.name must be a non-empty string')
|
||||
expectInvalid({ name: 'x' }, 'meta.description must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 42 }, 'meta.description must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 'd', whenToUse: 3 }, 'meta.whenToUse must be a string')
|
||||
})
|
||||
|
||||
it('rejects malformed phases, entry by entry', () => {
|
||||
expectInvalid({ name: 'x', description: 'd', phases: 'Scan' }, 'meta.phases must be an array')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: ['Scan'] }, 'meta.phases[0] must be an object')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: '' }] }, 'meta.phases[0].title must be a non-empty string')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', order: 1 }] }, 'meta.phases[0].order is not a recognized field')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', detail: 9 }] }, 'meta.phases[0].detail must be a string')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', provider: 9 }] }, 'meta.phases[0].provider must be a string')
|
||||
expectInvalid({ name: 'x', description: 'd', phases: [{ title: 'Scan', model: 9 }] }, 'meta.phases[0].model must be a string')
|
||||
})
|
||||
|
||||
it('names EVERY violation in one throw, not just the first', () => {
|
||||
expectInvalid(
|
||||
{ description: 7, extra: true, phases: [{ title: 'Scan' }, 'bad'] },
|
||||
'meta.extra is not a recognized field',
|
||||
'meta.name must be a non-empty string',
|
||||
'meta.description must be a non-empty string',
|
||||
'meta.phases[1] must be an object',
|
||||
)
|
||||
})
|
||||
})
|
||||
145
packages/workflow/workflow-worker-thread/tests/realm.spec.ts
Normal file
145
packages/workflow/workflow-worker-thread/tests/realm.spec.ts
Normal file
@@ -0,0 +1,145 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import * as vm from 'node:vm'
|
||||
import { materializeFromRealm, MaterializeError, renderThrown } from '../src/realm.ts'
|
||||
|
||||
/** Evaluate an expression inside a fresh vm realm and hand back the raw realm value. */
|
||||
function inRealm(expression: string): unknown {
|
||||
return vm.runInNewContext(`(${expression})`)
|
||||
}
|
||||
|
||||
/** The MaterializeError message for a value that must be rejected (throws if accepted). */
|
||||
function rejection(value: unknown): string {
|
||||
try {
|
||||
materializeFromRealm(value)
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof MaterializeError) return error.message
|
||||
throw error
|
||||
}
|
||||
throw new Error('expected the value to be rejected')
|
||||
}
|
||||
|
||||
describe('materializeFromRealm', () => {
|
||||
it('copies realm objects/arrays/scalars into host plain data', () => {
|
||||
const value = inRealm("{ a: 1, b: 'x', c: true, d: null, list: [1, [2, { deep: 'y' }]] }")
|
||||
const out = materializeFromRealm(value) as Record<string, unknown>
|
||||
expect(out).toEqual({ a: 1, b: 'x', c: true, d: null, list: [1, [2, { deep: 'y' }]] })
|
||||
// The copy is HOST data: prototypes are the host intrinsics.
|
||||
expect(Object.getPrototypeOf(out)).toBe(Object.prototype)
|
||||
expect(Array.isArray(out.list)).toBe(true)
|
||||
// And it round-trips through JSON byte-identically (the whole point).
|
||||
expect(JSON.parse(JSON.stringify(out))).toEqual(out)
|
||||
})
|
||||
|
||||
it('accepts undefined ONLY at the root (a valueless script return)', () => {
|
||||
expect(materializeFromRealm(undefined)).toBeUndefined()
|
||||
expect(rejection(inRealm('{ a: undefined }'))).toContain('value.a')
|
||||
})
|
||||
|
||||
it('invokes getters ordinarily — the getter RESULT is what crosses (trust premise)', () => {
|
||||
const counter = inRealm(`
|
||||
(() => {
|
||||
globalThis.reads = 0
|
||||
return { get x() { globalThis.reads += 1; return globalThis.reads } }
|
||||
})()
|
||||
`)
|
||||
expect(materializeFromRealm(counter)).toEqual({ x: 1 })
|
||||
})
|
||||
|
||||
it('a getter that THROWS surfaces as a MaterializeError carrying the rendered failure', () => {
|
||||
const hostile = inRealm("{ get x() { throw new Error('read failed') } }")
|
||||
const message = rejection(hostile)
|
||||
expect(message).toContain('reading the value threw')
|
||||
expect(message).toContain('read failed')
|
||||
})
|
||||
|
||||
it('a "__proto__" key becomes an OWN data property of the copy, never a prototype mutation', () => {
|
||||
const value: unknown = vm.runInNewContext('JSON.parse(\'{"__proto__": {"polluted": 1}, "ok": 2}\')')
|
||||
const out = materializeFromRealm(value) as Record<string, unknown>
|
||||
expect(Object.getPrototypeOf(out)).toBe(Object.prototype)
|
||||
expect(Object.prototype.hasOwnProperty.call(out, '__proto__')).toBe(true)
|
||||
expect(out.ok).toBe(2)
|
||||
// The host Object.prototype was NOT touched.
|
||||
expect(({} as Record<string, unknown>).polluted).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects functions, symbols (keys and values), and bigints with path-qualified messages', () => {
|
||||
expect(rejection(inRealm('{ fn: () => 1 }'))).toContain('value.fn')
|
||||
expect(rejection(inRealm("{ [Symbol('k')]: 1 }"))).toContain('symbol-keyed')
|
||||
expect(rejection(inRealm("{ s: Symbol('v') }"))).toContain('value.s')
|
||||
expect(rejection(inRealm('{ big: 1n }'))).toContain('value.big')
|
||||
expect(rejection(inRealm("[Symbol('x')]"))).toContain('value[0]')
|
||||
const taggedArray = inRealm("(() => { const a = [1]; a[Symbol('t')] = 1; return a })()")
|
||||
expect(rejection(taggedArray)).toContain('symbol-keyed')
|
||||
})
|
||||
|
||||
it('rejects non-finite numbers and undefined values inside containers', () => {
|
||||
expect(rejection(inRealm('{ n: NaN }'))).toContain('non-finite')
|
||||
expect(rejection(inRealm('[Infinity]'))).toContain('non-finite')
|
||||
})
|
||||
|
||||
it('rejects exotic prototypes (Date, Map, class instances) but accepts null-prototype data', () => {
|
||||
expect(rejection(inRealm('{ d: new Date(0) }'))).toContain('exotic prototype')
|
||||
expect(rejection(inRealm('new Map()'))).toContain('exotic prototype')
|
||||
expect(rejection(inRealm('(() => { class C { constructor() { this.x = 1 } } return new C() })()')))
|
||||
.toContain('exotic prototype')
|
||||
expect(materializeFromRealm(inRealm('Object.assign(Object.create(null), { a: 1 })'))).toEqual({ a: 1 })
|
||||
})
|
||||
|
||||
it('rejects cycles and accepts the same object reused as a sibling (a DAG)', () => {
|
||||
expect(rejection(inRealm('(() => { const o = {}; o.self = o; return o })()'))).toContain('circular')
|
||||
const dag = inRealm('(() => { const leaf = { v: 1 }; return { a: leaf, b: leaf } })()')
|
||||
expect(materializeFromRealm(dag)).toEqual({ a: { v: 1 }, b: { v: 1 } })
|
||||
})
|
||||
|
||||
it('rejects sparse arrays and non-index array properties; an array getter element materializes its value', () => {
|
||||
expect(rejection(inRealm('[1, , 3]'))).toContain('sparse')
|
||||
expect(rejection(inRealm('(() => { const a = [1]; a.total = 3; return a })()')))
|
||||
.toContain('non-index')
|
||||
expect(materializeFromRealm(inRealm('(() => { const a = [1]; Object.defineProperty(a, 0, { get: () => 7, enumerable: true }); return a })()')))
|
||||
.toEqual([7])
|
||||
})
|
||||
|
||||
it('skips non-enumerable own properties (matching JSON.stringify exactly)', () => {
|
||||
const value = inRealm(`(() => {
|
||||
const o = { visible: 1 }
|
||||
Object.defineProperty(o, 'hidden', { value: () => 1, enumerable: false })
|
||||
return o
|
||||
})()`)
|
||||
expect(materializeFromRealm(value)).toEqual({ visible: 1 })
|
||||
})
|
||||
|
||||
it('works on plain host values too (the boundary is realm-agnostic)', () => {
|
||||
expect(materializeFromRealm({ a: [1, 'x'] })).toEqual({ a: [1, 'x'] })
|
||||
expect(materializeFromRealm('str')).toBe('str')
|
||||
expect(materializeFromRealm(3)).toBe(3)
|
||||
expect(materializeFromRealm(false)).toBe(false)
|
||||
expect(materializeFromRealm(null)).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('renderThrown', () => {
|
||||
it('prefers the stack, for host and realm errors alike', () => {
|
||||
const host = renderThrown(new Error('host failure'))
|
||||
expect(host).toContain('host failure')
|
||||
expect(host).toContain('at ') // a real stack, not just the message
|
||||
const realmError: unknown = vm.runInNewContext('(() => { try { throw new Error("realm failure") } catch (e) { return e } })()')
|
||||
expect(renderThrown(realmError)).toContain('realm failure')
|
||||
})
|
||||
|
||||
it('falls back from stack to message to String()', () => {
|
||||
expect(renderThrown({ stack: 'custom data stack' })).toBe('custom data stack')
|
||||
const stackless = new Error('stackless failure')
|
||||
delete stackless.stack
|
||||
expect(renderThrown(stackless)).toBe('stackless failure')
|
||||
expect(renderThrown({ code: 42 })).toBe('[object Object]')
|
||||
expect(renderThrown('plain')).toBe('plain')
|
||||
expect(renderThrown(42)).toBe('42')
|
||||
expect(renderThrown(undefined)).toBe('undefined')
|
||||
expect(renderThrown(null)).toBe('null')
|
||||
})
|
||||
|
||||
it('is total: a value whose accessors/toString throw renders as a fixed label', () => {
|
||||
expect(renderThrown({ get stack() { throw new Error('nope') } })).toBe('[unrenderable thrown value]')
|
||||
expect(renderThrown({ [Symbol.toPrimitive]() { throw new Error('nope') } })).toBe('[unrenderable thrown value]')
|
||||
})
|
||||
})
|
||||
511
packages/workflow/workflow-worker-thread/tests/session.spec.ts
Normal file
511
packages/workflow/workflow-worker-thread/tests/session.spec.ts
Normal file
@@ -0,0 +1,511 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { MessageChannel } from 'node:worker_threads'
|
||||
import type { MessagePort } from 'node:worker_threads'
|
||||
import { HostToWorkerType, WorkerToHostType } from '../src/protocol.ts'
|
||||
import type { HostToWorkerMessage, WorkerToHostMessage } from '../src/protocol.ts'
|
||||
import { requireParentPort, runWorkerSession } from '../src/session.ts'
|
||||
import type { ChildResult, WorkerInit } from '../src/types.ts'
|
||||
|
||||
/** Default limits for in-process sessions (concurrency pinned; auto is machine-derived). */
|
||||
function limits(overrides?: Partial<WorkerInit['limits']>): WorkerInit['limits'] {
|
||||
return { maxConcurrentAgents: 8, maxTotalAgents: 1000, maxItemsPerCall: 4096, syncTimeoutMs: 5000, ...overrides }
|
||||
}
|
||||
|
||||
/** Wrap a body in the minimal valid meta header (the session receives it pre-extracted). */
|
||||
function init(body: string, args?: unknown, limitOverrides?: Partial<WorkerInit['limits']>): WorkerInit {
|
||||
return {
|
||||
meta: { name: 'test-flow', description: 'a test workflow' },
|
||||
body,
|
||||
...args !== undefined ? { args } : {},
|
||||
limits: limits(limitOverrides),
|
||||
}
|
||||
}
|
||||
|
||||
/** One scripted host over the other end of a MessageChannel. */
|
||||
interface FakeHost {
|
||||
port: MessagePort
|
||||
messages: WorkerToHostMessage[]
|
||||
/** Messages of one type, as they arrive. */
|
||||
ofType<T extends WorkerToHostMessage['type']>(type: T): Extract<WorkerToHostMessage, { type: T }>[]
|
||||
send(message: HostToWorkerMessage): void
|
||||
/** Resolves with the terminal result message. */
|
||||
result(): Promise<Extract<WorkerToHostMessage, { type: 'result' }>['result']>
|
||||
close(): void
|
||||
}
|
||||
|
||||
interface FakeHostOptions {
|
||||
/** Auto-respond to child-start: reply started + settled per child index. Omit a reply to leave the child pending. */
|
||||
reply?: (request: { prompt: string; schema?: unknown; provider?: string; model?: string }, index: number) => ChildResult | undefined
|
||||
/** Reject the start instead (child-start-error) when returning a string. */
|
||||
refuse?: (index: number) => string | undefined
|
||||
/** Auto-send `go` on `ready` (default true). */
|
||||
go?: boolean
|
||||
/** Manual mode: do NOT auto-answer child-start at all (the test scripts the replies). */
|
||||
manual?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Drive runWorkerSession IN-PROCESS over a MessageChannel: this is where the
|
||||
* worker-side files earn their coverage — code inside a real Worker is
|
||||
* invisible to main-process coverage. The fake host mirrors the real host's
|
||||
* protocol discipline (one started/start-error per start; settled/disposed
|
||||
* follow).
|
||||
*/
|
||||
function fakeHost(options?: FakeHostOptions): FakeHost {
|
||||
const channel = new MessageChannel()
|
||||
const messages: WorkerToHostMessage[] = []
|
||||
const resultGate = Promise.withResolvers<Extract<WorkerToHostMessage, { type: 'result' }>['result']>()
|
||||
let childIndex = 0
|
||||
channel.port1.on('message', (message: WorkerToHostMessage) => {
|
||||
messages.push(message)
|
||||
switch (message.type) {
|
||||
case WorkerToHostType.Ready:
|
||||
if (options?.go !== false) channel.port1.postMessage({ type: HostToWorkerType.Go } satisfies HostToWorkerMessage)
|
||||
break
|
||||
case WorkerToHostType.ChildStart: {
|
||||
if (options?.manual) break
|
||||
const index = childIndex
|
||||
childIndex += 1
|
||||
const refusal = options?.refuse?.(index)
|
||||
if (refusal !== undefined) {
|
||||
channel.port1.postMessage(
|
||||
{ type: HostToWorkerType.ChildStartError, callId: message.callId, rendered: refusal } satisfies HostToWorkerMessage,
|
||||
)
|
||||
break
|
||||
}
|
||||
channel.port1.postMessage({ type: HostToWorkerType.ChildStarted, callId: message.callId, childId: `child-${index}` } satisfies HostToWorkerMessage)
|
||||
const reply = options?.reply?.(message.request, index)
|
||||
if (reply !== undefined) {
|
||||
channel.port1.postMessage(
|
||||
{ type: HostToWorkerType.ChildSettled, callId: message.callId, result: reply } satisfies HostToWorkerMessage,
|
||||
)
|
||||
}
|
||||
break
|
||||
}
|
||||
case WorkerToHostType.ChildDispose:
|
||||
channel.port1.postMessage({ type: HostToWorkerType.ChildDisposed, callId: message.callId } satisfies HostToWorkerMessage)
|
||||
break
|
||||
case WorkerToHostType.Result:
|
||||
resultGate.resolve(message.result)
|
||||
break
|
||||
default:
|
||||
break
|
||||
}
|
||||
})
|
||||
return {
|
||||
port: channel.port2,
|
||||
messages,
|
||||
ofType: type => messages.filter((message): message is never => message.type === type),
|
||||
send: (message) => { channel.port1.postMessage(message) },
|
||||
result: () => resultGate.promise,
|
||||
close: () => { channel.port1.close() },
|
||||
}
|
||||
}
|
||||
|
||||
/** A completed text child result. */
|
||||
function text(reply: string): ChildResult {
|
||||
return { output: [{ type: 'text', text: reply }], stopReason: 'completed' }
|
||||
}
|
||||
|
||||
describe('runWorkerSession over an in-process MessageChannel', () => {
|
||||
it('runs a script end to end: ready/go handshake, phases, log, agents, result', async () => {
|
||||
const host = fakeHost({ reply: (_request, index) => text(`answer-${index}`) })
|
||||
const session = runWorkerSession(host.port, init(`
|
||||
phase('Scan')
|
||||
log('starting with ' + args.files.length + ' files')
|
||||
const answers = await pipeline(args.files, (prev, item) => agent('read ' + item))
|
||||
return { answers }
|
||||
`, { files: ['a.ts', 'b.ts'] }))
|
||||
const result = await host.result()
|
||||
await session
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
expect(result.value).toEqual({ answers: ['answer-0', 'answer-1'] })
|
||||
expect(host.messages[0]!.type).toBe('ready')
|
||||
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['Scan'])
|
||||
expect(host.ofType(WorkerToHostType.Log).map(m => m.message)).toEqual(['starting with 2 files'])
|
||||
expect(host.ofType(WorkerToHostType.AgentStart).map(m => m.info.childId)).toEqual(['child-0', 'child-1'])
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd).every(m => m.info.outcome === 'completed')).toBe(true)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('agent({schema}) forwards the schema on the start request and returns the structured value', async () => {
|
||||
const host = fakeHost({ reply: () => ({ output: [], structured: { files: ['x.ts'] }, stopReason: 'completed' }) })
|
||||
void runWorkerSession(host.port, init(`
|
||||
const found = await agent('list files', { schema: { type: 'object', properties: { files: { type: 'array', items: { type: 'string' } } } }, model: 'deepseek-v4-pro' })
|
||||
return { first: found.files[0] }
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual({ first: 'x.ts' })
|
||||
const start = host.ofType(WorkerToHostType.ChildStart)[0]!
|
||||
expect(start.request.schema).toEqual({ type: 'object', properties: { files: { type: 'array', items: { type: 'string' } } } })
|
||||
expect(start.request.model).toBe('deepseek-v4-pro')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('agent({provider}) forwards a provider without inventing a model', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init("return await agent('route me', { provider: 'openai' })"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toBe('ok')
|
||||
const start = host.ofType(WorkerToHostType.ChildStart)[0]!
|
||||
expect(start.request.provider).toBe('openai')
|
||||
expect(start.request.model).toBeUndefined()
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a schema child completing WITHOUT a structured value resolves null with a failed outcome', async () => {
|
||||
const host = fakeHost({ reply: () => text('prose, no structure') })
|
||||
void runWorkerSession(host.port, init("return await agent('p', { schema: { type: 'object' } })"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toBeNull()
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('failed')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child settling non-completed resolves null (scripts filter), never throwing into the script', async () => {
|
||||
const host = fakeHost({ reply: (_request, index) => index === 0 ? { output: [], stopReason: 'error' } : text('ok') })
|
||||
void runWorkerSession(host.port, init("return await parallel([() => agent('one'), () => agent('two')])"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual([null, 'ok'])
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd).map(m => m.info.outcome)).toEqual(expect.arrayContaining(['failed', 'completed']))
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a start refusal (child-start-error) is a fatal AGENT_START that kills the script through a combinator', async () => {
|
||||
const host = fakeHost({ refuse: () => 'no provider here' })
|
||||
void runWorkerSession(host.port, init("return await pipeline([1], () => agent('p'))"))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('agent() could not start a child')
|
||||
expect(result.error).toContain('no provider here')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child-failed message (infrastructure rejection) is fatal AGENT_RESULT with the paired failed outcome', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
try { await agent('p'); return 'unreachable' } catch (e) { return { name: e.name, code: e.code, fatal: e.fatal } }
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId, rendered: 'backend exploded' })
|
||||
const result = await host.result()
|
||||
expect(result.value).toMatchObject({ name: 'WorkflowError', code: 'AGENT_RESULT', fatal: true })
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('failed')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancel before go: the body never runs at all and the result is cancelled (a second cancel is a no-op)', async () => {
|
||||
const host = fakeHost({ go: false })
|
||||
const session = runWorkerSession(host.port, init("log('ran')\nreturn 123"))
|
||||
await vi.waitFor(() => { expect(host.messages.some(m => m.type === WorkerToHostType.Ready)).toBe(true) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'aborted before start' })
|
||||
// Idempotence: the first reason wins; a duplicate cancel changes nothing.
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'a later reason that must lose' })
|
||||
const result = await host.result()
|
||||
await session
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('aborted before start')
|
||||
expect(result.error).not.toContain('must lose')
|
||||
expect(result.value).toBeNull()
|
||||
expect(host.ofType(WorkerToHostType.Log)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a script with no return value resolves value: null', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init("await agent('p')"))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBeNull()
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancel mid-run: hooks throw at entry and the run reports cancelled', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
phase('before')
|
||||
try { await agent('x') } catch (e) {}
|
||||
try { phase('after') } catch (e) {}
|
||||
try { log('after') } catch (e) {}
|
||||
try { await parallel([() => 'ran']) } catch (e) {}
|
||||
try { await pipeline(['item'], p => p) } catch (e) {}
|
||||
return 'survived by catching'
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'stop everything' })
|
||||
// The real host settles the aborted child; mirror it.
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId, result: { output: [], stopReason: 'aborted' } })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(result.error).toContain('stop everything')
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
|
||||
// No post-cancel narration left the runtime (the hooks threw at entry).
|
||||
expect(host.ofType(WorkerToHostType.Phase).map(m => m.title)).toEqual(['before'])
|
||||
expect(host.ofType(WorkerToHostType.Log)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('cancellation between a queued waiter and its slot: the waiter rejects without a child-start', async () => {
|
||||
const host = fakeHost({ go: true })
|
||||
void runWorkerSession(host.port, init(
|
||||
"return await parallel([() => agent('a'), () => agent('b')])",
|
||||
undefined,
|
||||
{ maxConcurrentAgents: 1 },
|
||||
))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'raced' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
// Only the first agent ever reached the host.
|
||||
expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a stray (never-awaited) agent is reaped after settlement: cancel + dispose RPCs flow, no unhandled rejection', async () => {
|
||||
const unhandled: unknown[] = []
|
||||
const onUnhandled = (reason: unknown): void => { unhandled.push(reason) }
|
||||
process.on('unhandledRejection', onUnhandled)
|
||||
try {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init(`
|
||||
agent('stray, never awaited')
|
||||
return 'done without awaiting'
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId, result: { output: [], stopReason: 'aborted' } })
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId) })
|
||||
await new Promise(resolve => setTimeout(resolve, 20))
|
||||
expect(unhandled).toEqual([])
|
||||
host.close()
|
||||
} finally {
|
||||
process.off('unhandledRejection', onUnhandled)
|
||||
}
|
||||
})
|
||||
|
||||
it('an unparseable body settles an error result instead of dying without one (host pre-parse skew guard)', async () => {
|
||||
const host = fakeHost()
|
||||
await runWorkerSession(host.port, init('return ((('))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('does not parse')
|
||||
expect(result.agentsStarted).toBe(0)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a synchronous spin in the initial slice dies by the in-worker vm timeout', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init('while (true) {}', undefined, { syncTimeoutMs: 50 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error?.toLowerCase()).toContain('timed out')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a non-JSON return value fails loud as RESULT_UNSERIALIZABLE', async () => {
|
||||
const host = fakeHost()
|
||||
void runWorkerSession(host.port, init('return { when: new Date(0) }'))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('not plain JSON data')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('tolerates replies for unknown callIds (a teardown race): nothing crashes, the run completes', async () => {
|
||||
const host = fakeHost({ reply: () => text('fine') })
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId: 999, childId: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildStartError, callId: 999, rendered: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildSettled, callId: 999, result: text('ghost') })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId: 999, rendered: 'ghost' })
|
||||
host.send({ type: HostToWorkerType.ChildDisposed, callId: 999 })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toBe('fine')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('caps and malformed hook arguments reject loud (the runtime runs unchanged inside the session)', async () => {
|
||||
const cases: [string, string][] = [
|
||||
['return await agent(42)', 'non-empty prompt string'],
|
||||
["return await agent('')", 'non-empty prompt string'],
|
||||
["return await agent('p', 'opts')", 'options must be an object'],
|
||||
["return await agent('p', { label: 3 })", '"label" must be a string'],
|
||||
["return await agent('p', { get label() { throw new Error('read failed') } })", 'options must be plain JSON data'],
|
||||
["return await agent('p', { bogus: true })", '"bogus" is not recognized'],
|
||||
["return await agent('p', { effort: 'high' })", '"effort" is deferred and not supported by this engine (supported: label, phase, schema, provider, model)'],
|
||||
["return await agent('p', { schema: { type: 'object', oneOf: [] } })", 'outside the supported subset'],
|
||||
['return await parallel([() => 1, () => 2, () => 3])', 'over the per-call cap (2)'],
|
||||
['return await pipeline([1, 2, 3], (x) => x)', 'maxItemsPerCall'],
|
||||
["return await parallel('no')", 'parallel() requires an array'],
|
||||
['return await parallel([3])', 'item 0 is not a function'],
|
||||
["return await pipeline('no', () => 1)", 'pipeline() requires an items array'],
|
||||
['return await pipeline([1])', 'at least one stage'],
|
||||
["return await pipeline([1], 'x')", 'stage 0 is not a function'],
|
||||
["phase('')", 'phase() requires a non-empty title string'],
|
||||
['log(3)', 'log() requires a message string'],
|
||||
]
|
||||
for (const [body, expected] of cases) {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init(body, undefined, { maxItemsPerCall: 2 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain(expected)
|
||||
host.close()
|
||||
}
|
||||
})
|
||||
|
||||
it('combinator semantics: thunk/stage throws null the item; a forged fatal-shaped object stays null; real fatals propagate', async () => {
|
||||
const host = fakeHost({ reply: () => text('fine') })
|
||||
void runWorkerSession(host.port, init(`
|
||||
const viaParallel = await parallel([
|
||||
() => { throw new Error('boom') },
|
||||
() => agent('fine'),
|
||||
() => 'plain value',
|
||||
() => { throw { name: 'WorkflowError', fatal: true, message: 'forged fatal' } },
|
||||
])
|
||||
const viaPipeline = await pipeline([10, 20],
|
||||
(prev, item, index) => { if (item === 10) throw new Error('ordinary failure'); return 'kept-' + item + '-' + index },
|
||||
)
|
||||
return { viaParallel, viaPipeline }
|
||||
`))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.value).toEqual({
|
||||
viaParallel: [null, 'fine', 'plain value', null],
|
||||
viaPipeline: [null, 'kept-20-1'],
|
||||
})
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('trips the total-agent cap with a message naming the config knob', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init("await agent('1'); await agent('2'); await agent('3')", undefined, { maxTotalAgents: 2 }))
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('total agent cap (2)')
|
||||
expect(result.error).toContain('applicable maxTotalAgents limit')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('queued agents proceed through the concurrency semaphore in FIFO order', async () => {
|
||||
const host = fakeHost({ reply: request => text(`ok:${request.prompt}`) })
|
||||
void runWorkerSession(host.port, init(
|
||||
"return await parallel([1, 2, 3].map((n) => () => agent('job ' + n)))",
|
||||
undefined,
|
||||
{ maxConcurrentAgents: 1 },
|
||||
))
|
||||
const result = await host.result()
|
||||
expect(result.value).toEqual(['ok:job 1', 'ok:job 2', 'ok:job 3'])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('labels default from the prompt first line, truncated; explicit label/phase options win', async () => {
|
||||
const host = fakeHost({ reply: () => text('ok') })
|
||||
void runWorkerSession(host.port, init(`
|
||||
phase('Find')
|
||||
await agent('a prompt that is quite long and will surely get truncated down to a display label\\n'
|
||||
+ 'with a second line the label must not include')
|
||||
await agent('short', { label: 'named', phase: 'Custom' })
|
||||
return null
|
||||
`))
|
||||
await host.result()
|
||||
const starts = host.ofType(WorkerToHostType.AgentStart).map(m => m.info)
|
||||
expect(starts[0]).toMatchObject({ seq: 1, phase: 'Find' })
|
||||
expect(starts[0]!.label.length).toBeLessThanOrEqual(48)
|
||||
expect(starts[0]!.label).not.toContain('second line')
|
||||
expect(starts[1]).toMatchObject({ seq: 2, label: 'named', phase: 'Custom' })
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('non-text output blocks are filtered out of the text result', async () => {
|
||||
const host = fakeHost({
|
||||
reply: () => ({
|
||||
output: [
|
||||
{ type: 'text', text: 'first ' },
|
||||
{ type: 'tool_call', id: 'c1', name: 'x', arguments: {} } as never,
|
||||
{ type: 'text', text: 'second' },
|
||||
],
|
||||
stopReason: 'completed',
|
||||
}),
|
||||
})
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
const result = await host.result()
|
||||
expect(result.value).toBe('first second')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a cancel landing DURING the start round-trip disposes the fresh child and dies cancelled', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init("return await agent('p')"))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
// Simulate a teardown race by delivering cancellation before a stale start reply.
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'raced the start' })
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
await vi.waitFor(() => {
|
||||
expect(host.ofType(WorkerToHostType.ChildDispose).map(m => m.callId)).toContain(callId)
|
||||
})
|
||||
// The unpublished child is disposed without a lifecycle announcement.
|
||||
expect(host.ofType(WorkerToHostType.AgentStart)).toEqual([])
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a start refusal arriving after a cancel reads as the cancellation, not a broken seam', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init(`
|
||||
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code } }
|
||||
`))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'stopping' })
|
||||
host.send({ type: HostToWorkerType.ChildStartError, callId, rendered: 'workflow run cancelled: stopping' })
|
||||
const result = await host.result()
|
||||
// The run reports cancelled (the script died of CANCELLED, not AGENT_START).
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
host.close()
|
||||
})
|
||||
|
||||
it('a child result rejection while cancelled pairs a cancelled agent-end, and the run reports cancelled', async () => {
|
||||
const host = fakeHost({ manual: true })
|
||||
void runWorkerSession(host.port, init("return await agent('doomed')"))
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.ChildStart).length).toBe(1) })
|
||||
const callId = host.ofType(WorkerToHostType.ChildStart)[0]!.callId
|
||||
host.send({ type: HostToWorkerType.ChildStarted, callId, childId: 'child-0' })
|
||||
await vi.waitFor(() => { expect(host.ofType(WorkerToHostType.AgentStart).length).toBe(1) })
|
||||
host.send({ type: HostToWorkerType.Cancel, reason: 'user aborted' })
|
||||
host.send({ type: HostToWorkerType.ChildFailed, callId, rendered: 'backend crashed on abort' })
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('cancelled')
|
||||
expect(host.ofType(WorkerToHostType.AgentEnd)[0]!.info.outcome).toBe('cancelled')
|
||||
host.close()
|
||||
})
|
||||
|
||||
})
|
||||
|
||||
describe('the worker bootstrap', () => {
|
||||
it('requireParentPort narrows a real port and throws on the main thread', () => {
|
||||
const channel = new MessageChannel()
|
||||
expect(requireParentPort(channel.port1)).toBe(channel.port1)
|
||||
channel.port1.close()
|
||||
expect(() => requireParentPort(null)).toThrow(/inside a worker thread/)
|
||||
})
|
||||
|
||||
it('the entry module itself throws when loaded on the main thread (no parentPort)', async () => {
|
||||
// This import EXECUTES ../src/worker.ts on the main thread, which is what
|
||||
// covers the bootstrap file: requireParentPort throws before
|
||||
// runWorkerSession is reached.
|
||||
await expect(import('../src/worker.ts')).rejects.toThrow(/inside a worker thread/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* Keyless runtime smoke for the source-mode workflow worker. The Node
|
||||
* compatibility matrix runs this WHOLE file, so renaming or removing its test
|
||||
* cannot turn the runtime proof into a successful zero-match filter.
|
||||
*/
|
||||
|
||||
import { expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentProvider } from '@deepseek-ai/dsh-subagent'
|
||||
import WorkerThreadWorkflowEngine from '../src/index.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
// A fresh thread compiles the source runtime. Leave contention headroom on
|
||||
// shared CI runners without weakening any engine-level timeout assertion.
|
||||
vi.setConfig({ testTimeout: 30_000 })
|
||||
|
||||
it('runs the default config through the source worker', async () => {
|
||||
const ctx = new Context()
|
||||
const subagents = await ctx.plugin(SubagentRuntime)
|
||||
const provider: SubagentProvider = {
|
||||
name: 'spawn',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => Promise.reject(new Error('source-worker compat script must not start a child')),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
const engine = await ctx.plugin(WorkerThreadWorkflowEngine, {})
|
||||
const parent = { id: SessionId('workflow-compat-parent'), options: {} } as unknown as Agent
|
||||
try {
|
||||
const run = ctx.workflowEngine.start({
|
||||
script: 'return 6 * 7',
|
||||
meta: { name: 'source-worker-compat', description: 'exercise the unbuilt worker entry' },
|
||||
parent,
|
||||
})
|
||||
try {
|
||||
await expect(run.result).resolves.toMatchObject({ value: 42, stopReason: 'completed', agentsStarted: 0 })
|
||||
} finally {
|
||||
await run.dispose()
|
||||
}
|
||||
} finally {
|
||||
await engine.dispose()
|
||||
await subagents.dispose()
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,102 @@
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import LlmRuntime from '@deepseek-ai/dsh-llm'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRuntime from '@deepseek-ai/dsh-tools'
|
||||
import AgentRegistry from '@deepseek-ai/dsh-agent'
|
||||
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import * as Spawn from '@deepseek-ai/dsh-subagent-spawn-in-process'
|
||||
import WorkerThreadWorkflowEngine from '../src/index.ts'
|
||||
|
||||
/**
|
||||
* With-key e2e: a REAL script in a REAL worker thread
|
||||
* drives REAL spawn children against the live DeepSeek API — one plain child
|
||||
* and one schema'd child through the real structured-output runtime — and
|
||||
* the run's value, events, and child sessions are asserted from the outside
|
||||
* (never the script's self-report alone). Key-gated (self-skips without
|
||||
* DEEPSEEK_API_KEY).
|
||||
*/
|
||||
|
||||
let ctx: Context | undefined
|
||||
|
||||
afterEach(async () => {
|
||||
await ctx?.fiber.dispose()
|
||||
ctx = undefined
|
||||
})
|
||||
|
||||
async function harness(): Promise<Context> {
|
||||
const built = new Context()
|
||||
await built.plugin(LlmRuntime)
|
||||
await built.plugin(SessionStore)
|
||||
await built.plugin(SystemPrompt)
|
||||
await built.plugin(ToolRuntime)
|
||||
await built.plugin(AgentRegistry)
|
||||
await built.plugin(AgentLoop, { agents: [] })
|
||||
await built.plugin(LlmDeepSeek)
|
||||
await built.plugin(SubagentRuntime)
|
||||
await built.plugin(Spawn, { providerName: 'spawn' })
|
||||
await built.plugin(WorkerThreadWorkflowEngine, { provider: 'spawn' })
|
||||
return built
|
||||
}
|
||||
|
||||
const META = {
|
||||
name: 'e2e-worker-arithmetic',
|
||||
description: 'two real children through a worker thread: one prose, one structured',
|
||||
phases: [{ title: 'Ask' }, { title: 'Judge' }],
|
||||
}
|
||||
const SCRIPT = `phase('Ask')
|
||||
log('asking the prose child')
|
||||
const prose = await agent('Reply with exactly one short sentence: what is 2 + 2?')
|
||||
phase('Judge')
|
||||
const judged = await agent(
|
||||
'Here is an answer to the question "what is 2+2": ' + prose
|
||||
+ ' — report whether it contains the number 4 and your confidence between 0 and 1.',
|
||||
{ schema: { type: 'object', properties: { containsFour: { type: 'boolean' }, confidence: { type: 'number' } }, required: ['containsFour'] } },
|
||||
)
|
||||
return { prose, containsFour: judged === null ? null : judged.containsFour }`
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY)('worker workflow engine with-key e2e', () => {
|
||||
it('runs a two-phase script in a worker thread over real children, one through the structured runtime', async () => {
|
||||
ctx = await harness()
|
||||
const parentHandle = await ctx.agents.create({
|
||||
sessionId: 'wf-worker-e2e-session' as never,
|
||||
agentOptions: { provider: 'deepseek-official', model: 'deepseek-v4-flash' },
|
||||
})
|
||||
|
||||
const events: string[] = []
|
||||
const childIds: string[] = []
|
||||
for (const name of ['workflow/start', 'workflow/phase', 'workflow/log', 'workflow/agent-start', 'workflow/agent-end', 'workflow/end'] as const) {
|
||||
ctx.on(name, (...payload: unknown[]) => {
|
||||
events.push(name)
|
||||
if (name === 'workflow/agent-start') childIds.push((payload[1] as { childId: string }).childId)
|
||||
})
|
||||
}
|
||||
|
||||
const run = ctx.workflowEngine.start({ script: SCRIPT, meta: META, parent: parentHandle.agent })
|
||||
const result = await run.result
|
||||
await run.dispose()
|
||||
|
||||
expect(result.stopReason).toBe('completed')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
const value = result.value as { prose: string; containsFour: boolean | null }
|
||||
// World checks: the prose child really answered (a real completion), and
|
||||
// the structured child judged it against the REAL schema-forced tool.
|
||||
expect(value.prose.length).toBeGreaterThan(0)
|
||||
expect(value.containsFour).toBe(true)
|
||||
|
||||
expect(events[0]).toBe('workflow/start')
|
||||
expect(events.at(-1)).toBe('workflow/end')
|
||||
expect(events.filter(name => name === 'workflow/phase').length).toBe(2)
|
||||
expect(events.filter(name => name === 'workflow/agent-start').length).toBe(2)
|
||||
expect(childIds.length).toBe(2)
|
||||
// The children were disposed to quiescence after collection.
|
||||
for (const childId of childIds) {
|
||||
expect(ctx.agents.get(SessionId(childId))).toBeUndefined()
|
||||
}
|
||||
await parentHandle.dispose()
|
||||
}, 240_000)
|
||||
})
|
||||
File diff suppressed because it is too large
Load Diff
45
packages/workflow/workflow-worker-thread/tsconfig.json
Normal file
45
packages/workflow/workflow-worker-thread/tsconfig.json
Normal file
@@ -0,0 +1,45 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../util/brand"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../subagent/subagent"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
},
|
||||
{
|
||||
"path": "../workflow"
|
||||
},
|
||||
{
|
||||
"path": "../../runtime-diagnostics/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
29
packages/workflow/workflow-worker-thread/tsdown.config.ts
Normal file
29
packages/workflow/workflow-worker-thread/tsdown.config.ts
Normal file
@@ -0,0 +1,29 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/**
|
||||
* Build the engine and worker separately so each inlines shared modules; a
|
||||
* multi-entry build creates an unlisted chunk. The path-loaded worker is
|
||||
* CommonJS because pkg's VFS Worker hook compiles it in that format.
|
||||
*/
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['lib/types/index.js', 'lib/types/invariant.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
{
|
||||
entry: ['lib/types/worker.js'],
|
||||
outDir: 'lib',
|
||||
format: ['cjs'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
},
|
||||
])
|
||||
Reference in New Issue
Block a user