Merge origin/master into codex/invariant-service-seam
# Conflicts: # docs/module-graph.md
This commit is contained in:
Tianyi Cui
@@ -7,7 +7,8 @@ The workflow seam: a model-written JavaScript orchestration script that fans out
|
||||
| `workflow/` | Abstract workflow seam: service base class + run vocabulary + `workflow/*` events | `ctx.workflows` |
|
||||
| `workflow-workerthread/` | `node:worker_threads` engine: one worker per run; the script's vm context lives inside the worker, `agent()` bridges to `ctx.subagents` over the message port | (provides `ctx.workflows`) |
|
||||
| `tool-workflow/` | Model-facing `workflow` tool over `ctx.workflows` | (registers on `ctx.tools`) |
|
||||
| `tool-ralph/` | Fixed fresh-agent Ralph policy over `ctx.workflows` and a fresh structured-output subagent provider | (registers on `ctx.tools`) |
|
||||
|
||||
The interface lives at `workflow/workflow/`. The engine's `agent()` hook rides the [subagent seam](../subagent/README.md) (any registered provider; the shipped examples use `spawn`), and `agent({ schema })` rides the structured-output support the in-process backends implement. The worker thread isolates the SCRIPT — the host never blocks on it, and a cancelled run's post-grace termination is real — but it is NOT a security boundary; an isolated-vm/separate-process engine (actual sandboxing) swaps in behind the same interface if that ever matters.
|
||||
|
||||
The proposal, decisions, and deferred work: [.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md).
|
||||
The general script engine's decisions and deferred work live in the [dynamic-workflows Agent Note](../../.agents/notes/implemented/feature/2026-07-05-dynamic-workflows.md). The separate [Ralph consumer](../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.md) fixes the script and fresh-provider policy rather than adding another engine or an agent-loop mode.
|
||||
|
||||
91
packages/workflow/tool-ralph/README.md
Normal file
91
packages/workflow/tool-ralph/README.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# @deepseek-ai/dsh-tool-ralph
|
||||
|
||||
The model-facing `ralph` tool runs a fixed foreground workflow that gives one immutable objective to a sequence of fresh child agents. It demonstrates a specialized orchestration policy as an ordinary plugin over [`ctx.workflows`](../workflow/README.md) and [`ctx.subagents`](../../subagent/subagent/README.md): no Ralph mode or fresh-agent loop is added to `agent-loop`, and the same-session [goal domain](../../goal/goal/README.md) remains independent. The [Ralph Agent Note](../../../.agents/notes/implemented/feature/2026-07-19-fresh-agent-ralph-workflow-tool.md) owns the policy and deferred work.
|
||||
|
||||
## Contract
|
||||
|
||||
`ralph({ objective, maxRounds? })` waits for the entire run. The deployment config's `maxRounds` is both the default and a ceiling on a call override. Every Ralph round starts one child through `subagentProvider`; that provider must exist, support structured output, and report `inheritsParentContext: false`. The configured provider is carried as `WorkflowStartRequest.subagentProvider`, so the fixed script cannot inspect or change routing and the ordinary model-written `workflow` tool gains no provider selector. The resolved round cap is also carried as `WorkflowStartRequest.maxTotalAgents`, coordinating the fixed loop with the engine's total-child backstop; the engine rejects a Ralph cap above its deployment ceiling before publishing a run.
|
||||
|
||||
Each child receives only the immutable objective, its current Ralph round and cap, a shared-workspace-as-authority instruction, and the previous structured handoff. The workspace is long-term memory; parent conversation and prior child sessions are not seeded. Reports have `status: continue | complete | blocked`, a non-empty summary, evidence, next steps, and blocker text. Status-specific semantics and the serialized `maxHandoffChars` ceiling are validated inside the fixed workflow and again at the consumer boundary. Invalid, missing, or oversized reports fail the workflow instead of being truncated or mistaken for cap exhaustion.
|
||||
|
||||
The successful terminal tool result is `complete`, `blocked`, or `budget-limited`, with the last bounded report and number of rounds started. Completion and blocker labels explicitly say that a worker reported the outcome; they are not independent certification. `maxResultChars` bounds the complete successful text including its envelope and truncation marker, without altering the validated report used as a cross-round handoff.
|
||||
|
||||
An ordinary child failure produces an error naming the failed round and retaining the last successful handoff when one exists. Ralph does not retry that round. Fatal provider-start, transport, worker, or workflow failures remain workflow errors and may settle before the fixed script can return a handoff. Cancellation is also an error; partial output is never success.
|
||||
|
||||
## Lifecycle and cancellation
|
||||
|
||||
The caller's agent is the parent of every fresh child, preserving cwd and lineage without copying its conversation. `exec.signal` enters the workflow engine and is also bridged to `run.cancel()` for implementation independence. The tool awaits `run.result` and calls `run.dispose()` in `finally`, so a cancelled parent step waits for the engine's bounded termination and child quiescence before returning.
|
||||
|
||||
## Render intent
|
||||
|
||||
The pending call is a `generic` card titled `ralph`; the immutable objective is its `rawInput`. The result keeps the generic card. Both presentation functions depend only on tool arguments and the settled tool envelope.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `subagentProvider` | `spawn` | Fresh structured-output provider used for every round. |
|
||||
| `maxRounds` | `256` | Default and deployment ceiling for one Ralph run. |
|
||||
| `maxHandoffChars` | `16384` | Maximum serialized characters in one round report. |
|
||||
| `maxResultChars` | `16384` | Maximum characters in the complete successful parent result. |
|
||||
|
||||
All config values are normalized and validated when the plugin applies, including direct application outside Loader schema normalization. Provider capabilities are resolved immediately before each call because provider registration can change under plugin lifecycle and HMR.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### System prompt
|
||||
|
||||
#### What the model sees
|
||||
|
||||
Every parent request in this plugin's registration scope receives the fixed routing guidance below.
|
||||
|
||||
##### Ralph guidance
|
||||
|
||||
```markdown
|
||||
Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Completion and blockers are worker reports, not independent evaluation. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.
|
||||
```
|
||||
|
||||
#### Token effect
|
||||
|
||||
Small fixed guidance cost per request while the plugin is active.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Prefix-stable while the plugin scope and guidance text are unchanged. Activation or disposal may invalidate reuse from this prompt section.
|
||||
|
||||
### Tool schema
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The generated [`ralph` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tool-ralph) exposes one required `objective` string and one optional `maxRounds` number. Provider choice, handoff size, report schema, workflow script, and orchestration behavior are deployment-owned and absent from the call surface.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Small fixed schema cost on each request where the tool is visible.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Prefix-stable while the definition and visibility are unchanged.
|
||||
|
||||
### Child requests and parent result
|
||||
|
||||
#### What the model sees
|
||||
|
||||
Each child sees the standalone fixed round prompt plus the structured-output capture contract. The parent sees only the original call and one terminal result containing a worker-reported status, round count, and pretty-printed final report; intermediate child messages and reports do not enter the parent conversation. A failed ordinary child instead yields an error with its round number and, after round one, the last successful handoff.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Every round pays for a fresh child context. `maxHandoffChars` bounds cross-round state and `maxResultChars` independently bounds the complete successful parent text; child work remains outside the parent context.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Each fresh child has an independent request cache. The parent result appends after the reusable request prefix.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Completion is worker self-declaration** — there is no independent evaluator or verifier deciding whether the objective is actually complete; evaluator policy and evaluator-driven continuation are deferred.
|
||||
- **Foreground only** — there is no task id, background collection, process-resume checkpoint, scheduler, or wall-clock start policy.
|
||||
- **The workspace is the only cross-round long-term memory** — one bounded report is the explicit handoff, and uncommitted conversational reasoning disappears with each child.
|
||||
- **One round is one fresh child** — there is no within-round fan-out, model/provider switching, fork context, or model-call-selected provider.
|
||||
- **Ordinary child failure is terminal for the run** — the fixed script reports the failed round and last successful handoff but does not retry; fatal workflow infrastructure failures can end before that state is returned.
|
||||
- **Only round count bounds aggregate effort** — token, price, and elapsed-time budgets are deferred.
|
||||
59
packages/workflow/tool-ralph/package.json
Normal file
59
packages/workflow/tool-ralph/package.json
Normal file
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-tool-ralph",
|
||||
"description": "Model-facing fresh-agent Ralph loop over the workflow and subagent seams",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-agent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-subagent": "^0.0.1",
|
||||
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
|
||||
"@deepseek-ai/dsh-tools": "^0.0.1",
|
||||
"@deepseek-ai/dsh-workflow": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"dependencies": {
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cordisjs/plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop": "workspace:^",
|
||||
"@deepseek-ai/dsh-agent-loop-testkit": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent-inprocess": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent-spawn": "workspace:^",
|
||||
"@deepseek-ai/dsh-system-prompt": "workspace:^",
|
||||
"@deepseek-ai/dsh-tools": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow": "workspace:^",
|
||||
"@deepseek-ai/dsh-workflow-workerthread": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
456
packages/workflow/tool-ralph/src/index.ts
Normal file
456
packages/workflow/tool-ralph/src/index.ts
Normal file
@@ -0,0 +1,456 @@
|
||||
/**
|
||||
* Model-facing foreground Ralph loop over the workflow and subagent seams. A
|
||||
* fixed script starts one fresh structured-output child per round, carrying
|
||||
* only the immutable objective and the previous bounded handoff between them.
|
||||
* @module @deepseek-ai/dsh-tool-ralph
|
||||
*/
|
||||
|
||||
import type { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
|
||||
import type { SubagentProvider } from '@deepseek-ai/dsh-subagent'
|
||||
import { defineTool } from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolCallView, ToolResultView } from '@deepseek-ai/dsh-tools'
|
||||
import type { WorkflowResult, WorkflowRun } from '@deepseek-ai/dsh-workflow'
|
||||
// Declaration merge only: makes ctx.systemPrompt visible for section registration.
|
||||
import type {} from '@deepseek-ai/dsh-system-prompt'
|
||||
|
||||
export const name = 'tool-ralph'
|
||||
export const inject = ['tools', 'workflows', 'subagents', 'systemPrompt']
|
||||
|
||||
/** Deployment policy for the fixed Ralph workflow. */
|
||||
export interface Config {
|
||||
/** Fresh structured-output provider used for every round (default `spawn`). */
|
||||
subagentProvider?: string
|
||||
/** Default and deployment ceiling for one call's round count (default 256). */
|
||||
maxRounds?: number
|
||||
/** Maximum serialized characters in one structured handoff (default 16384). */
|
||||
maxHandoffChars?: number
|
||||
/** Maximum characters in a successful parent-facing terminal text (default 16384). */
|
||||
maxResultChars?: number
|
||||
}
|
||||
|
||||
/** Schemastery configuration for the Ralph tool. */
|
||||
export const Config: z<Config> = z.object({
|
||||
subagentProvider: z.string().default('spawn'),
|
||||
maxRounds: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(256),
|
||||
maxHandoffChars: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(16_384),
|
||||
maxResultChars: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(16_384),
|
||||
})
|
||||
|
||||
interface ResolvedConfig {
|
||||
readonly subagentProvider: string
|
||||
readonly maxRounds: number
|
||||
readonly maxHandoffChars: number
|
||||
readonly maxResultChars: number
|
||||
}
|
||||
|
||||
type RalphRoundStatus = 'continue' | 'complete' | 'blocked'
|
||||
|
||||
interface RalphRoundReport {
|
||||
readonly status: RalphRoundStatus
|
||||
readonly summary: string
|
||||
readonly evidence: string[]
|
||||
readonly nextSteps: string[]
|
||||
readonly blocker: string
|
||||
}
|
||||
|
||||
type RalphRunStatus = 'complete' | 'blocked' | 'budget-limited'
|
||||
|
||||
interface RalphRunResult {
|
||||
readonly status: RalphRunStatus
|
||||
readonly roundsStarted: number
|
||||
readonly report: RalphRoundReport
|
||||
}
|
||||
|
||||
interface RalphRoundFailure {
|
||||
readonly status: 'round-failed'
|
||||
readonly roundsStarted: number
|
||||
readonly lastReport?: RalphRoundReport
|
||||
}
|
||||
|
||||
type RalphTerminalResult = RalphRunResult | RalphRoundFailure
|
||||
|
||||
interface RalphCallArgs {
|
||||
objective: string
|
||||
maxRounds?: number
|
||||
}
|
||||
|
||||
const RALPH_META = {
|
||||
name: 'ralph-loop',
|
||||
description: 'Iterate toward one objective with a fresh child and bounded structured handoff per round.',
|
||||
phases: [{ title: 'Fresh-agent rounds', detail: 'One clean child context per Ralph round.' }],
|
||||
}
|
||||
|
||||
/**
|
||||
* Fixed, deployment-owned orchestration. The model supplies data only; it
|
||||
* cannot alter the loop, provider route, schema, or handoff validation.
|
||||
*/
|
||||
const RALPH_SCRIPT = String.raw`
|
||||
const reportSchema = {
|
||||
type: 'object',
|
||||
properties: {
|
||||
status: { type: 'string', enum: ['continue', 'complete', 'blocked'] },
|
||||
summary: { type: 'string' },
|
||||
evidence: { type: 'array', items: { type: 'string' } },
|
||||
nextSteps: { type: 'array', items: { type: 'string' } },
|
||||
blocker: { type: 'string' },
|
||||
},
|
||||
required: ['status', 'summary', 'evidence', 'nextSteps', 'blocker'],
|
||||
additionalProperties: false,
|
||||
}
|
||||
|
||||
function normalizedText(value) {
|
||||
return typeof value === 'string' && value.length > 0 && value === value.trim()
|
||||
}
|
||||
|
||||
function normalizedList(value) {
|
||||
return Array.isArray(value) && value.every(normalizedText)
|
||||
}
|
||||
|
||||
function validateReport(report) {
|
||||
if (report === null || typeof report !== 'object' || Array.isArray(report)) {
|
||||
throw new Error('Ralph child returned no structured round report')
|
||||
}
|
||||
if (!normalizedText(report.summary)) {
|
||||
throw new Error('Ralph round report summary must be non-empty and normalized')
|
||||
}
|
||||
if (!normalizedList(report.evidence) || !normalizedList(report.nextSteps)) {
|
||||
throw new Error('Ralph round report evidence and nextSteps must contain only non-empty normalized strings')
|
||||
}
|
||||
if (typeof report.blocker !== 'string' || report.blocker !== report.blocker.trim()) {
|
||||
throw new Error('Ralph round report blocker must be a normalized string')
|
||||
}
|
||||
switch (report.status) {
|
||||
case 'continue':
|
||||
if (report.nextSteps.length === 0 || report.blocker !== '') {
|
||||
throw new Error('a continuing Ralph report needs nextSteps and an empty blocker')
|
||||
}
|
||||
break
|
||||
case 'complete':
|
||||
if (report.evidence.length === 0 || report.nextSteps.length !== 0 || report.blocker !== '') {
|
||||
throw new Error('a complete Ralph report needs evidence, no nextSteps, and an empty blocker')
|
||||
}
|
||||
break
|
||||
case 'blocked':
|
||||
if (!normalizedText(report.blocker)) {
|
||||
throw new Error('a blocked Ralph report needs a concrete blocker')
|
||||
}
|
||||
break
|
||||
default:
|
||||
throw new Error('Ralph round report status is invalid')
|
||||
}
|
||||
const serialized = JSON.stringify(report)
|
||||
if (serialized.length > args.maxHandoffChars) {
|
||||
throw new Error('Ralph round report exceeds maxHandoffChars (' + serialized.length + ' > ' + args.maxHandoffChars + ')')
|
||||
}
|
||||
return report
|
||||
}
|
||||
|
||||
let previous
|
||||
phase('Fresh-agent rounds')
|
||||
for (let round = 1; round <= args.maxRounds; round += 1) {
|
||||
const prior = previous === undefined ? '(none — this is the first round)' : JSON.stringify(previous)
|
||||
const prompt = [
|
||||
'You are one fresh worker in a foreground Ralph loop. You receive no parent conversation and no prior child session. Do not call the ralph tool: this round already is its worker.',
|
||||
'Immutable objective:\n' + args.objective,
|
||||
'Ralph round: ' + round + ' of ' + args.maxRounds + '.',
|
||||
'The shared workspace and its current working tree are the long-term memory and source of truth. Inspect them before acting, preserve existing work, perform concrete in-scope work, and verify what you change. Treat the previous report only as a bounded handoff; confirm it against the workspace.',
|
||||
'Previous structured handoff:\n' + prior,
|
||||
'Return one report with exact normalized strings. Use status continue with at least one nextSteps entry while useful work remains; complete only with concrete evidence and no nextSteps; blocked only when no meaningful progress is possible without human input or an external-state change. blocker must be empty unless blocked.',
|
||||
].join('\n\n')
|
||||
const rawReport = await agent(prompt, {
|
||||
label: 'Ralph round ' + round,
|
||||
phase: 'Fresh-agent rounds',
|
||||
schema: reportSchema,
|
||||
})
|
||||
if (rawReport === null) {
|
||||
return { status: 'round-failed', roundsStarted: round, lastReport: previous ?? null }
|
||||
}
|
||||
const report = validateReport(rawReport)
|
||||
if (report.status === 'complete') return { status: 'complete', roundsStarted: round, report }
|
||||
if (report.status === 'blocked') return { status: 'blocked', roundsStarted: round, report }
|
||||
previous = report
|
||||
}
|
||||
return { status: 'budget-limited', roundsStarted: args.maxRounds, report: previous }
|
||||
`
|
||||
|
||||
const DESCRIPTION = 'Run a foreground fresh-agent Ralph loop toward one immutable objective. '
|
||||
+ 'Use only when the direct human explicitly asks for Ralph or fresh-agent iteration. Each round '
|
||||
+ 'opens a new child with no parent conversation or prior child session; the shared workspace is '
|
||||
+ 'long-term memory, and only a bounded structured report crosses rounds. The call returns when '
|
||||
+ 'a worker reports completion or a concrete blocker, or at the round limit. Ordinary long-running same-session work '
|
||||
+ 'belongs to goal tools.'
|
||||
|
||||
/** Validate defaults even when a caller invokes apply() without Loader normalization. */
|
||||
function resolveConfig(config: Config): ResolvedConfig {
|
||||
const subagentProvider = config.subagentProvider ?? 'spawn'
|
||||
const maxRounds = config.maxRounds ?? 256
|
||||
const maxHandoffChars = config.maxHandoffChars ?? 16_384
|
||||
const maxResultChars = config.maxResultChars ?? 16_384
|
||||
if (subagentProvider.length === 0 || subagentProvider !== subagentProvider.trim()) {
|
||||
throw new TypeError('subagentProvider must be a non-empty normalized string')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxRounds) || maxRounds < 1) {
|
||||
throw new TypeError('maxRounds must be a positive safe integer')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxHandoffChars) || maxHandoffChars < 1) {
|
||||
throw new TypeError('maxHandoffChars must be a positive safe integer')
|
||||
}
|
||||
if (!Number.isSafeInteger(maxResultChars) || maxResultChars < 1) {
|
||||
throw new TypeError('maxResultChars must be a positive safe integer')
|
||||
}
|
||||
return { subagentProvider, maxRounds, maxHandoffChars, maxResultChars }
|
||||
}
|
||||
|
||||
/** Resolve one model-selected cap against the deployment ceiling. */
|
||||
function resolveMaxRounds(requested: number | undefined, ceiling: number): number {
|
||||
const value = requested ?? ceiling
|
||||
if (!Number.isSafeInteger(value) || value < 1) {
|
||||
throw new TypeError('Ralph maxRounds must be a positive safe integer')
|
||||
}
|
||||
if (value > ceiling) {
|
||||
throw new TypeError(`Ralph maxRounds ${value} exceeds the deployment ceiling ${ceiling}`)
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Require the configured route to mean a genuinely fresh structured child. */
|
||||
function requireFreshProvider(ctx: Context, name: string): SubagentProvider {
|
||||
const provider = ctx.subagents.getProvider(name)
|
||||
if (provider === undefined) {
|
||||
throw new Error(`Ralph subagent provider "${name}" is not registered`)
|
||||
}
|
||||
if (!provider.capabilities.outputSchema) {
|
||||
throw new Error(`Ralph subagent provider "${name}" does not support structured output`)
|
||||
}
|
||||
if (provider.inheritsParentContext) {
|
||||
throw new Error(`Ralph subagent provider "${name}" inherits parent context; Ralph requires a fresh provider`)
|
||||
}
|
||||
return provider
|
||||
}
|
||||
|
||||
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||
return typeof value === 'object' && value !== null && !Array.isArray(value)
|
||||
}
|
||||
|
||||
function normalizedText(value: unknown): value is string {
|
||||
return typeof value === 'string' && value.length > 0 && value === value.trim()
|
||||
}
|
||||
|
||||
function normalizedList(value: unknown): value is string[] {
|
||||
return Array.isArray(value) && value.every(normalizedText)
|
||||
}
|
||||
|
||||
/** Defensively decode the fixed script's report across an implementation seam. */
|
||||
function readReport(value: unknown, expectedStatus: RalphRoundStatus, maxChars: number): RalphRoundReport {
|
||||
if (!isRecord(value)
|
||||
|| Object.keys(value).sort().join(',') !== 'blocker,evidence,nextSteps,status,summary'
|
||||
|| value['status'] !== expectedStatus
|
||||
|| !normalizedText(value['summary'])
|
||||
|| !normalizedList(value['evidence'])
|
||||
|| !normalizedList(value['nextSteps'])
|
||||
|| typeof value['blocker'] !== 'string'
|
||||
|| value['blocker'] !== value['blocker'].trim()) {
|
||||
throw new Error('Ralph workflow returned a malformed round report')
|
||||
}
|
||||
const report: RalphRoundReport = {
|
||||
status: expectedStatus,
|
||||
summary: value['summary'],
|
||||
evidence: value['evidence'],
|
||||
nextSteps: value['nextSteps'],
|
||||
blocker: value['blocker'],
|
||||
}
|
||||
if (expectedStatus === 'continue' && (report.nextSteps.length === 0 || report.blocker !== '')) {
|
||||
throw new Error('Ralph workflow returned an invalid continuing report')
|
||||
}
|
||||
if (expectedStatus === 'complete'
|
||||
&& (report.evidence.length === 0 || report.nextSteps.length !== 0 || report.blocker !== '')) {
|
||||
throw new Error('Ralph workflow returned an invalid completion report')
|
||||
}
|
||||
if (expectedStatus === 'blocked' && !normalizedText(report.blocker)) {
|
||||
throw new Error('Ralph workflow returned an invalid blocked report')
|
||||
}
|
||||
const chars = JSON.stringify(report).length
|
||||
if (chars > maxChars) {
|
||||
throw new Error(`Ralph workflow returned an oversized handoff (${chars} > ${maxChars})`)
|
||||
}
|
||||
return report
|
||||
}
|
||||
|
||||
/** Defensively decode the fixed script's terminal value. */
|
||||
function readRunResult(value: unknown, maxRounds: number, maxHandoffChars: number): RalphTerminalResult {
|
||||
if (!isRecord(value)
|
||||
|| typeof value['roundsStarted'] !== 'number'
|
||||
|| !Number.isSafeInteger(value['roundsStarted'])
|
||||
|| value['roundsStarted'] < 1
|
||||
|| value['roundsStarted'] > maxRounds) {
|
||||
throw new Error('Ralph workflow returned a malformed terminal result')
|
||||
}
|
||||
const roundsStarted = value['roundsStarted']
|
||||
switch (value['status']) {
|
||||
case 'complete':
|
||||
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
|
||||
throw new Error('Ralph workflow returned a malformed terminal result')
|
||||
}
|
||||
return { status: 'complete', roundsStarted, report: readReport(value['report'], 'complete', maxHandoffChars) }
|
||||
case 'blocked':
|
||||
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
|
||||
throw new Error('Ralph workflow returned a malformed terminal result')
|
||||
}
|
||||
return { status: 'blocked', roundsStarted, report: readReport(value['report'], 'blocked', maxHandoffChars) }
|
||||
case 'budget-limited':
|
||||
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
|
||||
throw new Error('Ralph workflow returned a malformed terminal result')
|
||||
}
|
||||
if (roundsStarted !== maxRounds) {
|
||||
throw new Error('Ralph workflow returned budget-limited before the round limit')
|
||||
}
|
||||
return { status: 'budget-limited', roundsStarted, report: readReport(value['report'], 'continue', maxHandoffChars) }
|
||||
case 'round-failed': {
|
||||
if (Object.keys(value).sort().join(',') !== 'lastReport,roundsStarted,status') {
|
||||
throw new Error('Ralph workflow returned a malformed terminal result')
|
||||
}
|
||||
if (roundsStarted === 1) {
|
||||
if (value['lastReport'] !== null) {
|
||||
throw new Error('Ralph workflow returned an invalid first-round failure')
|
||||
}
|
||||
return { status: 'round-failed', roundsStarted }
|
||||
}
|
||||
if (value['lastReport'] === null) {
|
||||
throw new Error('Ralph workflow returned a round failure without its last handoff')
|
||||
}
|
||||
return {
|
||||
status: 'round-failed',
|
||||
roundsStarted,
|
||||
lastReport: readReport(value['lastReport'], 'continue', maxHandoffChars),
|
||||
}
|
||||
}
|
||||
default:
|
||||
throw new Error('Ralph workflow returned an unknown terminal status')
|
||||
}
|
||||
}
|
||||
|
||||
/** A non-clean workflow finish is an error, never a partial Ralph success. */
|
||||
function stopReasonError(result: WorkflowResult): string | undefined {
|
||||
switch (result.stopReason) {
|
||||
case 'completed':
|
||||
return undefined
|
||||
case 'cancelled':
|
||||
return `Ralph workflow was cancelled${result.error === undefined ? '' : ` (${result.error})`}`
|
||||
case 'error':
|
||||
return `Ralph workflow failed: ${result.error ?? 'unknown error'}`
|
||||
/* v8 ignore start -- WorkflowStopReason is closed; a future variant must fail loud here. */
|
||||
default:
|
||||
return `Ralph workflow ended abnormally (${String(result.stopReason satisfies never)})`
|
||||
/* v8 ignore stop */
|
||||
}
|
||||
}
|
||||
|
||||
const TRUNCATION_NOTICE = '\n… [truncated]'
|
||||
|
||||
/** Bound complete parent-facing text, including its envelope and truncation marker. */
|
||||
function boundResult(text: string, maxChars: number): string {
|
||||
if (text.length <= maxChars) return text
|
||||
if (maxChars <= TRUNCATION_NOTICE.length) return TRUNCATION_NOTICE.slice(0, maxChars)
|
||||
return `${text.slice(0, maxChars - TRUNCATION_NOTICE.length)}${TRUNCATION_NOTICE}`
|
||||
}
|
||||
|
||||
/** Render the fixed terminal envelope without presenting self-report as certification. */
|
||||
function renderResult(result: RalphRunResult, maxChars: number): string {
|
||||
const rounds = `${result.roundsStarted} round${result.roundsStarted === 1 ? '' : 's'}`
|
||||
let text: string
|
||||
switch (result.status) {
|
||||
case 'complete':
|
||||
text = `Ralph worker reported completion after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
|
||||
break
|
||||
case 'blocked':
|
||||
text = `Ralph worker reported a blocker after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
|
||||
break
|
||||
case 'budget-limited':
|
||||
text = `Ralph reached its ${rounds} limit; the worker reported work remaining.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
|
||||
break
|
||||
}
|
||||
return boundResult(text, maxChars)
|
||||
}
|
||||
|
||||
/** Render an ordinary child failure with the most recent durable handoff. */
|
||||
function renderRoundFailure(result: RalphRoundFailure, maxChars: number): string {
|
||||
const header = `Ralph round ${result.roundsStarted} child failed before producing a structured report.`
|
||||
const text = result.lastReport === undefined
|
||||
? `${header}\nNo previous handoff was available.`
|
||||
: `${header}\nLast successful handoff:\n${JSON.stringify(result.lastReport, null, 2)}`
|
||||
return boundResult(text, maxChars)
|
||||
}
|
||||
|
||||
function presentCall(args: RalphCallArgs): ToolCallView {
|
||||
return { card: 'generic', title: 'ralph', rawInput: args.objective }
|
||||
}
|
||||
|
||||
function presentResult(args: RalphCallArgs, result: { content: ContentBlock[]; isError: boolean }): ToolResultView {
|
||||
void args
|
||||
void result
|
||||
return { card: 'generic' }
|
||||
}
|
||||
|
||||
/** Register the fixed Ralph tool and its explicit-ask usage policy. */
|
||||
export function apply(ctx: Context, config: Config): void {
|
||||
const resolved = resolveConfig(config)
|
||||
ctx.systemPrompt.section({
|
||||
name: 'tool:ralph',
|
||||
order: 116,
|
||||
text: 'Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Completion and blockers are worker reports, not independent evaluation. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.',
|
||||
})
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'ralph',
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
objective: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
description: 'The immutable completion objective for every fresh Ralph round.',
|
||||
},
|
||||
maxRounds: {
|
||||
type: 'number',
|
||||
description: 'Optional positive safe-integer round cap, bounded by the deployment ceiling.',
|
||||
},
|
||||
},
|
||||
async execute(args, exec): Promise<ContentBlock[]> {
|
||||
const parent = exec.agent
|
||||
if (parent === undefined) {
|
||||
throw new Error('Ralph tool requires a calling agent (exec.agent was undefined)')
|
||||
}
|
||||
const objective = args.objective.trim()
|
||||
if (objective.length === 0) throw new Error('Ralph objective must be a non-empty string')
|
||||
const maxRounds = resolveMaxRounds(args.maxRounds, resolved.maxRounds)
|
||||
void requireFreshProvider(ctx, resolved.subagentProvider)
|
||||
|
||||
const run: WorkflowRun = ctx.workflows.start({
|
||||
script: RALPH_SCRIPT,
|
||||
meta: RALPH_META,
|
||||
args: { objective, maxRounds, maxHandoffChars: resolved.maxHandoffChars },
|
||||
subagentProvider: resolved.subagentProvider,
|
||||
maxTotalAgents: maxRounds,
|
||||
parent,
|
||||
...exec.signal === undefined ? {} : { signal: exec.signal },
|
||||
})
|
||||
const onAbort = (): void => { run.cancel('parent step aborted') }
|
||||
exec.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
if (exec.signal?.aborted) run.cancel('parent step aborted')
|
||||
|
||||
try {
|
||||
const settled = await run.result
|
||||
const error = stopReasonError(settled)
|
||||
if (error !== undefined) throw new Error(error)
|
||||
const value = readRunResult(settled.value, maxRounds, resolved.maxHandoffChars)
|
||||
if (value.status === 'round-failed') throw new Error(renderRoundFailure(value, resolved.maxResultChars))
|
||||
return [{ type: 'text', text: renderResult(value, resolved.maxResultChars) }]
|
||||
} finally {
|
||||
exec.signal?.removeEventListener('abort', onAbort)
|
||||
await run.dispose()
|
||||
}
|
||||
},
|
||||
presentCall,
|
||||
presentResult,
|
||||
}))
|
||||
}
|
||||
30
packages/workflow/tool-ralph/src/invariant.ts
Normal file
30
packages/workflow/tool-ralph/src/invariant.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-tool-ralph`.
|
||||
* @module @deepseek-ai/dsh-tool-ralph/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-tool-ralph'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'tool-ralph-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this model-facing orchestration adapter owns no independent event stream;
|
||||
* workflow and subagent owners validate the runs and child lifecycles it starts.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
267
packages/workflow/tool-ralph/tests/integration.spec.ts
Normal file
267
packages/workflow/tool-ralph/tests/integration.spec.ts
Normal file
@@ -0,0 +1,267 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
|
||||
import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-inprocess'
|
||||
import * as spawn from '@deepseek-ai/dsh-subagent-spawn'
|
||||
import WorkerWorkflowEngine from '@deepseek-ai/dsh-workflow-workerthread'
|
||||
import { MockAdapter, maxTokensResponse, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
|
||||
import * as toolRalph from '../src/index.ts'
|
||||
|
||||
type MockScript = ConstructorParameters<typeof MockAdapter>[0]
|
||||
|
||||
/** Mount the shipped Ralph execution stack around one keyless model script. */
|
||||
async function mountRalph(script: MockScript, config: toolRalph.Config) {
|
||||
const ctx = new Context()
|
||||
const adapter = new MockAdapter(script)
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(spawn, { providerName: 'spawn' })
|
||||
await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
await ctx.plugin(toolRalph, config)
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
const parentHandle = await ctx.agents.create({
|
||||
sessionId: SessionId('ralph-parent'),
|
||||
meta: { cwd: '/tmp/ralph-shared-workspace' },
|
||||
agentOptions: { provider: 'mock', model: 'mock' },
|
||||
})
|
||||
return { ctx, adapter, parentHandle, parent: parentHandle.agent }
|
||||
}
|
||||
|
||||
describe('dsh-tool-ralph over the real spawn and worker-thread stack', () => {
|
||||
it('uses distinct empty-seed children, shared cwd, and only the prior bounded handoff', async () => {
|
||||
const firstReport = {
|
||||
status: 'continue',
|
||||
summary: 'ROUND_ONE_HANDOFF',
|
||||
evidence: ['Created migration-a.ts.'],
|
||||
nextSteps: ['Finish migration-b.ts.'],
|
||||
blocker: '',
|
||||
}
|
||||
const finalReport = {
|
||||
status: 'complete',
|
||||
summary: 'Both migration slices are complete.',
|
||||
evidence: ['Focused migration tests pass.'],
|
||||
nextSteps: [],
|
||||
blocker: '',
|
||||
}
|
||||
const ctx = new Context()
|
||||
const adapter = new MockAdapter([
|
||||
textResponse('PARENT_HISTORY_MARKER'),
|
||||
toolCallResponse('round-1', STRUCTURED_OUTPUT_TOOL, firstReport),
|
||||
toolCallResponse('round-2', STRUCTURED_OUTPUT_TOOL, finalReport),
|
||||
])
|
||||
await mountAgentLoopTestDependencies(ctx)
|
||||
await ctx.plugin(AgentLoop, { agents: [] })
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(spawn, { providerName: 'spawn' })
|
||||
await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
await ctx.plugin(toolRalph, { maxRounds: 2 })
|
||||
ctx.llm.registerAdapter(['mock'], adapter)
|
||||
|
||||
const parentHandle = await ctx.agents.create({
|
||||
sessionId: SessionId('ralph-parent'),
|
||||
meta: { cwd: '/tmp/ralph-shared-workspace' },
|
||||
agentOptions: { provider: 'mock', model: 'mock' },
|
||||
})
|
||||
const parent = parentHandle.agent
|
||||
parent.send([{ type: 'text', text: 'PARENT_PROMPT_MARKER' }])
|
||||
await parent.whenIdle()
|
||||
|
||||
const children: Agent[] = []
|
||||
const phases: string[] = []
|
||||
ctx.on('workflow/phase', (_run, title) => { phases.push(title) })
|
||||
ctx.on('workflow/agent-start', (_run, child) => {
|
||||
const agent = ctx.agents.get(child.childId)
|
||||
expect(agent).toBeDefined()
|
||||
children.push(agent!)
|
||||
})
|
||||
const result = await ctx.tools.execute({
|
||||
callId: CallId('ralph-integration'),
|
||||
name: 'ralph',
|
||||
arguments: { objective: 'Complete both migration slices.', maxRounds: 2 },
|
||||
agent: parent,
|
||||
})
|
||||
|
||||
expect(result.isError).toBe(false)
|
||||
expect((result.content[0] as { text: string }).text)
|
||||
.toContain('Ralph worker reported completion after 2 rounds.')
|
||||
expect(phases).toEqual(['Fresh-agent rounds'])
|
||||
expect(children).toHaveLength(2)
|
||||
expect(new Set(children.map(child => child.id)).size).toBe(2)
|
||||
for (const child of children) {
|
||||
expect(child.session.header.cwd).toBe('/tmp/ralph-shared-workspace')
|
||||
expect(child.session.header.parentSession).toBe(parent.session.header.id)
|
||||
expect(child.session.header.seedLength).toBeUndefined()
|
||||
expect(ctx.agents.get(child.id)).toBeUndefined()
|
||||
}
|
||||
|
||||
expect(adapter.requests).toHaveLength(3)
|
||||
const firstChildRequest = JSON.stringify(adapter.requests[1]!.messages)
|
||||
const secondChildRequest = JSON.stringify(adapter.requests[2]!.messages)
|
||||
expect(firstChildRequest).not.toContain('PARENT_PROMPT_MARKER')
|
||||
expect(firstChildRequest).not.toContain('PARENT_HISTORY_MARKER')
|
||||
expect(firstChildRequest).not.toContain('ROUND_ONE_HANDOFF')
|
||||
expect(secondChildRequest).not.toContain('PARENT_PROMPT_MARKER')
|
||||
expect(secondChildRequest).not.toContain('PARENT_HISTORY_MARKER')
|
||||
expect(secondChildRequest).toContain('ROUND_ONE_HANDOFF')
|
||||
|
||||
await parentHandle.dispose()
|
||||
})
|
||||
|
||||
it('reports the failed round and last good handoff when a child fails', async () => {
|
||||
const firstReport = {
|
||||
status: 'continue',
|
||||
summary: 'ROUND_ONE_HANDOFF',
|
||||
evidence: ['Created migration-a.ts.'],
|
||||
nextSteps: ['Finish migration-b.ts.'],
|
||||
blocker: '',
|
||||
}
|
||||
const { ctx, parent, parentHandle } = await mountRalph([
|
||||
toolCallResponse('round-1', STRUCTURED_OUTPUT_TOOL, firstReport),
|
||||
maxTokensResponse('unfinished child output'),
|
||||
], { maxRounds: 2 })
|
||||
const children: Agent[] = []
|
||||
ctx.on('workflow/agent-start', (_run, child) => {
|
||||
const agent = ctx.agents.get(child.childId)
|
||||
if (agent !== undefined) children.push(agent)
|
||||
})
|
||||
|
||||
const result = await ctx.tools.execute({
|
||||
callId: CallId('ralph-child-failure'),
|
||||
name: 'ralph',
|
||||
arguments: { objective: 'Complete both migration slices.', maxRounds: 2 },
|
||||
agent: parent,
|
||||
})
|
||||
|
||||
expect(result.isError).toBe(true)
|
||||
const text = (result.content[0] as { text: string }).text
|
||||
expect(text).toContain('Ralph round 2 child failed before producing a structured report.')
|
||||
expect(text).toContain('Last successful handoff:')
|
||||
expect(text).toContain('ROUND_ONE_HANDOFF')
|
||||
expect(children).toHaveLength(2)
|
||||
for (const child of children) expect(ctx.agents.get(child.id)).toBeUndefined()
|
||||
await parentHandle.dispose()
|
||||
})
|
||||
|
||||
it.each([
|
||||
{
|
||||
name: 'blocked',
|
||||
report: {
|
||||
status: 'blocked',
|
||||
summary: 'External authorization is required.',
|
||||
evidence: ['The local implementation is ready.'],
|
||||
nextSteps: ['Continue after authorization.'],
|
||||
blocker: 'The required external authorization is unavailable.',
|
||||
},
|
||||
config: { maxRounds: 2 },
|
||||
expectedError: false,
|
||||
expectedText: 'Ralph worker reported a blocker after 1 round.',
|
||||
},
|
||||
{
|
||||
name: 'budget-limited',
|
||||
report: {
|
||||
status: 'continue',
|
||||
summary: 'One slice is complete.',
|
||||
evidence: ['The first focused test passes.'],
|
||||
nextSteps: ['Implement the remaining slice.'],
|
||||
blocker: '',
|
||||
},
|
||||
config: { maxRounds: 1 },
|
||||
expectedError: false,
|
||||
expectedText: 'Ralph reached its 1 round limit; the worker reported work remaining.',
|
||||
},
|
||||
{
|
||||
name: 'unnormalized report',
|
||||
report: {
|
||||
status: 'continue',
|
||||
summary: ' padded summary ',
|
||||
evidence: ['A focused test passes.'],
|
||||
nextSteps: ['Continue implementation.'],
|
||||
blocker: '',
|
||||
},
|
||||
config: { maxRounds: 1 },
|
||||
expectedError: true,
|
||||
expectedText: 'summary must be non-empty and normalized',
|
||||
},
|
||||
{
|
||||
name: 'invalid continuing report',
|
||||
report: {
|
||||
status: 'continue',
|
||||
summary: 'Work remains.',
|
||||
evidence: ['A focused test passes.'],
|
||||
nextSteps: [],
|
||||
blocker: '',
|
||||
},
|
||||
config: { maxRounds: 1 },
|
||||
expectedError: true,
|
||||
expectedText: 'a continuing Ralph report needs nextSteps and an empty blocker',
|
||||
},
|
||||
{
|
||||
name: 'oversized report',
|
||||
report: {
|
||||
status: 'continue',
|
||||
summary: 'x'.repeat(300),
|
||||
evidence: ['A focused test passes.'],
|
||||
nextSteps: ['Continue implementation.'],
|
||||
blocker: '',
|
||||
},
|
||||
config: { maxRounds: 1, maxHandoffChars: 100 },
|
||||
expectedError: true,
|
||||
expectedText: 'Ralph round report exceeds maxHandoffChars',
|
||||
},
|
||||
])('enforces the fixed script for $name', async ({ report, config, expectedError, expectedText }) => {
|
||||
const { ctx, parent, parentHandle } = await mountRalph([
|
||||
toolCallResponse('round-report', STRUCTURED_OUTPUT_TOOL, report),
|
||||
], config)
|
||||
|
||||
const result = await ctx.tools.execute({
|
||||
callId: CallId('ralph-script-enforcement'),
|
||||
name: 'ralph',
|
||||
arguments: { objective: 'Complete the scoped work.', maxRounds: config.maxRounds },
|
||||
agent: parent,
|
||||
})
|
||||
|
||||
expect(result.isError).toBe(expectedError)
|
||||
expect((result.content[0] as { text: string }).text).toContain(expectedText)
|
||||
await parentHandle.dispose()
|
||||
})
|
||||
|
||||
it('cancels the real worker and fresh child to quiescence', { timeout: 20_000 }, async () => {
|
||||
const { ctx, parent, parentHandle } = await mountRalph(['hang'], { maxRounds: 2 })
|
||||
const children: Agent[] = []
|
||||
const outcomes: string[] = []
|
||||
let resolveChildStarted!: (child: Agent) => void
|
||||
const childStarted = new Promise<Agent>((resolve) => { resolveChildStarted = resolve })
|
||||
ctx.on('workflow/agent-start', (_run, child) => {
|
||||
const agent = ctx.agents.get(child.childId)
|
||||
if (agent !== undefined) {
|
||||
children.push(agent)
|
||||
resolveChildStarted(agent)
|
||||
}
|
||||
})
|
||||
ctx.on('workflow/agent-end', (_run, child) => { outcomes.push(child.outcome) })
|
||||
const controller = new AbortController()
|
||||
const pending = ctx.tools.execute({
|
||||
callId: CallId('ralph-real-cancel'),
|
||||
name: 'ralph',
|
||||
arguments: { objective: 'Keep working until cancelled.', maxRounds: 2 },
|
||||
agent: parent,
|
||||
signal: controller.signal,
|
||||
})
|
||||
await childStarted
|
||||
|
||||
controller.abort()
|
||||
const result = await pending
|
||||
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('Ralph workflow was cancelled')
|
||||
expect(outcomes).toEqual(['cancelled'])
|
||||
expect(ctx.agents.get(children[0]!.id)).toBeUndefined()
|
||||
await parentHandle.dispose()
|
||||
})
|
||||
})
|
||||
380
packages/workflow/tool-ralph/tests/tool-ralph.spec.ts
Normal file
380
packages/workflow/tool-ralph/tests/tool-ralph.spec.ts
Normal file
@@ -0,0 +1,380 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentCapabilities, SubagentProvider, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
|
||||
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
|
||||
import ToolRegistry from '@deepseek-ai/dsh-tools'
|
||||
import type { ToolExecutionResult } from '@deepseek-ai/dsh-tools'
|
||||
import { WorkflowRunId, WorkflowService } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowResult, WorkflowRun, WorkflowStartRequest } from '@deepseek-ai/dsh-workflow'
|
||||
import * as toolRalph from '../src/index.ts'
|
||||
|
||||
class StubEngine extends WorkflowService {
|
||||
requests: WorkflowStartRequest[] = []
|
||||
cancels: string[] = []
|
||||
disposed = 0
|
||||
settle!: (result: WorkflowResult) => void
|
||||
startError: Error | undefined
|
||||
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
if (this.startError !== undefined) throw this.startError
|
||||
this.requests.push(request)
|
||||
const result = new Promise<WorkflowResult>((resolve) => { this.settle = resolve })
|
||||
return {
|
||||
id: WorkflowRunId(`ralph-${this.requests.length}`),
|
||||
meta: request.meta,
|
||||
result,
|
||||
cancel: (reason?: string) => {
|
||||
this.cancels.push(reason ?? 'cancelled')
|
||||
this.settle({
|
||||
value: null,
|
||||
stopReason: 'cancelled',
|
||||
...reason === undefined ? {} : { error: reason },
|
||||
agentsStarted: 0,
|
||||
})
|
||||
},
|
||||
dispose: () => {
|
||||
this.disposed += 1
|
||||
return Promise.resolve()
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class StubProvider implements SubagentProvider {
|
||||
readonly name = 'fresh'
|
||||
readonly capabilities: SubagentCapabilities
|
||||
readonly inheritsParentContext: boolean
|
||||
|
||||
constructor(options?: { outputSchema?: boolean; inheritsParentContext?: boolean }) {
|
||||
this.capabilities = {
|
||||
outputSchema: options?.outputSchema ?? true,
|
||||
depthLimit: true,
|
||||
toolFilter: true,
|
||||
persona: true,
|
||||
}
|
||||
this.inheritsParentContext = options?.inheritsParentContext ?? false
|
||||
}
|
||||
|
||||
start(_request: SubagentStartRequest): Promise<SubagentRun> {
|
||||
return Promise.reject(new Error('StubProvider.start must not be reached behind StubEngine'))
|
||||
}
|
||||
}
|
||||
|
||||
interface SetupOptions {
|
||||
config?: toolRalph.Config
|
||||
provider?: StubProvider | false
|
||||
}
|
||||
|
||||
async function setup(options?: SetupOptions) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(SubagentService)
|
||||
const provider = options?.provider === false ? undefined : options?.provider ?? new StubProvider()
|
||||
if (provider !== undefined) ctx.subagents.registerProvider(provider)
|
||||
await ctx.plugin(StubEngine)
|
||||
const config: toolRalph.Config = { subagentProvider: 'fresh' }
|
||||
if (options?.config?.subagentProvider !== undefined) config.subagentProvider = options.config.subagentProvider
|
||||
if (options?.config?.maxRounds !== undefined) config.maxRounds = options.config.maxRounds
|
||||
if (options?.config?.maxHandoffChars !== undefined) config.maxHandoffChars = options.config.maxHandoffChars
|
||||
if (options?.config?.maxResultChars !== undefined) config.maxResultChars = options.config.maxResultChars
|
||||
const fiber = await ctx.plugin(toolRalph, config)
|
||||
const parent = { id: SessionId('caller'), options: {} } as unknown as Agent
|
||||
return { ctx, engine: ctx.workflows as StubEngine, parent, fiber }
|
||||
}
|
||||
|
||||
function execute(
|
||||
ctx: Context,
|
||||
args: unknown,
|
||||
extra?: { agent?: Agent; signal?: AbortSignal },
|
||||
): Promise<ToolExecutionResult> {
|
||||
return ctx.tools.execute({
|
||||
callId: CallId('ralph-call'),
|
||||
name: 'ralph',
|
||||
arguments: args,
|
||||
...extra?.agent === undefined ? {} : { agent: extra.agent },
|
||||
...extra?.signal === undefined ? {} : { signal: extra.signal },
|
||||
})
|
||||
}
|
||||
|
||||
const CONTINUE = {
|
||||
status: 'continue',
|
||||
summary: 'Implemented the first slice.',
|
||||
evidence: ['Focused tests pass.'],
|
||||
nextSteps: ['Implement the second slice.'],
|
||||
blocker: '',
|
||||
}
|
||||
|
||||
const COMPLETE = {
|
||||
status: 'complete',
|
||||
summary: 'The objective is complete.',
|
||||
evidence: ['All required gates pass.'],
|
||||
nextSteps: [],
|
||||
blocker: '',
|
||||
}
|
||||
|
||||
const BLOCKED = {
|
||||
status: 'blocked',
|
||||
summary: 'No local work can progress.',
|
||||
evidence: ['The required remote service is unavailable.'],
|
||||
nextSteps: ['Retry after service recovery.'],
|
||||
blocker: 'The required remote service is unavailable.',
|
||||
}
|
||||
|
||||
async function settleCompleted(
|
||||
engine: StubEngine,
|
||||
pending: Promise<ToolExecutionResult>,
|
||||
value: unknown,
|
||||
agentsStarted = 1,
|
||||
): Promise<ToolExecutionResult> {
|
||||
await vi.waitFor(() => { expect(engine.requests.length).toBeGreaterThan(0) })
|
||||
engine.settle({ value, stopReason: 'completed', agentsStarted })
|
||||
return pending
|
||||
}
|
||||
|
||||
describe('dsh-tool-ralph', () => {
|
||||
it('starts the fixed workflow through the configured fresh provider and renders completion', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxRounds: 9, maxHandoffChars: 9000 } })
|
||||
const pending = execute(ctx, { objective: ' Finish the migration. ', maxRounds: 4 }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(1) })
|
||||
expect(engine.requests[0]).toMatchObject({
|
||||
meta: { name: 'ralph-loop' },
|
||||
args: { objective: 'Finish the migration.', maxRounds: 4, maxHandoffChars: 9000 },
|
||||
subagentProvider: 'fresh',
|
||||
maxTotalAgents: 4,
|
||||
parent,
|
||||
})
|
||||
expect(engine.requests[0]!.script).toContain("status: 'budget-limited'")
|
||||
const result = await settleCompleted(engine, pending, {
|
||||
status: 'complete',
|
||||
roundsStarted: 1,
|
||||
report: COMPLETE,
|
||||
})
|
||||
expect(result.isError).toBe(false)
|
||||
expect((result.content[0] as { text: string }).text)
|
||||
.toContain('Ralph worker reported completion after 1 round.')
|
||||
expect((result.content[0] as { text: string }).text).toContain('All required gates pass.')
|
||||
expect(engine.disposed).toBe(1)
|
||||
})
|
||||
|
||||
it('renders blocked and budget-limited terminal outcomes as bounded successful results', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxRounds: 2 } })
|
||||
const blocked = execute(ctx, { objective: 'Ship it.' }, { agent: parent })
|
||||
const blockedResult = await settleCompleted(engine, blocked, {
|
||||
status: 'blocked',
|
||||
roundsStarted: 2,
|
||||
report: BLOCKED,
|
||||
}, 2)
|
||||
expect((blockedResult.content[0] as { text: string }).text)
|
||||
.toContain('Ralph worker reported a blocker after 2 rounds.')
|
||||
|
||||
const limited = execute(ctx, { objective: 'Ship it.' }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(2) })
|
||||
const limitedResult = await settleCompleted(engine, limited, {
|
||||
status: 'budget-limited',
|
||||
roundsStarted: 2,
|
||||
report: CONTINUE,
|
||||
}, 2)
|
||||
expect((limitedResult.content[0] as { text: string }).text)
|
||||
.toContain('Ralph reached its 2 rounds limit; the worker reported work remaining.')
|
||||
})
|
||||
|
||||
it('bounds the complete parent result and labels worker-reported completion', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxResultChars: 160 } })
|
||||
const pending = execute(ctx, { objective: 'Ship it.' }, { agent: parent })
|
||||
const result = await settleCompleted(engine, pending, {
|
||||
status: 'complete',
|
||||
roundsStarted: 1,
|
||||
report: { ...COMPLETE, evidence: ['x'.repeat(500)] },
|
||||
})
|
||||
const text = (result.content[0] as { text: string }).text
|
||||
expect(text).toHaveLength(160)
|
||||
expect(text).toContain('Ralph worker reported completion after 1 round.')
|
||||
expect(text).toMatch(/… \[truncated\]$/)
|
||||
})
|
||||
|
||||
it('honors a result limit shorter than the truncation marker', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxResultChars: 5 } })
|
||||
const result = await settleCompleted(engine, execute(ctx, { objective: 'Ship it.' }, { agent: parent }), {
|
||||
status: 'complete',
|
||||
roundsStarted: 1,
|
||||
report: COMPLETE,
|
||||
})
|
||||
expect((result.content[0] as { text: string }).text).toBe('\n… [t')
|
||||
})
|
||||
|
||||
it('reports an ordinary child failure with the failed round and last durable handoff', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxRounds: 2 } })
|
||||
const first = execute(ctx, { objective: 'Ship it.', maxRounds: 2 }, { agent: parent })
|
||||
const firstResult = await settleCompleted(engine, first, {
|
||||
status: 'round-failed',
|
||||
roundsStarted: 1,
|
||||
lastReport: null,
|
||||
})
|
||||
expect(firstResult.isError).toBe(true)
|
||||
expect((firstResult.content[0] as { text: string }).text).toContain('Ralph round 1 child failed')
|
||||
expect((firstResult.content[0] as { text: string }).text).toContain('No previous handoff was available.')
|
||||
|
||||
const later = execute(ctx, { objective: 'Ship it.', maxRounds: 2 }, { agent: parent })
|
||||
const laterResult = await settleCompleted(engine, later, {
|
||||
status: 'round-failed',
|
||||
roundsStarted: 2,
|
||||
lastReport: CONTINUE,
|
||||
})
|
||||
expect(laterResult.isError).toBe(true)
|
||||
expect((laterResult.content[0] as { text: string }).text).toContain('Ralph round 2 child failed')
|
||||
expect((laterResult.content[0] as { text: string }).text).toContain('Implemented the first slice.')
|
||||
})
|
||||
|
||||
it('maps workflow error and cancellation reasons to tool errors and always disposes', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const failed = execute(ctx, { objective: 'Work.' }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(1) })
|
||||
engine.settle({ value: null, stopReason: 'error', error: 'child report malformed', agentsStarted: 1 })
|
||||
expect(((await failed).content[0] as { text: string }).text)
|
||||
.toContain('Ralph workflow failed: child report malformed')
|
||||
|
||||
const unknown = execute(ctx, { objective: 'Work.' }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(2) })
|
||||
engine.settle({ value: null, stopReason: 'error', agentsStarted: 0 })
|
||||
expect(((await unknown).content[0] as { text: string }).text).toContain('unknown error')
|
||||
|
||||
const cancelled = execute(ctx, { objective: 'Work.' }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(3) })
|
||||
engine.settle({ value: null, stopReason: 'cancelled', error: 'user stopped', agentsStarted: 0 })
|
||||
expect(((await cancelled).content[0] as { text: string }).text).toContain('cancelled (user stopped)')
|
||||
|
||||
const bare = execute(ctx, { objective: 'Work.' }, { agent: parent })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(4) })
|
||||
engine.settle({ value: null, stopReason: 'cancelled', agentsStarted: 0 })
|
||||
expect(((await bare).content[0] as { text: string }).text).toMatch(/cancelled$/)
|
||||
expect(engine.disposed).toBe(4)
|
||||
})
|
||||
|
||||
it('bridges mid-flight and already-aborted parent signals to cancellation', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
const controller = new AbortController()
|
||||
const pending = execute(ctx, { objective: 'Work.' }, { agent: parent, signal: controller.signal })
|
||||
await vi.waitFor(() => { expect(engine.requests).toHaveLength(1) })
|
||||
controller.abort()
|
||||
expect((await pending).isError).toBe(true)
|
||||
|
||||
const already = new AbortController()
|
||||
already.abort()
|
||||
expect((await execute(ctx, { objective: 'Work.' }, { agent: parent, signal: already.signal })).isError).toBe(true)
|
||||
expect(engine.cancels).toEqual(['parent step aborted', 'parent step aborted'])
|
||||
expect(engine.disposed).toBe(2)
|
||||
})
|
||||
|
||||
it('rejects absent authority, empty objectives, bad round caps, and schema-invalid calls before start', async () => {
|
||||
const { ctx, engine, parent } = await setup({ config: { maxRounds: 3 } })
|
||||
expect((await execute(ctx, { objective: 'Work.' })).isError).toBe(true)
|
||||
expect((await execute(ctx, { objective: ' ' }, { agent: parent })).isError).toBe(true)
|
||||
for (const maxRounds of [0, 1.5, Number.NaN, 4]) {
|
||||
expect((await execute(ctx, { objective: 'Work.', maxRounds }, { agent: parent })).isError).toBe(true)
|
||||
}
|
||||
const missing = await execute(ctx, {}, { agent: parent })
|
||||
expect(missing.error?.code).toBe('INVALID_ARGS')
|
||||
expect(engine.requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('rejects missing, unstructured, and parent-context-inheriting provider routes', async () => {
|
||||
const missing = await setup({ provider: false })
|
||||
expect(((await execute(missing.ctx, { objective: 'Work.' }, { agent: missing.parent })).content[0] as { text: string }).text)
|
||||
.toContain('is not registered')
|
||||
expect(missing.engine.requests).toHaveLength(0)
|
||||
|
||||
const unstructured = await setup({ provider: new StubProvider({ outputSchema: false }) })
|
||||
expect(((await execute(unstructured.ctx, { objective: 'Work.' }, { agent: unstructured.parent })).content[0] as { text: string }).text)
|
||||
.toContain('does not support structured output')
|
||||
|
||||
const inherited = await setup({ provider: new StubProvider({ inheritsParentContext: true }) })
|
||||
expect(((await execute(inherited.ctx, { objective: 'Work.' }, { agent: inherited.parent })).content[0] as { text: string }).text)
|
||||
.toContain('inherits parent context')
|
||||
})
|
||||
|
||||
it('rejects invalid direct-apply config before touching injected services', () => {
|
||||
expect(() => { toolRalph.apply(new Context(), { subagentProvider: ' ' }) }).toThrow('non-empty normalized')
|
||||
expect(() => { toolRalph.apply(new Context(), { maxRounds: 0 }) }).toThrow('positive safe integer')
|
||||
expect(() => { toolRalph.apply(new Context(), { maxHandoffChars: 1.5 }) }).toThrow('positive safe integer')
|
||||
expect(() => { toolRalph.apply(new Context(), { maxResultChars: 0 }) }).toThrow('positive safe integer')
|
||||
})
|
||||
|
||||
it('turns malformed fixed-workflow terminal values and reports into errors', async () => {
|
||||
const cases: { value: unknown; message: string; config?: toolRalph.Config }[] = [
|
||||
{ value: null, message: 'malformed terminal result' },
|
||||
{ value: { status: 'complete', roundsStarted: 0, report: COMPLETE }, message: 'malformed terminal result' },
|
||||
{ value: { status: 'complete', roundsStarted: 3, report: COMPLETE }, message: 'malformed terminal result', config: { maxRounds: 2 } },
|
||||
{ value: { status: 'mystery', roundsStarted: 1, report: COMPLETE }, message: 'unknown terminal status' },
|
||||
{ value: { status: 'budget-limited', roundsStarted: 1, report: CONTINUE }, message: 'before the round limit', config: { maxRounds: 2 } },
|
||||
{ value: { status: 'complete', roundsStarted: 1, report: null }, message: 'malformed round report' },
|
||||
{ value: { status: 'complete', roundsStarted: 1, report: COMPLETE, extra: true }, message: 'malformed terminal result' },
|
||||
{ value: { status: 'blocked', roundsStarted: 1, report: BLOCKED, extra: true }, message: 'malformed terminal result' },
|
||||
{ value: { status: 'budget-limited', roundsStarted: 1, report: CONTINUE, extra: true }, message: 'malformed terminal result', config: { maxRounds: 1 } },
|
||||
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, status: 'continue' } }, message: 'malformed round report' },
|
||||
{ value: { status: 'budget-limited', roundsStarted: 1, report: { ...CONTINUE, nextSteps: [] } }, message: 'invalid continuing report', config: { maxRounds: 1 } },
|
||||
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, evidence: [] } }, message: 'invalid completion report' },
|
||||
{ value: { status: 'blocked', roundsStarted: 1, report: { ...BLOCKED, blocker: '' } }, message: 'invalid blocked report' },
|
||||
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, summary: 'x'.repeat(500) } }, message: 'oversized handoff', config: { maxHandoffChars: 100 } },
|
||||
{ value: { status: 'round-failed', roundsStarted: 1 }, message: 'malformed terminal result' },
|
||||
{ value: { status: 'round-failed', roundsStarted: 1, lastReport: CONTINUE }, message: 'invalid first-round failure' },
|
||||
{ value: { status: 'round-failed', roundsStarted: 2, lastReport: null }, message: 'without its last handoff', config: { maxRounds: 2 } },
|
||||
{ value: { status: 'round-failed', roundsStarted: 2, lastReport: { ...CONTINUE, nextSteps: [] } }, message: 'invalid continuing report', config: { maxRounds: 2 } },
|
||||
]
|
||||
for (const testCase of cases) {
|
||||
const { ctx, engine, parent } = await setup(
|
||||
testCase.config === undefined ? undefined : { config: testCase.config },
|
||||
)
|
||||
const result = await settleCompleted(
|
||||
engine,
|
||||
execute(ctx, { objective: 'Work.', ...testCase.config?.maxRounds === undefined ? {} : { maxRounds: testCase.config.maxRounds } }, { agent: parent }),
|
||||
testCase.value,
|
||||
)
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain(testCase.message)
|
||||
}
|
||||
})
|
||||
|
||||
it('surfaces a synchronous engine start failure without inventing a run', async () => {
|
||||
const { ctx, engine, parent } = await setup()
|
||||
engine.startError = new Error('engine refused fixed script')
|
||||
const result = await execute(ctx, { objective: 'Work.' }, { agent: parent })
|
||||
expect(result.isError).toBe(true)
|
||||
expect((result.content[0] as { text: string }).text).toContain('engine refused fixed script')
|
||||
expect(engine.disposed).toBe(0)
|
||||
})
|
||||
|
||||
it('registers scoped guidance and pure replay-safe generic presentation', async () => {
|
||||
const { ctx, fiber } = await setup()
|
||||
const section = (await ctx.systemPrompt.assemble()).sections.find(candidate => candidate.name === 'tool:ralph')
|
||||
expect(section?.text).toContain('ONLY when the direct human explicitly asks')
|
||||
expect(section?.text).toContain('worker reports, not independent evaluation')
|
||||
const tool = ctx.tools.get('ralph')!
|
||||
expect(tool.description).toContain('worker reports completion')
|
||||
expect(tool.presentCall!({ objective: 'Finish it.' })).toEqual({
|
||||
card: 'generic',
|
||||
title: 'ralph',
|
||||
rawInput: 'Finish it.',
|
||||
})
|
||||
expect(tool.presentResult!({ objective: 'Finish it.' }, { content: [], isError: false })).toEqual({ card: 'generic' })
|
||||
expect(tool.presentCall!({ nope: true })).toBeUndefined()
|
||||
await fiber.dispose()
|
||||
expect(ctx.tools.get('ralph')).toBeUndefined()
|
||||
expect((await ctx.systemPrompt.assemble()).sections.some(candidate => candidate.name === 'tool:ralph')).toBe(false)
|
||||
})
|
||||
|
||||
it('has the namespace-plugin export shape', () => {
|
||||
expect('default' in toolRalph).toBe(false)
|
||||
expect(toolRalph.name).toBe('tool-ralph')
|
||||
expect(toolRalph.inject).toEqual(['tools', 'workflows', 'subagents', 'systemPrompt'])
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(toolRalph) as Record<string, unknown>
|
||||
expect(unwrapped).toBe(toolRalph)
|
||||
expect(typeof unwrapped.apply).toBe('function')
|
||||
})
|
||||
})
|
||||
42
packages/workflow/tool-ralph/tsconfig.json
Normal file
42
packages/workflow/tool-ralph/tsconfig.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/agent"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../../subagent/subagent"
|
||||
},
|
||||
{
|
||||
"path": "../../core/system-prompt"
|
||||
},
|
||||
{
|
||||
"path": "../../core/tools"
|
||||
},
|
||||
{
|
||||
"path": "../workflow"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -230,6 +230,12 @@ describe('dsh-tool-workflow', () => {
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry)
|
||||
await ctx.plugin(SubagentService)
|
||||
ctx.subagents.registerProvider({
|
||||
name: 'spawn',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => Promise.reject(new Error('the parked-script fixture must not start a child')),
|
||||
})
|
||||
await ctx.plugin(WorkerWorkflowEngine, { disposeGraceMs: 30 })
|
||||
await ctx.plugin(toolWorkflow, {})
|
||||
const parent = { id: SessionId('caller'), options: {} } as unknown as Agent
|
||||
|
||||
@@ -34,12 +34,12 @@ Unknown options, malformed arguments, unsupported schemas, tripped caps, provide
|
||||
|
||||
## Run sequence
|
||||
|
||||
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
`start()` validates meta, parses the body, resolves a registered normalized provider route, and resolves any per-run total-child cap before creating a worker or publishing `workflow/start`. A requested `maxTotalAgents` must be a positive safe integer no greater than the engine's configured deployment ceiling. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
|
||||
|
||||
For each `agent()` call:
|
||||
|
||||
1. The worker sends `child-start` with a plain-data prompt and options.
|
||||
2. The host calls the configured provider through async `SubagentService.start`, passing the workflow's parent and one canonical per-run abort signal.
|
||||
2. The host calls the start request's provider override, or otherwise the configured provider, through async `SubagentService.start`, passing the workflow's parent and one canonical per-run abort signal. Provider choice applies to every child in that run and is not visible to the script.
|
||||
3. If start rejects, the host sends `child-start-error`; provider startup has already reached quiescence and no child lifecycle event is emitted.
|
||||
4. If start fulfills while the workflow still admits work, the host records the run, observes `result`, then sends `child-started`. Even an already-settled result is forwarded afterward, preserving start-before-result order.
|
||||
5. The worker emits paired `workflow/agent-start` and `workflow/agent-end` narration and requests child disposal after collection.
|
||||
@@ -81,6 +81,8 @@ The host keeps a ledger of forwarded child starts. A graceful worker supplies th
|
||||
| `syncTimeoutMs` | `5000` | VM timeout for the script's initial synchronous slice. |
|
||||
| `disposeGraceMs` | `5000` | Bound before force-settlement/termination and for public disposal. |
|
||||
|
||||
An owning consumer may set `WorkflowStartRequest.subagentProvider` and `WorkflowStartRequest.maxTotalAgents` for one run. These are engine-level policy, not script hooks or model-facing options; the ordinary `workflow` tool leaves both unset. A per-run total-child cap may lower but never raise the configured `maxTotalAgents` ceiling.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Child-agent requests
|
||||
|
||||
@@ -73,6 +73,36 @@ function assertBodyParses(body: string, name: string): void {
|
||||
}
|
||||
}
|
||||
|
||||
/** Resolve one run's provider route before publishing work. */
|
||||
function resolveSubagentProvider(ctx: Context, configured: string, override: string | undefined): string {
|
||||
const provider = override ?? configured
|
||||
if (provider.length === 0 || provider !== provider.trim()) {
|
||||
throw new WorkflowError(
|
||||
'workflow subagentProvider must be a non-empty normalized string',
|
||||
'INVALID_ARGUMENT',
|
||||
)
|
||||
}
|
||||
if (ctx.subagents.getProvider(provider) === undefined) {
|
||||
throw new WorkflowError(`no subagent provider registered for "${provider}"`, 'AGENT_START')
|
||||
}
|
||||
return provider
|
||||
}
|
||||
|
||||
/** Resolve one run's total-child cap against the engine deployment ceiling. */
|
||||
function resolveMaxTotalAgents(requested: number | undefined, ceiling: number): number {
|
||||
if (requested === undefined) return ceiling
|
||||
if (!Number.isSafeInteger(requested) || requested < 1) {
|
||||
throw new WorkflowError('workflow maxTotalAgents must be a positive safe integer', 'INVALID_ARGUMENT')
|
||||
}
|
||||
if (requested > ceiling) {
|
||||
throw new WorkflowError(
|
||||
`workflow maxTotalAgents ${requested} exceeds the engine ceiling ${ceiling}`,
|
||||
'INVALID_ARGUMENT',
|
||||
)
|
||||
}
|
||||
return requested
|
||||
}
|
||||
|
||||
/**
|
||||
* The worker-thread engine service. `start()` validates the script up front
|
||||
* (meta + a host-side body parse) and returns a {@link WorkflowRun} whose
|
||||
@@ -113,13 +143,15 @@ class WorkerWorkflowEngine extends WorkflowService {
|
||||
start(request: WorkflowStartRequest): WorkflowRun {
|
||||
const meta = validateMeta(request.meta)
|
||||
assertBodyParses(request.script, meta.name)
|
||||
const subagentProvider = resolveSubagentProvider(this.ctx, this.config.provider, request.subagentProvider)
|
||||
const maxTotalAgents = resolveMaxTotalAgents(request.maxTotalAgents, this.config.maxTotalAgents)
|
||||
const id = WorkflowRunId(randomUUID())
|
||||
const info: WorkflowRunInfo = { id, meta }
|
||||
const limits: WorkerLimits = {
|
||||
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
|
||||
? Math.min(16, Math.max(1, availableParallelism() - 2))
|
||||
: this.config.maxConcurrentAgents,
|
||||
maxTotalAgents: this.config.maxTotalAgents,
|
||||
maxTotalAgents,
|
||||
maxItemsPerCall: this.config.maxItemsPerCall,
|
||||
syncTimeoutMs: this.config.syncTimeoutMs,
|
||||
}
|
||||
@@ -144,7 +176,7 @@ class WorkerWorkflowEngine extends WorkflowService {
|
||||
meta,
|
||||
request.parent,
|
||||
init,
|
||||
this.config.provider,
|
||||
subagentProvider,
|
||||
this.config.disposeGraceMs,
|
||||
{
|
||||
phase: (title) => { this.emitWorkflowEvent('workflow/phase', info, title) },
|
||||
|
||||
@@ -255,7 +255,7 @@ export class WorkflowExecution {
|
||||
const opts = this.readAgentOptions(rawOpts)
|
||||
if (this.started >= this.limits.maxTotalAgents) {
|
||||
throw new WorkflowError(
|
||||
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise maxTotalAgents in the engine config if the scale is intentional`,
|
||||
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise the applicable maxTotalAgents limit if the scale is intentional`,
|
||||
'AGENT_CAP',
|
||||
)
|
||||
}
|
||||
|
||||
@@ -27,16 +27,30 @@ import WorkerWorkflowEngine from '@deepseek-ai/dsh-workflow-workerthread'
|
||||
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SubagentService)
|
||||
await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
let selectedStarts = 0
|
||||
ctx.subagents.registerProvider({
|
||||
name: 'built-selected',
|
||||
capabilities: { outputSchema: true, depthLimit: false, toolFilter: false, persona: false },
|
||||
inheritsParentContext: false,
|
||||
async start() {
|
||||
selectedStarts += 1
|
||||
return {
|
||||
id: 'built-child',
|
||||
result: Promise.resolve({ output: [], structured: { answer: 42 }, stopReason: 'completed' }),
|
||||
dispose: () => Promise.resolve(),
|
||||
}
|
||||
},
|
||||
})
|
||||
await ctx.plugin(WorkerWorkflowEngine, { provider: 'must-not-be-used' })
|
||||
const run = ctx.workflows.start({
|
||||
script: 'return 6 * 7',
|
||||
script: "const value = await agent('answer', { schema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] } }); return value.answer",
|
||||
meta: { name: 'built-smoke', description: 'built worker smoke' },
|
||||
// A zero-agent script never touches the provider.
|
||||
subagentProvider: 'built-selected',
|
||||
parent: { id: 'built-smoke-parent', options: {} },
|
||||
})
|
||||
const result = await run.result
|
||||
await run.dispose()
|
||||
if (result.stopReason !== 'completed' || result.value !== 42) {
|
||||
if (result.stopReason !== 'completed' || result.value !== 42 || selectedStarts !== 1) {
|
||||
console.error('unexpected result: ' + JSON.stringify(result))
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
@@ -392,6 +392,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
|
||||
const result = await host.result()
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('total agent cap (2)')
|
||||
expect(result.error).toContain('applicable maxTotalAgents limit')
|
||||
expect(result.agentsStarted).toBe(2)
|
||||
host.close()
|
||||
})
|
||||
|
||||
@@ -8,6 +8,7 @@ import { expect, it, vi } from 'vitest'
|
||||
import { Context } from 'cordis'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentProvider } from '@deepseek-ai/dsh-subagent'
|
||||
import WorkerWorkflowEngine from '../src/index.ts'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
@@ -18,6 +19,13 @@ vi.setConfig({ testTimeout: 30_000 })
|
||||
it('runs the default config through the source worker', async () => {
|
||||
const ctx = new Context()
|
||||
const subagents = await ctx.plugin(SubagentService)
|
||||
const provider: SubagentProvider = {
|
||||
name: 'spawn',
|
||||
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: true },
|
||||
inheritsParentContext: false,
|
||||
start: () => Promise.reject(new Error('source-worker compat script must not start a child')),
|
||||
}
|
||||
ctx.subagents.registerProvider(provider)
|
||||
const engine = await ctx.plugin(WorkerWorkflowEngine, {})
|
||||
const parent = { id: SessionId('workflow-compat-parent'), options: {} } as unknown as Agent
|
||||
try {
|
||||
|
||||
@@ -6,7 +6,7 @@ import Loader from '@cordisjs/plugin-loader'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
import SubagentService from '@deepseek-ai/dsh-subagent'
|
||||
import type { SubagentCapabilities, SubagentProvider, SubagentResult, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
|
||||
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
|
||||
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRun, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
|
||||
import * as workerEngineModule from '../src/index.ts'
|
||||
import WorkerWorkflowEngine, { type Config } from '../src/index.ts'
|
||||
import { HostToWorkerType, WorkerToHostType } from '../src/protocol.ts'
|
||||
@@ -232,6 +232,97 @@ describe('dsh-workflow-workerthread', () => {
|
||||
expect(provider.runs[0]!.request.agentOptions).toEqual({ provider: 'openai' })
|
||||
})
|
||||
|
||||
it('a start-request provider override selects every child without changing the engine default', async () => {
|
||||
const { ctx, parent, provider } = await setup()
|
||||
const selected = new StubProvider('selected', () => text('selected reply'))
|
||||
ctx.subagents.registerProvider(selected)
|
||||
|
||||
const overridden = ctx.workflows.start({
|
||||
...scripted("return await agent('route this run')"),
|
||||
parent,
|
||||
subagentProvider: 'selected',
|
||||
})
|
||||
expect((await overridden.result).value).toBe('selected reply')
|
||||
await overridden.dispose()
|
||||
expect(selected.runs).toHaveLength(1)
|
||||
expect(provider.runs).toHaveLength(0)
|
||||
|
||||
const ordinary = await run(ctx, parent, scripted("return await agent('use the default')"))
|
||||
expect(ordinary.value).toBe('stub reply')
|
||||
expect(provider.runs).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('rejects invalid start-request provider routes before publishing a run', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
let starts = 0
|
||||
ctx.on('workflow/start', () => { starts += 1 })
|
||||
const messages: string[] = []
|
||||
for (const subagentProvider of ['', 'missing']) {
|
||||
let run: WorkflowRun | undefined
|
||||
let thrown: unknown
|
||||
try {
|
||||
run = ctx.workflows.start({
|
||||
...scripted("return 'must not start'"),
|
||||
parent,
|
||||
subagentProvider,
|
||||
})
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
await run?.dispose()
|
||||
messages.push(thrown instanceof Error ? thrown.message : '')
|
||||
}
|
||||
|
||||
expect(messages).toEqual([
|
||||
'workflow subagentProvider must be a non-empty normalized string',
|
||||
'no subagent provider registered for "missing"',
|
||||
])
|
||||
expect(starts).toBe(0)
|
||||
})
|
||||
|
||||
it('rejects invalid per-run total-agent caps before publishing a run', async () => {
|
||||
const { ctx, parent } = await setup({ config: { maxTotalAgents: 2 } })
|
||||
let starts = 0
|
||||
ctx.on('workflow/start', () => { starts += 1 })
|
||||
const errors: unknown[] = []
|
||||
for (const maxTotalAgents of [0, 1.5, Number.NaN, 3]) {
|
||||
try {
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted("return 'must not start'"),
|
||||
parent,
|
||||
maxTotalAgents,
|
||||
})
|
||||
await handle.dispose()
|
||||
} catch (error: unknown) {
|
||||
errors.push(error)
|
||||
}
|
||||
}
|
||||
|
||||
expect(errors.slice(0, 3)).toEqual(Array(3).fill(expect.objectContaining({
|
||||
code: 'INVALID_ARGUMENT',
|
||||
message: 'workflow maxTotalAgents must be a positive safe integer',
|
||||
})))
|
||||
expect(errors[3]).toMatchObject({
|
||||
code: 'INVALID_ARGUMENT',
|
||||
message: 'workflow maxTotalAgents 3 exceeds the engine ceiling 2',
|
||||
})
|
||||
expect(starts).toBe(0)
|
||||
})
|
||||
|
||||
it('enforces a per-run total-agent cap below the engine ceiling', async () => {
|
||||
const { ctx, parent } = await setup({ config: { maxTotalAgents: 2 } })
|
||||
const handle = ctx.workflows.start({
|
||||
...scripted("await agent('first'); await agent('second'); return 'unreachable'"),
|
||||
parent,
|
||||
maxTotalAgents: 1,
|
||||
})
|
||||
const result = await handle.result
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.agentsStarted).toBe(1)
|
||||
expect(result.error).toContain('total agent cap (1)')
|
||||
await handle.dispose()
|
||||
})
|
||||
|
||||
it('a fatal hook error inside the worker kills the script and reports the error', async () => {
|
||||
const { ctx, parent } = await setup()
|
||||
const result = await run(ctx, parent, scripted("return await parallel([() => agent('x', { isolation: 'worktree' })])"))
|
||||
@@ -239,11 +330,18 @@ describe('dsh-workflow-workerthread', () => {
|
||||
expect(result.error).toContain('"isolation" is deferred')
|
||||
})
|
||||
|
||||
it('a provider start failure crosses back as a fatal AGENT_START error', async () => {
|
||||
it('rejects an unregistered configured provider before publishing a run', async () => {
|
||||
const { ctx, parent } = await setup({ config: { provider: 'nonexistent' } })
|
||||
const result = await run(ctx, parent, scripted("return await pipeline([1], () => agent('p'))"))
|
||||
expect(result.stopReason).toBe('error')
|
||||
expect(result.error).toContain('agent() could not start a child')
|
||||
let thrown: unknown
|
||||
try {
|
||||
ctx.workflows.start({ ...scripted("return 'must not start'"), parent })
|
||||
} catch (error: unknown) {
|
||||
thrown = error
|
||||
}
|
||||
expect(thrown).toMatchObject({
|
||||
code: 'AGENT_START',
|
||||
message: 'no subagent provider registered for "nonexistent"',
|
||||
})
|
||||
})
|
||||
|
||||
it('waits for async provider start before announcing a result that settled early', async () => {
|
||||
|
||||
@@ -6,11 +6,11 @@ The workflow seam (`ctx.workflows`) executes a model-written orchestration scrip
|
||||
|
||||
## Service and run contract
|
||||
|
||||
`WorkflowService.start(request): WorkflowRun` validates enough synchronously to reject a malformed meta block or unparseable script before a run exists. Once returned, `WorkflowRun.result` never rejects: execution failures resolve with `stopReason: 'error'`, and cancellation resolves with `cancelled` within the engine's bounded grace.
|
||||
`WorkflowService.start(request): WorkflowRun` validates enough synchronously to reject a malformed meta block, unparseable script, unavailable provider route, or unsupported per-run limit before a run exists. Once returned, `WorkflowRun.result` never rejects: execution failures resolve with `stopReason: 'error'`, and cancellation resolves with `cancelled` within the engine's bounded grace.
|
||||
|
||||
A run is holder-owned. Engine-plugin unload prevents new starts but does not revoke accepted runs. The holder must call `dispose()` on every path; disposal cancels remaining work and reaches or abandons quiescence within the documented bound.
|
||||
|
||||
`WorkflowStartRequest` contains `{ meta, script, args?, parent, signal? }`. `parent` attributes every child agent to the invoking agent. `meta` and `args` are plain data, not script fragments.
|
||||
`WorkflowStartRequest` contains `{ meta, script, args?, subagentProvider?, maxTotalAgents?, parent, signal? }`. `parent` attributes every child agent to the invoking agent. `subagentProvider` optionally routes every child in that run without exposing provider choice to the script; omission uses the engine's configured provider. `maxTotalAgents` optionally lowers the engine's deployment ceiling for one run and is likewise invisible to the script. An implementation rejects invalid routes and limits synchronously. `meta` and `args` are plain data, not script fragments.
|
||||
|
||||
`WorkflowRun` exposes `{ id, meta, result, cancel(reason?), dispose() }`. `WorkflowResult` contains `{ value, stopReason, error?, agentsStarted }`; `value` is plain JSON data or `null`.
|
||||
|
||||
|
||||
@@ -70,6 +70,17 @@ export interface WorkflowStartRequest {
|
||||
meta: WorkflowMeta
|
||||
/** Optional input exposed verbatim to the script as the `args` global. */
|
||||
args?: unknown
|
||||
/**
|
||||
* Optional engine-wide child-provider override for this run. The workflow
|
||||
* script cannot observe or replace it; omission uses the engine's configured
|
||||
* provider.
|
||||
*/
|
||||
subagentProvider?: string
|
||||
/**
|
||||
* Optional per-run total-child ceiling. Implementations reject values above
|
||||
* their deployment ceiling before publishing the run.
|
||||
*/
|
||||
maxTotalAgents?: number
|
||||
/** The agent on whose behalf the run executes (parent of every child). */
|
||||
parent: Agent
|
||||
/** Cancels the run when aborted (the tool's `exec.signal`). */
|
||||
|
||||
Reference in New Issue
Block a user