fix(ralph): harden execution boundaries

This commit is contained in:
Tianyi Cui
2026-07-20 12:56:09 +08:00
parent c6f8247ac1
commit 60eea35df5
37 changed files with 535 additions and 81 deletions

View File

@@ -4,11 +4,13 @@ The model-facing `ralph` tool runs a fixed foreground workflow that gives one im
## Contract
`ralph({ objective, maxRounds? })` waits for the entire run. The deployment config's `maxRounds` is both the default and a ceiling on a call override. Every Ralph round starts one child through `subagentProvider`; that provider must exist, support structured output, and report `inheritsParentContext: false`. The configured provider is carried as `WorkflowStartRequest.subagentProvider`, so the fixed script cannot inspect or change routing and the ordinary model-written `workflow` tool gains no provider selector.
`ralph({ objective, maxRounds? })` waits for the entire run. The deployment config's `maxRounds` is both the default and a ceiling on a call override. Every Ralph round starts one child through `subagentProvider`; that provider must exist, support structured output, and report `inheritsParentContext: false`. The configured provider is carried as `WorkflowStartRequest.subagentProvider`, so the fixed script cannot inspect or change routing and the ordinary model-written `workflow` tool gains no provider selector. The resolved round cap is also carried as `WorkflowStartRequest.maxTotalAgents`, coordinating the fixed loop with the engine's total-child backstop; the engine rejects a Ralph cap above its deployment ceiling before publishing a run.
Each child receives only the immutable objective, its current Ralph round and cap, a shared-workspace-as-authority instruction, and the previous structured handoff. The workspace is long-term memory; parent conversation and prior child sessions are not seeded. Reports have `status: continue | complete | blocked`, a non-empty summary, evidence, next steps, and blocker text. Status-specific semantics and the serialized `maxHandoffChars` ceiling are validated inside the fixed workflow and again at the consumer boundary. Invalid, missing, or oversized reports fail the workflow instead of being truncated or mistaken for cap exhaustion.
The terminal tool result is `complete`, `blocked`, or `budget-limited`, with the last bounded report and number of rounds started. Child self-declaration determines completion in this cut. A workflow failure or cancellation is an error result; partial output is never success.
The successful terminal tool result is `complete`, `blocked`, or `budget-limited`, with the last bounded report and number of rounds started. Completion and blocker labels explicitly say that a worker reported the outcome; they are not independent certification. `maxResultChars` bounds the complete successful text including its envelope and truncation marker, without altering the validated report used as a cross-round handoff.
An ordinary child failure produces an error naming the failed round and retaining the last successful handoff when one exists. Ralph does not retry that round. Fatal provider-start, transport, worker, or workflow failures remain workflow errors and may settle before the fixed script can return a handoff. Cancellation is also an error; partial output is never success.
## Lifecycle and cancellation
@@ -25,6 +27,7 @@ The pending call is a `generic` card titled `ralph`; the immutable objective is
| `subagentProvider` | `spawn` | Fresh structured-output provider used for every round. |
| `maxRounds` | `256` | Default and deployment ceiling for one Ralph run. |
| `maxHandoffChars` | `16384` | Maximum serialized characters in one round report. |
| `maxResultChars` | `16384` | Maximum characters in the complete successful parent result. |
All config values are normalized and validated when the plugin applies, including direct application outside Loader schema normalization. Provider capabilities are resolved immediately before each call because provider registration can change under plugin lifecycle and HMR.
@@ -39,7 +42,7 @@ Every parent request in this plugin's registration scope receives the fixed rout
##### Ralph guidance
```markdown
Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.
Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Completion and blockers are worker reports, not independent evaluation. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.
```
#### Token effect
@@ -68,11 +71,11 @@ Prefix-stable while the definition and visibility are unchanged.
#### What the model sees
Each child sees the standalone fixed round prompt plus the structured-output capture contract. The parent sees only the original call and one terminal result containing status, round count, and pretty-printed final report; intermediate child messages and reports do not enter the parent conversation.
Each child sees the standalone fixed round prompt plus the structured-output capture contract. The parent sees only the original call and one terminal result containing a worker-reported status, round count, and pretty-printed final report; intermediate child messages and reports do not enter the parent conversation. A failed ordinary child instead yields an error with its round number and, after round one, the last successful handoff.
#### Token effect
Every round pays for a fresh child context. The parent result is bounded indirectly by `maxHandoffChars`; child work remains outside the parent context.
Every round pays for a fresh child context. `maxHandoffChars` bounds cross-round state and `maxResultChars` independently bounds the complete successful parent text; child work remains outside the parent context.
#### KV Cache effect
@@ -84,4 +87,5 @@ Each fresh child has an independent request cache. The parent result appends aft
- **Foreground only** — there is no task id, background collection, process-resume checkpoint, scheduler, or wall-clock start policy.
- **The workspace is the only cross-round long-term memory** — one bounded report is the explicit handoff, and uncommitted conversational reasoning disappears with each child.
- **One round is one fresh child** — there is no within-round fan-out, model/provider switching, fork context, or model-call-selected provider.
- **Ordinary child failure is terminal for the run** — the fixed script reports the failed round and last successful handoff but does not retry; fatal workflow infrastructure failures can end before that state is returned.
- **Only round count bounds aggregate effort** — token, price, and elapsed-time budgets are deferred.

View File

@@ -26,6 +26,8 @@ export interface Config {
maxRounds?: number
/** Maximum serialized characters in one structured handoff (default 16384). */
maxHandoffChars?: number
/** Maximum characters in a successful parent-facing terminal text (default 16384). */
maxResultChars?: number
}
/** Schemastery configuration for the Ralph tool. */
@@ -33,12 +35,14 @@ export const Config: z<Config> = z.object({
subagentProvider: z.string().default('spawn'),
maxRounds: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(256),
maxHandoffChars: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(16_384),
maxResultChars: z.number().step(1).min(1).max(Number.MAX_SAFE_INTEGER).default(16_384),
})
interface ResolvedConfig {
readonly subagentProvider: string
readonly maxRounds: number
readonly maxHandoffChars: number
readonly maxResultChars: number
}
type RalphRoundStatus = 'continue' | 'complete' | 'blocked'
@@ -59,6 +63,14 @@ interface RalphRunResult {
readonly report: RalphRoundReport
}
interface RalphRoundFailure {
readonly status: 'round-failed'
readonly roundsStarted: number
readonly lastReport?: RalphRoundReport
}
type RalphTerminalResult = RalphRunResult | RalphRoundFailure
interface RalphCallArgs {
objective: string
maxRounds?: number
@@ -136,8 +148,8 @@ function validateReport(report) {
}
let previous
phase('Fresh-agent rounds')
for (let round = 1; round <= args.maxRounds; round += 1) {
phase('Fresh-agent rounds')
const prior = previous === undefined ? '(none — this is the first round)' : JSON.stringify(previous)
const prompt = [
'You are one fresh worker in a foreground Ralph loop. You receive no parent conversation and no prior child session. Do not call the ralph tool: this round already is its worker.',
@@ -147,11 +159,15 @@ for (let round = 1; round <= args.maxRounds; round += 1) {
'Previous structured handoff:\n' + prior,
'Return one report with exact normalized strings. Use status continue with at least one nextSteps entry while useful work remains; complete only with concrete evidence and no nextSteps; blocked only when no meaningful progress is possible without human input or an external-state change. blocker must be empty unless blocked.',
].join('\n\n')
const report = validateReport(await agent(prompt, {
const rawReport = await agent(prompt, {
label: 'Ralph round ' + round,
phase: 'Fresh-agent rounds',
schema: reportSchema,
}))
})
if (rawReport === null) {
return { status: 'round-failed', roundsStarted: round, lastReport: previous ?? null }
}
const report = validateReport(rawReport)
if (report.status === 'complete') return { status: 'complete', roundsStarted: round, report }
if (report.status === 'blocked') return { status: 'blocked', roundsStarted: round, report }
previous = report
@@ -162,8 +178,8 @@ return { status: 'budget-limited', roundsStarted: args.maxRounds, report: previo
const DESCRIPTION = 'Run a foreground fresh-agent Ralph loop toward one immutable objective. '
+ 'Use only when the direct human explicitly asks for Ralph or fresh-agent iteration. Each round '
+ 'opens a new child with no parent conversation or prior child session; the shared workspace is '
+ 'long-term memory, and only a bounded structured report crosses rounds. The call returns on '
+ 'completion, a concrete blocker, or the round limit. Ordinary long-running same-session work '
+ 'long-term memory, and only a bounded structured report crosses rounds. The call returns when '
+ 'a worker reports completion or a concrete blocker, or at the round limit. Ordinary long-running same-session work '
+ 'belongs to goal tools.'
/** Validate defaults even when a caller invokes apply() without Loader normalization. */
@@ -171,6 +187,7 @@ function resolveConfig(config: Config): ResolvedConfig {
const subagentProvider = config.subagentProvider ?? 'spawn'
const maxRounds = config.maxRounds ?? 256
const maxHandoffChars = config.maxHandoffChars ?? 16_384
const maxResultChars = config.maxResultChars ?? 16_384
if (subagentProvider.length === 0 || subagentProvider !== subagentProvider.trim()) {
throw new TypeError('subagentProvider must be a non-empty normalized string')
}
@@ -180,7 +197,10 @@ function resolveConfig(config: Config): ResolvedConfig {
if (!Number.isSafeInteger(maxHandoffChars) || maxHandoffChars < 1) {
throw new TypeError('maxHandoffChars must be a positive safe integer')
}
return { subagentProvider, maxRounds, maxHandoffChars }
if (!Number.isSafeInteger(maxResultChars) || maxResultChars < 1) {
throw new TypeError('maxResultChars must be a positive safe integer')
}
return { subagentProvider, maxRounds, maxHandoffChars, maxResultChars }
}
/** Resolve one model-selected cap against the deployment ceiling. */
@@ -259,9 +279,8 @@ function readReport(value: unknown, expectedStatus: RalphRoundStatus, maxChars:
}
/** Defensively decode the fixed script's terminal value. */
function readRunResult(value: unknown, maxRounds: number, maxHandoffChars: number): RalphRunResult {
function readRunResult(value: unknown, maxRounds: number, maxHandoffChars: number): RalphTerminalResult {
if (!isRecord(value)
|| Object.keys(value).sort().join(',') !== 'report,roundsStarted,status'
|| typeof value['roundsStarted'] !== 'number'
|| !Number.isSafeInteger(value['roundsStarted'])
|| value['roundsStarted'] < 1
@@ -271,14 +290,42 @@ function readRunResult(value: unknown, maxRounds: number, maxHandoffChars: numbe
const roundsStarted = value['roundsStarted']
switch (value['status']) {
case 'complete':
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
throw new Error('Ralph workflow returned a malformed terminal result')
}
return { status: 'complete', roundsStarted, report: readReport(value['report'], 'complete', maxHandoffChars) }
case 'blocked':
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
throw new Error('Ralph workflow returned a malformed terminal result')
}
return { status: 'blocked', roundsStarted, report: readReport(value['report'], 'blocked', maxHandoffChars) }
case 'budget-limited':
if (Object.keys(value).sort().join(',') !== 'report,roundsStarted,status') {
throw new Error('Ralph workflow returned a malformed terminal result')
}
if (roundsStarted !== maxRounds) {
throw new Error('Ralph workflow returned budget-limited before the round limit')
}
return { status: 'budget-limited', roundsStarted, report: readReport(value['report'], 'continue', maxHandoffChars) }
case 'round-failed': {
if (Object.keys(value).sort().join(',') !== 'lastReport,roundsStarted,status') {
throw new Error('Ralph workflow returned a malformed terminal result')
}
if (roundsStarted === 1) {
if (value['lastReport'] !== null) {
throw new Error('Ralph workflow returned an invalid first-round failure')
}
return { status: 'round-failed', roundsStarted }
}
if (value['lastReport'] === null) {
throw new Error('Ralph workflow returned a round failure without its last handoff')
}
return {
status: 'round-failed',
roundsStarted,
lastReport: readReport(value['lastReport'], 'continue', maxHandoffChars),
}
}
default:
throw new Error('Ralph workflow returned an unknown terminal status')
}
@@ -300,17 +347,40 @@ function stopReasonError(result: WorkflowResult): string | undefined {
}
}
/** Render the fixed terminal envelope without dropping the bounded report. */
function renderResult(result: RalphRunResult): string {
const TRUNCATION_NOTICE = '\n… [truncated]'
/** Bound complete parent-facing text, including its envelope and truncation marker. */
function boundResult(text: string, maxChars: number): string {
if (text.length <= maxChars) return text
if (maxChars <= TRUNCATION_NOTICE.length) return TRUNCATION_NOTICE.slice(0, maxChars)
return `${text.slice(0, maxChars - TRUNCATION_NOTICE.length)}${TRUNCATION_NOTICE}`
}
/** Render the fixed terminal envelope without presenting self-report as certification. */
function renderResult(result: RalphRunResult, maxChars: number): string {
const rounds = `${result.roundsStarted} round${result.roundsStarted === 1 ? '' : 's'}`
let text: string
switch (result.status) {
case 'complete':
return `Ralph completed after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
text = `Ralph worker reported completion after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
break
case 'blocked':
return `Ralph blocked after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
text = `Ralph worker reported a blocker after ${rounds}.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
break
case 'budget-limited':
return `Ralph reached its ${rounds} limit with work remaining.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
text = `Ralph reached its ${rounds} limit; the worker reported work remaining.\nFinal report:\n${JSON.stringify(result.report, null, 2)}`
break
}
return boundResult(text, maxChars)
}
/** Render an ordinary child failure with the most recent durable handoff. */
function renderRoundFailure(result: RalphRoundFailure, maxChars: number): string {
const header = `Ralph round ${result.roundsStarted} child failed before producing a structured report.`
const text = result.lastReport === undefined
? `${header}\nNo previous handoff was available.`
: `${header}\nLast successful handoff:\n${JSON.stringify(result.lastReport, null, 2)}`
return boundResult(text, maxChars)
}
function presentCall(args: RalphCallArgs): ToolCallView {
@@ -329,7 +399,7 @@ export function apply(ctx: Context, config: Config): void {
ctx.systemPrompt.section({
name: 'tool:ralph',
order: 116,
text: 'Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.',
text: 'Use the ralph tool ONLY when the direct human explicitly asks for a Ralph loop or fresh-agent iterative execution. Each Ralph round starts a fresh child with no conversation seed and uses the shared workspace as durable memory. Completion and blockers are worker reports, not independent evaluation. Use same-session goal tools for ordinary long-running objectives, and plain subagents or workflows for bounded delegation and fan-out.',
})
ctx.tools.register(defineTool({
name: 'ralph',
@@ -360,6 +430,7 @@ export function apply(ctx: Context, config: Config): void {
meta: RALPH_META,
args: { objective, maxRounds, maxHandoffChars: resolved.maxHandoffChars },
subagentProvider: resolved.subagentProvider,
maxTotalAgents: maxRounds,
parent,
...exec.signal === undefined ? {} : { signal: exec.signal },
})
@@ -372,7 +443,8 @@ export function apply(ctx: Context, config: Config): void {
const error = stopReasonError(settled)
if (error !== undefined) throw new Error(error)
const value = readRunResult(settled.value, maxRounds, resolved.maxHandoffChars)
return [{ type: 'text', text: renderResult(value) }]
if (value.status === 'round-failed') throw new Error(renderRoundFailure(value, resolved.maxResultChars))
return [{ type: 'text', text: renderResult(value, resolved.maxResultChars) }]
} finally {
exec.signal?.removeEventListener('abort', onAbort)
await run.dispose()

View File

@@ -1,4 +1,4 @@
import { describe, expect, it } from 'vitest'
import { describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import type { Agent } from '@deepseek-ai/dsh-agent'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
@@ -10,9 +10,31 @@ import SubagentService from '@deepseek-ai/dsh-subagent'
import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-inprocess'
import * as spawn from '@deepseek-ai/dsh-subagent-spawn'
import WorkerWorkflowEngine from '@deepseek-ai/dsh-workflow-workerthread'
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import { MockAdapter, maxTokensResponse, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import * as toolRalph from '../src/index.ts'
type MockScript = ConstructorParameters<typeof MockAdapter>[0]
/** Mount the shipped Ralph execution stack around one keyless model script. */
async function mountRalph(script: MockScript, config: toolRalph.Config) {
const ctx = new Context()
const adapter = new MockAdapter(script)
await mountAgentLoopTestDependencies(ctx)
await ctx.plugin(Invariants)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(SubagentService)
await ctx.plugin(spawn, { providerName: 'spawn' })
await ctx.plugin(WorkerWorkflowEngine, {})
await ctx.plugin(toolRalph, config)
ctx.llm.registerAdapter(['mock'], adapter)
const parentHandle = await ctx.agents.create({
sessionId: SessionId('ralph-parent'),
meta: { cwd: '/tmp/ralph-shared-workspace' },
agentOptions: { provider: 'mock', model: 'mock' },
})
return { ctx, adapter, parentHandle, parent: parentHandle.agent }
}
describe('dsh-tool-ralph over the real spawn and worker-thread stack', () => {
it('uses distinct empty-seed children, shared cwd, and only the prior bounded handoff', async () => {
const firstReport = {
@@ -54,6 +76,8 @@ describe('dsh-tool-ralph over the real spawn and worker-thread stack', () => {
await parent.whenIdle()
const children: Agent[] = []
const phases: string[] = []
ctx.on('workflow/phase', (_run, title) => { phases.push(title) })
ctx.on('workflow/agent-start', (_run, child) => {
const agent = ctx.agents.get(child.childId)
expect(agent).toBeDefined()
@@ -67,7 +91,9 @@ describe('dsh-tool-ralph over the real spawn and worker-thread stack', () => {
})
expect(result.isError).toBe(false)
expect((result.content[0] as { text: string }).text).toContain('Ralph completed after 2 rounds.')
expect((result.content[0] as { text: string }).text)
.toContain('Ralph worker reported completion after 2 rounds.')
expect(phases).toEqual(['Fresh-agent rounds'])
expect(children).toHaveLength(2)
expect(new Set(children.map(child => child.id)).size).toBe(2)
for (const child of children) {
@@ -89,4 +115,151 @@ describe('dsh-tool-ralph over the real spawn and worker-thread stack', () => {
await parentHandle.dispose()
})
it('reports the failed round and last good handoff when a child fails', async () => {
const firstReport = {
status: 'continue',
summary: 'ROUND_ONE_HANDOFF',
evidence: ['Created migration-a.ts.'],
nextSteps: ['Finish migration-b.ts.'],
blocker: '',
}
const { ctx, parent, parentHandle } = await mountRalph([
toolCallResponse('round-1', STRUCTURED_OUTPUT_TOOL, firstReport),
maxTokensResponse('unfinished child output'),
], { maxRounds: 2 })
const children: Agent[] = []
ctx.on('workflow/agent-start', (_run, child) => {
const agent = ctx.agents.get(child.childId)
if (agent !== undefined) children.push(agent)
})
const result = await ctx.tools.execute({
callId: CallId('ralph-child-failure'),
name: 'ralph',
arguments: { objective: 'Complete both migration slices.', maxRounds: 2 },
agent: parent,
})
expect(result.isError).toBe(true)
const text = (result.content[0] as { text: string }).text
expect(text).toContain('Ralph round 2 child failed before producing a structured report.')
expect(text).toContain('Last successful handoff:')
expect(text).toContain('ROUND_ONE_HANDOFF')
expect(children).toHaveLength(2)
for (const child of children) expect(ctx.agents.get(child.id)).toBeUndefined()
await parentHandle.dispose()
})
it.each([
{
name: 'blocked',
report: {
status: 'blocked',
summary: 'External authorization is required.',
evidence: ['The local implementation is ready.'],
nextSteps: ['Continue after authorization.'],
blocker: 'The required external authorization is unavailable.',
},
config: { maxRounds: 2 },
expectedError: false,
expectedText: 'Ralph worker reported a blocker after 1 round.',
},
{
name: 'budget-limited',
report: {
status: 'continue',
summary: 'One slice is complete.',
evidence: ['The first focused test passes.'],
nextSteps: ['Implement the remaining slice.'],
blocker: '',
},
config: { maxRounds: 1 },
expectedError: false,
expectedText: 'Ralph reached its 1 round limit; the worker reported work remaining.',
},
{
name: 'unnormalized report',
report: {
status: 'continue',
summary: ' padded summary ',
evidence: ['A focused test passes.'],
nextSteps: ['Continue implementation.'],
blocker: '',
},
config: { maxRounds: 1 },
expectedError: true,
expectedText: 'summary must be non-empty and normalized',
},
{
name: 'invalid continuing report',
report: {
status: 'continue',
summary: 'Work remains.',
evidence: ['A focused test passes.'],
nextSteps: [],
blocker: '',
},
config: { maxRounds: 1 },
expectedError: true,
expectedText: 'a continuing Ralph report needs nextSteps and an empty blocker',
},
{
name: 'oversized report',
report: {
status: 'continue',
summary: 'x'.repeat(300),
evidence: ['A focused test passes.'],
nextSteps: ['Continue implementation.'],
blocker: '',
},
config: { maxRounds: 1, maxHandoffChars: 100 },
expectedError: true,
expectedText: 'Ralph round report exceeds maxHandoffChars',
},
])('enforces the fixed script for $name', async ({ report, config, expectedError, expectedText }) => {
const { ctx, parent, parentHandle } = await mountRalph([
toolCallResponse('round-report', STRUCTURED_OUTPUT_TOOL, report),
], config)
const result = await ctx.tools.execute({
callId: CallId('ralph-script-enforcement'),
name: 'ralph',
arguments: { objective: 'Complete the scoped work.', maxRounds: config.maxRounds },
agent: parent,
})
expect(result.isError).toBe(expectedError)
expect((result.content[0] as { text: string }).text).toContain(expectedText)
await parentHandle.dispose()
})
it('cancels the real worker and fresh child to quiescence', async () => {
const { ctx, parent, parentHandle } = await mountRalph(['hang'], { maxRounds: 2 })
const children: Agent[] = []
const outcomes: string[] = []
ctx.on('workflow/agent-start', (_run, child) => {
const agent = ctx.agents.get(child.childId)
if (agent !== undefined) children.push(agent)
})
ctx.on('workflow/agent-end', (_run, child) => { outcomes.push(child.outcome) })
const controller = new AbortController()
const pending = ctx.tools.execute({
callId: CallId('ralph-real-cancel'),
name: 'ralph',
arguments: { objective: 'Keep working until cancelled.', maxRounds: 2 },
agent: parent,
signal: controller.signal,
})
await vi.waitFor(() => { expect(children).toHaveLength(1) })
controller.abort()
const result = await pending
expect(result.isError).toBe(true)
expect((result.content[0] as { text: string }).text).toContain('Ralph workflow was cancelled')
expect(outcomes).toEqual(['cancelled'])
expect(ctx.agents.get(children[0]!.id)).toBeUndefined()
await parentHandle.dispose()
})
})

View File

@@ -82,6 +82,7 @@ async function setup(options?: SetupOptions) {
if (options?.config?.subagentProvider !== undefined) config.subagentProvider = options.config.subagentProvider
if (options?.config?.maxRounds !== undefined) config.maxRounds = options.config.maxRounds
if (options?.config?.maxHandoffChars !== undefined) config.maxHandoffChars = options.config.maxHandoffChars
if (options?.config?.maxResultChars !== undefined) config.maxResultChars = options.config.maxResultChars
const fiber = await ctx.plugin(toolRalph, config)
const parent = { id: SessionId('caller'), options: {} } as unknown as Agent
return { ctx, engine: ctx.workflows as StubEngine, parent, fiber }
@@ -145,6 +146,7 @@ describe('dsh-tool-ralph', () => {
meta: { name: 'ralph-loop' },
args: { objective: 'Finish the migration.', maxRounds: 4, maxHandoffChars: 9000 },
subagentProvider: 'fresh',
maxTotalAgents: 4,
parent,
})
expect(engine.requests[0]!.script).toContain("status: 'budget-limited'")
@@ -154,7 +156,8 @@ describe('dsh-tool-ralph', () => {
report: COMPLETE,
})
expect(result.isError).toBe(false)
expect((result.content[0] as { text: string }).text).toContain('Ralph completed after 1 round.')
expect((result.content[0] as { text: string }).text)
.toContain('Ralph worker reported completion after 1 round.')
expect((result.content[0] as { text: string }).text).toContain('All required gates pass.')
expect(engine.disposed).toBe(1)
})
@@ -167,7 +170,8 @@ describe('dsh-tool-ralph', () => {
roundsStarted: 2,
report: BLOCKED,
}, 2)
expect((blockedResult.content[0] as { text: string }).text).toContain('Ralph blocked after 2 rounds.')
expect((blockedResult.content[0] as { text: string }).text)
.toContain('Ralph worker reported a blocker after 2 rounds.')
const limited = execute(ctx, { objective: 'Ship it.' }, { agent: parent })
await vi.waitFor(() => { expect(engine.requests).toHaveLength(2) })
@@ -177,7 +181,54 @@ describe('dsh-tool-ralph', () => {
report: CONTINUE,
}, 2)
expect((limitedResult.content[0] as { text: string }).text)
.toContain('Ralph reached its 2 rounds limit with work remaining.')
.toContain('Ralph reached its 2 rounds limit; the worker reported work remaining.')
})
it('bounds the complete parent result and labels worker-reported completion', async () => {
const { ctx, engine, parent } = await setup({ config: { maxResultChars: 160 } })
const pending = execute(ctx, { objective: 'Ship it.' }, { agent: parent })
const result = await settleCompleted(engine, pending, {
status: 'complete',
roundsStarted: 1,
report: { ...COMPLETE, evidence: ['x'.repeat(500)] },
})
const text = (result.content[0] as { text: string }).text
expect(text).toHaveLength(160)
expect(text).toContain('Ralph worker reported completion after 1 round.')
expect(text).toMatch(/… \[truncated\]$/)
})
it('honors a result limit shorter than the truncation marker', async () => {
const { ctx, engine, parent } = await setup({ config: { maxResultChars: 5 } })
const result = await settleCompleted(engine, execute(ctx, { objective: 'Ship it.' }, { agent: parent }), {
status: 'complete',
roundsStarted: 1,
report: COMPLETE,
})
expect((result.content[0] as { text: string }).text).toBe('\n… [t')
})
it('reports an ordinary child failure with the failed round and last durable handoff', async () => {
const { ctx, engine, parent } = await setup({ config: { maxRounds: 2 } })
const first = execute(ctx, { objective: 'Ship it.', maxRounds: 2 }, { agent: parent })
const firstResult = await settleCompleted(engine, first, {
status: 'round-failed',
roundsStarted: 1,
lastReport: null,
})
expect(firstResult.isError).toBe(true)
expect((firstResult.content[0] as { text: string }).text).toContain('Ralph round 1 child failed')
expect((firstResult.content[0] as { text: string }).text).toContain('No previous handoff was available.')
const later = execute(ctx, { objective: 'Ship it.', maxRounds: 2 }, { agent: parent })
const laterResult = await settleCompleted(engine, later, {
status: 'round-failed',
roundsStarted: 2,
lastReport: CONTINUE,
})
expect(laterResult.isError).toBe(true)
expect((laterResult.content[0] as { text: string }).text).toContain('Ralph round 2 child failed')
expect((laterResult.content[0] as { text: string }).text).toContain('Implemented the first slice.')
})
it('maps workflow error and cancellation reasons to tool errors and always disposes', async () => {
@@ -251,6 +302,7 @@ describe('dsh-tool-ralph', () => {
expect(() => { toolRalph.apply(new Context(), { subagentProvider: ' ' }) }).toThrow('non-empty normalized')
expect(() => { toolRalph.apply(new Context(), { maxRounds: 0 }) }).toThrow('positive safe integer')
expect(() => { toolRalph.apply(new Context(), { maxHandoffChars: 1.5 }) }).toThrow('positive safe integer')
expect(() => { toolRalph.apply(new Context(), { maxResultChars: 0 }) }).toThrow('positive safe integer')
})
it('turns malformed fixed-workflow terminal values and reports into errors', async () => {
@@ -261,11 +313,18 @@ describe('dsh-tool-ralph', () => {
{ value: { status: 'mystery', roundsStarted: 1, report: COMPLETE }, message: 'unknown terminal status' },
{ value: { status: 'budget-limited', roundsStarted: 1, report: CONTINUE }, message: 'before the round limit', config: { maxRounds: 2 } },
{ value: { status: 'complete', roundsStarted: 1, report: null }, message: 'malformed round report' },
{ value: { status: 'complete', roundsStarted: 1, report: COMPLETE, extra: true }, message: 'malformed terminal result' },
{ value: { status: 'blocked', roundsStarted: 1, report: BLOCKED, extra: true }, message: 'malformed terminal result' },
{ value: { status: 'budget-limited', roundsStarted: 1, report: CONTINUE, extra: true }, message: 'malformed terminal result', config: { maxRounds: 1 } },
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, status: 'continue' } }, message: 'malformed round report' },
{ value: { status: 'budget-limited', roundsStarted: 1, report: { ...CONTINUE, nextSteps: [] } }, message: 'invalid continuing report', config: { maxRounds: 1 } },
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, evidence: [] } }, message: 'invalid completion report' },
{ value: { status: 'blocked', roundsStarted: 1, report: { ...BLOCKED, blocker: '' } }, message: 'invalid blocked report' },
{ value: { status: 'complete', roundsStarted: 1, report: { ...COMPLETE, summary: 'x'.repeat(500) } }, message: 'oversized handoff', config: { maxHandoffChars: 100 } },
{ value: { status: 'round-failed', roundsStarted: 1 }, message: 'malformed terminal result' },
{ value: { status: 'round-failed', roundsStarted: 1, lastReport: CONTINUE }, message: 'invalid first-round failure' },
{ value: { status: 'round-failed', roundsStarted: 2, lastReport: null }, message: 'without its last handoff', config: { maxRounds: 2 } },
{ value: { status: 'round-failed', roundsStarted: 2, lastReport: { ...CONTINUE, nextSteps: [] } }, message: 'invalid continuing report', config: { maxRounds: 2 } },
]
for (const testCase of cases) {
const { ctx, engine, parent } = await setup(
@@ -294,7 +353,9 @@ describe('dsh-tool-ralph', () => {
const { ctx, fiber } = await setup()
const section = (await ctx.systemPrompt.assemble()).sections.find(candidate => candidate.name === 'tool:ralph')
expect(section?.text).toContain('ONLY when the direct human explicitly asks')
expect(section?.text).toContain('worker reports, not independent evaluation')
const tool = ctx.tools.get('ralph')!
expect(tool.description).toContain('worker reports completion')
expect(tool.presentCall!({ objective: 'Finish it.' })).toEqual({
card: 'generic',
title: 'ralph',

View File

@@ -34,7 +34,7 @@ Unknown options, malformed arguments, unsupported schemas, tripped caps, provide
## Run sequence
`start()` validates meta and parses the body, creates the worker, and returns a holder-owned `WorkflowRun`. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
`start()` validates meta, parses the body, resolves a registered normalized provider route, and resolves any per-run total-child cap before creating a worker or publishing `workflow/start`. A requested `maxTotalAgents` must be a positive safe integer no greater than the engine's configured deployment ceiling. Source mode installs TypeScript transforms through a data-URL bootstrap; built mode passes sibling `lib/worker.cjs` as a filesystem path because pkg's VFS hook expects CommonJS. Both work under ordinary Node. A ready/go handshake prevents a start-signal cancellation racing worker boot from executing the script's initial synchronous slice.
For each `agent()` call:
@@ -81,7 +81,7 @@ The host keeps a ledger of forwarded child starts. A graceful worker supplies th
| `syncTimeoutMs` | `5000` | VM timeout for the script's initial synchronous slice. |
| `disposeGraceMs` | `5000` | Bound before force-settlement/termination and for public disposal. |
An owning consumer may set `WorkflowStartRequest.subagentProvider` for one run. This is an engine-level route, not a script hook or a model-facing option; the ordinary `workflow` tool leaves it unset.
An owning consumer may set `WorkflowStartRequest.subagentProvider` and `WorkflowStartRequest.maxTotalAgents` for one run. These are engine-level policy, not script hooks or model-facing options; the ordinary `workflow` tool leaves both unset. A per-run total-child cap may lower but never raise the configured `maxTotalAgents` ceiling.
## Model Experience

View File

@@ -73,6 +73,36 @@ function assertBodyParses(body: string, name: string): void {
}
}
/** Resolve one run's provider route before publishing work. */
function resolveSubagentProvider(ctx: Context, configured: string, override: string | undefined): string {
const provider = override ?? configured
if (provider.length === 0 || provider !== provider.trim()) {
throw new WorkflowError(
'workflow subagentProvider must be a non-empty normalized string',
'INVALID_ARGUMENT',
)
}
if (ctx.subagents.getProvider(provider) === undefined) {
throw new WorkflowError(`no subagent provider registered for "${provider}"`, 'AGENT_START')
}
return provider
}
/** Resolve one run's total-child cap against the engine deployment ceiling. */
function resolveMaxTotalAgents(requested: number | undefined, ceiling: number): number {
if (requested === undefined) return ceiling
if (!Number.isSafeInteger(requested) || requested < 1) {
throw new WorkflowError('workflow maxTotalAgents must be a positive safe integer', 'INVALID_ARGUMENT')
}
if (requested > ceiling) {
throw new WorkflowError(
`workflow maxTotalAgents ${requested} exceeds the engine ceiling ${ceiling}`,
'INVALID_ARGUMENT',
)
}
return requested
}
/**
* The worker-thread engine service. `start()` validates the script up front
* (meta + a host-side body parse) and returns a {@link WorkflowRun} whose
@@ -113,13 +143,15 @@ class WorkerWorkflowEngine extends WorkflowService {
start(request: WorkflowStartRequest): WorkflowRun {
const meta = validateMeta(request.meta)
assertBodyParses(request.script, meta.name)
const subagentProvider = resolveSubagentProvider(this.ctx, this.config.provider, request.subagentProvider)
const maxTotalAgents = resolveMaxTotalAgents(request.maxTotalAgents, this.config.maxTotalAgents)
const id = WorkflowRunId(randomUUID())
const info: WorkflowRunInfo = { id, meta }
const limits: WorkerLimits = {
maxConcurrentAgents: this.config.maxConcurrentAgents === 0
? Math.min(16, Math.max(1, availableParallelism() - 2))
: this.config.maxConcurrentAgents,
maxTotalAgents: this.config.maxTotalAgents,
maxTotalAgents,
maxItemsPerCall: this.config.maxItemsPerCall,
syncTimeoutMs: this.config.syncTimeoutMs,
}
@@ -137,7 +169,6 @@ class WorkerWorkflowEngine extends WorkflowService {
// the now-inactive engine fiber and break the seam's holder-owned lifetime.
const runCtx = this.ctx
const subagents = runCtx.subagents
const subagentProvider = request.subagentProvider ?? this.config.provider
const workerRun = new WorkerRun(
runCtx,
subagents,

View File

@@ -255,7 +255,7 @@ export class WorkflowExecution {
const opts = this.readAgentOptions(rawOpts)
if (this.started >= this.limits.maxTotalAgents) {
throw new WorkflowError(
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise maxTotalAgents in the engine config if the scale is intentional`,
`this run reached its total agent cap (${this.limits.maxTotalAgents}) — a runaway-loop backstop; raise the applicable maxTotalAgents limit if the scale is intentional`,
'AGENT_CAP',
)
}

View File

@@ -392,6 +392,7 @@ describe('runWorkerSession over an in-process MessageChannel', () => {
const result = await host.result()
expect(result.stopReason).toBe('error')
expect(result.error).toContain('total agent cap (2)')
expect(result.error).toContain('applicable maxTotalAgents limit')
expect(result.agentsStarted).toBe(2)
host.close()
})

View File

@@ -8,6 +8,7 @@ import { expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import type { Agent } from '@deepseek-ai/dsh-agent'
import SubagentService from '@deepseek-ai/dsh-subagent'
import type { SubagentProvider } from '@deepseek-ai/dsh-subagent'
import WorkerWorkflowEngine from '../src/index.ts'
import { SessionId } from '@deepseek-ai/dsh-session'
@@ -18,6 +19,13 @@ vi.setConfig({ testTimeout: 30_000 })
it('runs the default config through the source worker', async () => {
const ctx = new Context()
const subagents = await ctx.plugin(SubagentService)
const provider: SubagentProvider = {
name: 'spawn',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: true, persona: true },
inheritsParentContext: false,
start: () => Promise.reject(new Error('source-worker compat script must not start a child')),
}
ctx.subagents.registerProvider(provider)
const engine = await ctx.plugin(WorkerWorkflowEngine, {})
const parent = { id: SessionId('workflow-compat-parent'), options: {} } as unknown as Agent
try {

View File

@@ -6,7 +6,7 @@ import Loader from '@cordisjs/plugin-loader'
import type { Agent } from '@deepseek-ai/dsh-agent'
import SubagentService from '@deepseek-ai/dsh-subagent'
import type { SubagentCapabilities, SubagentProvider, SubagentResult, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRun, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
import * as workerEngineModule from '../src/index.ts'
import WorkerWorkflowEngine, { type Config } from '../src/index.ts'
import { HostToWorkerType, WorkerToHostType } from '../src/protocol.ts'
@@ -252,6 +252,77 @@ describe('dsh-workflow-workerthread', () => {
expect(provider.runs).toHaveLength(1)
})
it('rejects invalid start-request provider routes before publishing a run', async () => {
const { ctx, parent } = await setup()
let starts = 0
ctx.on('workflow/start', () => { starts += 1 })
const messages: string[] = []
for (const subagentProvider of ['', 'missing']) {
let run: WorkflowRun | undefined
let thrown: unknown
try {
run = ctx.workflows.start({
...scripted("return 'must not start'"),
parent,
subagentProvider,
})
} catch (error: unknown) {
thrown = error
}
await run?.dispose()
messages.push(thrown instanceof Error ? thrown.message : '')
}
expect(messages).toEqual([
'workflow subagentProvider must be a non-empty normalized string',
'no subagent provider registered for "missing"',
])
expect(starts).toBe(0)
})
it('rejects invalid per-run total-agent caps before publishing a run', async () => {
const { ctx, parent } = await setup({ config: { maxTotalAgents: 2 } })
let starts = 0
ctx.on('workflow/start', () => { starts += 1 })
const errors: unknown[] = []
for (const maxTotalAgents of [0, 1.5, Number.NaN, 3]) {
try {
const handle = ctx.workflows.start({
...scripted("return 'must not start'"),
parent,
maxTotalAgents,
})
await handle.dispose()
} catch (error: unknown) {
errors.push(error)
}
}
expect(errors.slice(0, 3)).toEqual(Array(3).fill(expect.objectContaining({
code: 'INVALID_ARGUMENT',
message: 'workflow maxTotalAgents must be a positive safe integer',
})))
expect(errors[3]).toMatchObject({
code: 'INVALID_ARGUMENT',
message: 'workflow maxTotalAgents 3 exceeds the engine ceiling 2',
})
expect(starts).toBe(0)
})
it('enforces a per-run total-agent cap below the engine ceiling', async () => {
const { ctx, parent } = await setup({ config: { maxTotalAgents: 2 } })
const handle = ctx.workflows.start({
...scripted("await agent('first'); await agent('second'); return 'unreachable'"),
parent,
maxTotalAgents: 1,
})
const result = await handle.result
expect(result.stopReason).toBe('error')
expect(result.agentsStarted).toBe(1)
expect(result.error).toContain('total agent cap (1)')
await handle.dispose()
})
it('a fatal hook error inside the worker kills the script and reports the error', async () => {
const { ctx, parent } = await setup()
const result = await run(ctx, parent, scripted("return await parallel([() => agent('x', { isolation: 'worktree' })])"))
@@ -259,11 +330,18 @@ describe('dsh-workflow-workerthread', () => {
expect(result.error).toContain('"isolation" is deferred')
})
it('a provider start failure crosses back as a fatal AGENT_START error', async () => {
it('rejects an unregistered configured provider before publishing a run', async () => {
const { ctx, parent } = await setup({ config: { provider: 'nonexistent' } })
const result = await run(ctx, parent, scripted("return await pipeline([1], () => agent('p'))"))
expect(result.stopReason).toBe('error')
expect(result.error).toContain('agent() could not start a child')
let thrown: unknown
try {
ctx.workflows.start({ ...scripted("return 'must not start'"), parent })
} catch (error: unknown) {
thrown = error
}
expect(thrown).toMatchObject({
code: 'AGENT_START',
message: 'no subagent provider registered for "nonexistent"',
})
})
it('waits for async provider start before announcing a result that settled early', async () => {

View File

@@ -6,11 +6,11 @@ The workflow seam (`ctx.workflows`) executes a model-written orchestration scrip
## Service and run contract
`WorkflowService.start(request): WorkflowRun` validates enough synchronously to reject a malformed meta block or unparseable script before a run exists. Once returned, `WorkflowRun.result` never rejects: execution failures resolve with `stopReason: 'error'`, and cancellation resolves with `cancelled` within the engine's bounded grace.
`WorkflowService.start(request): WorkflowRun` validates enough synchronously to reject a malformed meta block, unparseable script, unavailable provider route, or unsupported per-run limit before a run exists. Once returned, `WorkflowRun.result` never rejects: execution failures resolve with `stopReason: 'error'`, and cancellation resolves with `cancelled` within the engine's bounded grace.
A run is holder-owned. Engine-plugin unload prevents new starts but does not revoke accepted runs. The holder must call `dispose()` on every path; disposal cancels remaining work and reaches or abandons quiescence within the documented bound.
`WorkflowStartRequest` contains `{ meta, script, args?, subagentProvider?, parent, signal? }`. `parent` attributes every child agent to the invoking agent. `subagentProvider` optionally routes every child in that run without exposing provider choice to the script; omission uses the engine's configured provider. `meta` and `args` are plain data, not script fragments.
`WorkflowStartRequest` contains `{ meta, script, args?, subagentProvider?, maxTotalAgents?, parent, signal? }`. `parent` attributes every child agent to the invoking agent. `subagentProvider` optionally routes every child in that run without exposing provider choice to the script; omission uses the engine's configured provider. `maxTotalAgents` optionally lowers the engine's deployment ceiling for one run and is likewise invisible to the script. An implementation rejects invalid routes and limits synchronously. `meta` and `args` are plain data, not script fragments.
`WorkflowRun` exposes `{ id, meta, result, cancel(reason?), dispose() }`. `WorkflowResult` contains `{ value, stopReason, error?, agentsStarted }`; `value` is plain JSON data or `null`.

View File

@@ -76,6 +76,11 @@ export interface WorkflowStartRequest {
* provider.
*/
subagentProvider?: string
/**
* Optional per-run total-child ceiling. Implementations reject values above
* their deployment ceiling before publishing the run.
*/
maxTotalAgents?: number
/** The agent on whose behalf the run executes (parent of every child). */
parent: Agent
/** Cancels the run when aborted (the tool's `exec.signal`). */