Merge remote-tracking branch 'origin/master' into feat/mcp-client

This commit is contained in:
lintianle
2026-07-08 12:08:24 +08:00
116 changed files with 3628 additions and 459 deletions

View File

@@ -71,6 +71,12 @@ A `defineTool` tool also **validates the model-generated arguments against its `
See `defineTool`, `validateArgs`, `ToolArgsError`, `SchemaSpec`, `InferArgs`, and `schemaSpecToJsonSchema` in the public API for details.
### Structured-output schema subset
A separate vocabulary for callers that DEMAND a machine-readable value from an agent — the subagent seam's `SubagentStartRequest.outputSchema` (and, by extension, a workflow's `agent({ schema })`). Unlike `SchemaSpec` (the author-facing DSL for tool parameters), a `StructuredOutputSchema` is an object-rooted **raw JSON Schema subset** as data: it travels verbatim to the model as a forced tool's `parameters`, and the produced value is validated against it.
The subset is deliberately narrow and REJECTS LOUD outside it — accepting a keyword the validator doesn't enforce would validate less than the schema promises (accepted-then-ignored). Supported: single-string `type` (`object`/`array`/`string`/`number`/`integer`/`boolean`/`null`; type arrays rejected), `properties`/`required`/`additionalProperties` (boolean; every `required` key must be declared), `items`, scalar-only `enum`/`const`; annotations (`description`/`title`/`default`/`examples`) are ignored but must still be JSON data. `assertSupportedOutputSchema(schema)` throws `OutputSchemaError` (`code: 'UNSUPPORTED_SCHEMA'`, listing every violation) for anything else; `validateStructuredValue(schema, value)` returns path-qualified violations (empty = valid, total — never throws).
### Tool-owned UI presentation
A tool owns how ITS calls render in a UI (an editor's tool-call card, a CLI log line) — a UI plugin must NOT special-case tool names. A `ToolDefinition` may declare two optional, pure, display-only methods that return a **`card`-tagged render intent** (a discriminated union — a tool declares its card kind once and a UI bridge switches on `card`):

View File

@@ -28,6 +28,16 @@ export {
type JsonSchemaObject,
} from './schema.ts'
export {
assertSupportedOutputSchema,
validateStructuredValue,
OutputSchemaError,
type StructuredOutputSchema,
type StructuredSchemaNode,
type StructuredSchemaType,
type StructuredScalar,
} from './json-schema.ts'
// The render-intent vocabulary a tool declares via `presentCall`/`presentResult`
// lives in its own UI-facing module; re-export it so `@deepseek-ai/dsh-tools`
// stays the single public surface for consumers (producers + the ACP bridge).

View File

@@ -0,0 +1,345 @@
/**
* Structured-output JSON Schema subset: the vocabulary a caller uses to demand
* a machine-readable result from a subagent (`SubagentStartRequest.outputSchema`)
* or a workflow `agent()` call.
*
* This is deliberately NOT full JSON Schema. The schema travels verbatim to the
* model as a forced tool's `parameters`, and the value the model produces is
* validated here — so every accepted keyword must be one this module actually
* enforces. Accepting a keyword we don't enforce would validate less than the
* schema promises (accepted-then-ignored), so anything outside the subset is
* REJECTED LOUD by {@link assertSupportedOutputSchema} instead. The subset:
*
* - `type` — a single string (`object`/`array`/`string`/`number`/`integer`/
* `boolean`/`null`); type ARRAYS (`["string","null"]`) are rejected.
* - `properties`/`required`/`additionalProperties` (boolean) on objects; every
* `required` key must be declared in `properties`. `additionalProperties`
* absent keeps standard JSON Schema semantics (extra keys allowed).
* - `items` on arrays (absent ⇒ any JSON items).
* - `enum` (non-empty, scalars only) and `const` (scalar) on scalar types.
* - Annotations `description`/`title`/`default`/`examples` are allowed and
* ignored (they constrain nothing), except that they must still be JSON data
* — the schema is serialized onto the wire, so a non-JSON annotation would be
* silently mangled.
*
* Values checked by {@link validateStructuredValue} are expected to be plain
* host-realm JSON data (model tool-call arguments are parsed wire JSON; a
* caller holding foreign-realm data materializes it first).
*
* @module dsh-tools/json-schema
*/
import { assertNever, HarnessError } from '@deepseek-ai/dsh-llm'
/** The scalar values `enum`/`const` may carry (finite numbers only). */
export type StructuredScalar = string | number | boolean | null
/** The `type` keywords the subset accepts. */
export type StructuredSchemaType = 'object' | 'array' | 'string' | 'number' | 'integer' | 'boolean' | 'null'
/**
* One node of the structured-output schema subset. Recursive via `properties`
* and `items`; see the module doc for the exact keyword semantics.
*/
export interface StructuredSchemaNode {
type: StructuredSchemaType
/** Nested property schemas (`type: 'object'` only). */
properties?: Record<string, StructuredSchemaNode>
/** Required property names; each must appear in `properties`. */
required?: string[]
/** `false` rejects undeclared keys; absent/`true` allows them (JSON Schema default). */
additionalProperties?: boolean
/** Item schema (`type: 'array'` only); absent ⇒ any JSON items. */
items?: StructuredSchemaNode
/** Allowed values (scalar types only). */
enum?: StructuredScalar[]
/** The single allowed value (scalar types only). */
const?: StructuredScalar
/** Annotation, ignored for validation. */
description?: string
/** Annotation, ignored for validation. */
title?: string
/** Annotation, ignored for validation (must still be JSON data). */
default?: unknown
/** Annotation, ignored for validation (must still be JSON data). */
examples?: unknown
}
/** A structured-output schema: an OBJECT-rooted {@link StructuredSchemaNode}. */
export type StructuredOutputSchema = StructuredSchemaNode & { type: 'object' }
/**
* Thrown by {@link assertSupportedOutputSchema} when a schema falls outside the
* supported subset. Extends {@link HarnessError} (`code: 'UNSUPPORTED_SCHEMA'`)
* so seam code and tool results can route on it; `violations` lists every
* offending path, not just the first.
*/
export class OutputSchemaError extends HarnessError {
/** The individual violation messages, in walk order. */
readonly violations: string[]
constructor(violations: string[]) {
super(`unsupported output schema: ${violations.join('; ')}`, 'UNSUPPORTED_SCHEMA')
this.name = 'OutputSchemaError'
this.violations = violations
}
}
/** The keywords the subset accepts, checked (`constraint`) or ignored (`annotation`). */
const CONSTRAINT_KEYWORDS = new Set(['type', 'properties', 'required', 'additionalProperties', 'items', 'enum', 'const'])
const ANNOTATION_KEYWORDS = new Set(['description', 'title', 'default', 'examples'])
const SCHEMA_TYPES: readonly StructuredSchemaType[] = ['object', 'array', 'string', 'number', 'integer', 'boolean', 'null']
/**
* Whether a value is a PLAIN JSON object — non-null, non-array, and with a
* prototype chain of at most one link (`null`-proto, or any realm's
* `Object.prototype`, whose own prototype is `null`). Realm-agnostic on
* purpose: a schema materialized in another realm carries THAT realm's
* `Object.prototype`, which an identity check would wrongly reject. Exotic
* hosts (`Date`, `Map`, class instances) have longer chains and are rejected —
* they would serialize lossily (`Date` → string, `Map` → `{}`) instead of
* failing loud.
*/
function isObjectLike(value: unknown): value is Record<string, unknown> {
if (typeof value !== 'object' || value === null || Array.isArray(value)) return false
const proto: unknown = Object.getPrototypeOf(value)
return proto === null || Object.getPrototypeOf(proto) === null
}
/** Whether a value is a supported scalar (`enum`/`const` member): string, finite number, boolean, or null. */
function isStructuredScalar(value: unknown): value is StructuredScalar {
return value === null || typeof value === 'string' || typeof value === 'boolean'
|| (typeof value === 'number' && Number.isFinite(value))
}
/**
* Whether a value is JSON data (annotation payloads only): scalars, arrays, and
* object-likes of such values. Realm-agnostic on purpose (no prototype check) —
* the schema may have been materialized from another realm; structural JSON-ness
* is what the wire needs. Cycles are rejected via `seen`.
*/
function isJsonData(value: unknown, seen: Set<object>): boolean {
if (isStructuredScalar(value)) return true
// The scalar check above already returned for null, so `object` here is a real object.
if (typeof value !== 'object') return false
if (seen.has(value)) return false
seen.add(value)
try {
if (Array.isArray(value)) return value.every(entry => isJsonData(entry, seen))
// A non-plain object (Date, Map, class instance) is NOT JSON data even when
// it has no enumerable values — it would serialize lossily, not loudly.
if (!isObjectLike(value)) return false
return Object.values(value).every(entry => isJsonData(entry, seen))
} finally {
seen.delete(value)
}
}
/** Collect subset violations for one schema node (recursive walk). */
function checkSchemaNode(node: unknown, path: string, violations: string[], seen: Set<object>): void {
if (!isObjectLike(node)) {
violations.push(`${path} must be a schema object`)
return
}
if (seen.has(node)) {
violations.push(`${path} is circular`)
return
}
seen.add(node)
for (const key of Object.keys(node)) {
if (CONSTRAINT_KEYWORDS.has(key)) continue
if (ANNOTATION_KEYWORDS.has(key)) {
if (!isJsonData(node[key], new Set())) violations.push(`${path}.${key} annotation must be JSON data`)
continue
}
violations.push(`${path}.${key} is not a supported keyword (subset: type/properties/required/additionalProperties/items/enum/const + annotations)`)
}
if (typeof node.description !== 'undefined' && typeof node.description !== 'string') {
violations.push(`${path}.description must be a string`)
}
if (typeof node.title !== 'undefined' && typeof node.title !== 'string') {
violations.push(`${path}.title must be a string`)
}
const type = node.type
if (typeof type !== 'string' || !(SCHEMA_TYPES as readonly unknown[]).includes(type)) {
violations.push(Array.isArray(type)
? `${path}.type must be a single type string (type arrays are not supported)`
: `${path}.type must be one of ${SCHEMA_TYPES.join('/')}`)
seen.delete(node)
return
}
const schemaType = type as StructuredSchemaType
// Keywords that only make sense on one type are rejected elsewhere — an
// `items` on an object (or `properties` on a string) is a schema-author bug
// the subset surfaces rather than ignores.
const allowedFor: Record<string, StructuredSchemaType[]> = {
properties: ['object'],
required: ['object'],
additionalProperties: ['object'],
items: ['array'],
enum: ['string', 'number', 'integer', 'boolean', 'null'],
const: ['string', 'number', 'integer', 'boolean', 'null'],
}
for (const [key, types] of Object.entries(allowedFor)) {
if (key in node && !types.includes(schemaType)) {
violations.push(`${path}.${key} is not supported on type "${schemaType}"`)
}
}
switch (schemaType) {
case 'object': {
const properties = node.properties
if (properties !== undefined) {
if (!isObjectLike(properties)) {
violations.push(`${path}.properties must be an object of schemas`)
} else {
for (const [key, child] of Object.entries(properties)) {
checkSchemaNode(child, `${path}.properties.${key}`, violations, seen)
}
}
}
const required = node.required
if (required !== undefined) {
if (!Array.isArray(required) || required.some(entry => typeof entry !== 'string')) {
violations.push(`${path}.required must be an array of strings`)
} else {
const declared = isObjectLike(properties) ? properties : {}
// The guard above proved every entry is a string.
for (const key of required as string[]) {
// Own-property check: `in` would let inherited names (`toString`)
// satisfy the declared-in-properties contract via the prototype.
if (!Object.hasOwn(declared, key)) violations.push(`${path}.required names "${key}" which is not in properties`)
}
}
}
if (node.additionalProperties !== undefined && typeof node.additionalProperties !== 'boolean') {
violations.push(`${path}.additionalProperties must be a boolean`)
}
break
}
case 'array': {
if (node.items !== undefined) checkSchemaNode(node.items, `${path}.items`, violations, seen)
break
}
case 'string':
case 'number':
case 'integer':
case 'boolean':
case 'null': {
const allowed = node.enum
if (allowed !== undefined) {
if (!Array.isArray(allowed) || allowed.length === 0 || !allowed.every(entry => isStructuredScalar(entry))) {
violations.push(`${path}.enum must be a non-empty array of scalars`)
}
}
if ('const' in node && !isStructuredScalar(node.const)) {
violations.push(`${path}.const must be a scalar`)
}
break
}
/* v8 ignore start -- defensive: schemaType was membership-checked against SCHEMA_TYPES above, so no runtime value reaches here */
default:
assertNever(schemaType, 'assertSupportedOutputSchema')
/* v8 ignore stop */
}
seen.delete(node)
}
/**
* Assert `schema` is a supported {@link StructuredOutputSchema} — object-rooted
* and entirely within the enforced subset. Throws {@link OutputSchemaError}
* (`UNSUPPORTED_SCHEMA`) listing EVERY violation; returns (and narrows) on
* success. Call this at the seam boundary, before any child is created.
* @param schema - the caller-supplied schema (unknown until asserted).
* @returns nothing — the assertion signature narrows `schema` to
* {@link StructuredOutputSchema} in the caller's scope on normal return.
*/
export function assertSupportedOutputSchema(schema: unknown): asserts schema is StructuredOutputSchema {
const violations: string[] = []
checkSchemaNode(schema, 'schema', violations, new Set())
if (violations.length === 0 && (schema as StructuredSchemaNode).type !== 'object') {
violations.push('schema.type must be "object" (structured output is object-rooted)')
}
if (violations.length > 0) throw new OutputSchemaError(violations)
}
/** Collect violations for one value against an (already asserted) schema node. */
function checkValue(node: StructuredSchemaNode, value: unknown, path: string): string[] {
switch (node.type) {
case 'object': {
if (!isObjectLike(value)) return [`"${path}" must be an object`]
const violations: string[] = []
const properties = node.properties ?? {}
// Own-property discipline throughout: JSON carries own enumerable
// properties only, so an inherited `toString` must not satisfy
// `required`, dodge `additionalProperties: false`, or be validated as if
// the value carried it.
for (const key of node.required ?? []) {
if (!Object.hasOwn(value, key) || value[key] === undefined) violations.push(`missing required property "${path}.${key}"`)
}
for (const [key, child] of Object.entries(properties)) {
if (!Object.hasOwn(value, key) || value[key] === undefined) continue
violations.push(...checkValue(child, value[key], `${path}.${key}`))
}
if (node.additionalProperties === false) {
for (const key of Object.keys(value)) {
if (!Object.hasOwn(properties, key)) violations.push(`"${path}.${key}" is not a declared property (additionalProperties: false)`)
}
}
return violations
}
case 'array': {
if (!Array.isArray(value)) return [`"${path}" must be an array`]
if (!node.items) return []
const items = node.items
return value.flatMap((entry, index) => checkValue(items, entry, `${path}[${index}]`))
}
case 'string': {
if (typeof value !== 'string') return [`"${path}" must be a string`]
break
}
case 'number': {
if (typeof value !== 'number' || !Number.isFinite(value)) return [`"${path}" must be a finite number`]
break
}
case 'integer': {
if (typeof value !== 'number' || !Number.isInteger(value)) return [`"${path}" must be an integer`]
break
}
case 'boolean': {
if (typeof value !== 'boolean') return [`"${path}" must be a boolean`]
break
}
case 'null': {
if (value !== null) return [`"${path}" must be null`]
break
}
default:
return assertNever(node.type, 'validateStructuredValue')
}
// Scalar constraint checks, shared by every scalar branch above.
if (node.enum && !node.enum.includes(value)) {
return [`"${path}" must be one of ${JSON.stringify(node.enum)}`]
}
if ('const' in node && value !== node.const) {
return [`"${path}" must be ${JSON.stringify(node.const)}`]
}
return []
}
/**
* Validate a value against an (already {@link assertSupportedOutputSchema}-
* asserted) schema. Returns human-readable, path-qualified violation messages
* — empty means valid. Total: never throws, however malformed the value.
* @param schema - the asserted schema to check against.
* @param value - the candidate value (e.g. parsed tool-call arguments).
* @returns every violation found, in walk order (empty = valid).
*/
export function validateStructuredValue(schema: StructuredOutputSchema, value: unknown): string[] {
return checkValue(schema, value, 'value')
}

View File

@@ -0,0 +1,304 @@
import { describe, expect, it } from 'vitest'
import {
assertSupportedOutputSchema,
OutputSchemaError,
validateStructuredValue,
type StructuredOutputSchema,
} from '../src/json-schema.ts'
/** Assert-and-narrow helper: the asserted schema, typed. */
function asserted(schema: unknown): StructuredOutputSchema {
assertSupportedOutputSchema(schema)
return schema
}
/** The violations OutputSchemaError carries for a bad schema (throws if it passes). */
function violationsOf(schema: unknown): string[] {
try {
assertSupportedOutputSchema(schema)
} catch (error: unknown) {
if (error instanceof OutputSchemaError) return error.violations
throw error
}
throw new Error('expected the schema to be rejected')
}
describe('assertSupportedOutputSchema', () => {
it('accepts a representative subset schema (all supported keywords)', () => {
const schema = asserted({
type: 'object',
description: 'a finding',
title: 'Finding',
properties: {
file: { type: 'string', description: 'path' },
line: { type: 'integer' },
severity: { type: 'string', enum: ['low', 'high'] },
kind: { type: 'string', const: 'bug' },
score: { type: 'number' },
confirmed: { type: 'boolean' },
parent: { type: 'null' },
tags: { type: 'array', items: { type: 'string' } },
nested: {
type: 'object',
properties: { x: { type: 'number', default: 3, examples: [1, 2] } },
additionalProperties: false,
},
anything: { type: 'array' },
},
required: ['file', 'line'],
additionalProperties: true,
})
expect(schema.type).toBe('object')
})
it('rejects a non-object root (scalar/array-rooted schemas)', () => {
expect(violationsOf({ type: 'string' })).toEqual(['schema.type must be "object" (structured output is object-rooted)'])
expect(violationsOf({ type: 'array', items: { type: 'string' } }))
.toContain('schema.type must be "object" (structured output is object-rooted)')
})
it('rejects non-object schema nodes and missing/unknown type', () => {
expect(violationsOf('nope')).toEqual(['schema must be a schema object'])
expect(violationsOf(null)).toEqual(['schema must be a schema object'])
expect(violationsOf([])).toEqual(['schema must be a schema object'])
expect(violationsOf({})).toEqual(['schema.type must be one of object/array/string/number/integer/boolean/null'])
expect(violationsOf({ type: 'tuple' })[0]).toMatch(/type must be one of/)
expect(violationsOf({ type: 'object', properties: { a: 'str' } })).toEqual(['schema.properties.a must be a schema object'])
})
it('rejects type ARRAYS with a dedicated message', () => {
expect(violationsOf({ type: ['string', 'null'] }))
.toEqual(['schema.type must be a single type string (type arrays are not supported)'])
})
it('rejects unsupported constraint keywords loudly (never accepted-then-ignored)', () => {
for (const keyword of ['oneOf', 'anyOf', 'allOf', 'not', 'pattern', 'minimum', 'maxLength', '$ref']) {
const bad = violationsOf({ type: 'object', [keyword]: [] })
expect(bad.some(v => v.includes(`schema.${keyword} is not a supported keyword`))).toBe(true)
}
})
it('reports EVERY violation, not just the first', () => {
const bad = violationsOf({
type: 'object',
pattern: 'x',
properties: { a: { type: 'weird' }, b: { type: 'string', minimum: 1 } },
})
expect(bad.length).toBe(3)
})
it('rejects keywords on the wrong type (items on object, properties on string, enum on object)', () => {
expect(violationsOf({ type: 'object', items: { type: 'string' } }))
.toEqual(['schema.items is not supported on type "object"'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string', properties: {} } } }))
.toEqual(['schema.properties.a.properties is not supported on type "string"'])
expect(violationsOf({ type: 'object', enum: [1] }))
.toEqual(['schema.enum is not supported on type "object"'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'array', const: 1 } } }))
.toEqual(['schema.properties.a.const is not supported on type "array"'])
})
it('validates required: must be string[] naming declared properties', () => {
expect(violationsOf({ type: 'object', required: 'file' }))
.toEqual(['schema.required must be an array of strings'])
expect(violationsOf({ type: 'object', required: [1] }))
.toEqual(['schema.required must be an array of strings'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string' } }, required: ['b'] }))
.toEqual(['schema.required names "b" which is not in properties'])
expect(violationsOf({ type: 'object', required: ['a'] }))
.toEqual(['schema.required names "a" which is not in properties'])
})
it('validates additionalProperties must be boolean and enum/const must be scalars', () => {
expect(violationsOf({ type: 'object', additionalProperties: {} }))
.toEqual(['schema.additionalProperties must be a boolean'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: [] } } }))
.toEqual(['schema.properties.a.enum must be a non-empty array of scalars'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: [{}] } } }))
.toEqual(['schema.properties.a.enum must be a non-empty array of scalars'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string', enum: 'x' } } }))
.toEqual(['schema.properties.a.enum must be a non-empty array of scalars'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'number', enum: [Number.NaN] } } }))
.toEqual(['schema.properties.a.enum must be a non-empty array of scalars'])
expect(violationsOf({ type: 'object', properties: { a: { type: 'string', const: {} } } }))
.toEqual(['schema.properties.a.const must be a scalar'])
})
it('rejects non-string description/title and non-JSON annotation payloads', () => {
expect(violationsOf({ type: 'object', description: 7 }))
.toEqual(['schema.description must be a string'])
expect(violationsOf({ type: 'object', title: 7 }))
.toEqual(['schema.title must be a string'])
expect(violationsOf({ type: 'object', default: () => 1 }))
.toEqual(['schema.default annotation must be JSON data'])
expect(violationsOf({ type: 'object', examples: [undefined] }))
.toEqual(['schema.examples annotation must be JSON data'])
expect(violationsOf({ type: 'object', examples: [Number.POSITIVE_INFINITY] }))
.toEqual(['schema.examples annotation must be JSON data'])
// A cyclic annotation payload is caught by the JSON-data walk.
const cyclicAnnotation: Record<string, unknown> = {}
cyclicAnnotation.self = cyclicAnnotation
expect(violationsOf({ type: 'object', default: cyclicAnnotation }))
.toEqual(['schema.default annotation must be JSON data'])
// Object/array annotations that ARE JSON data pass.
asserted({ type: 'object', default: { a: [1, 'x', null, true] } })
})
it('rejects a circular schema instead of recursing forever', () => {
const node: Record<string, unknown> = { type: 'object' }
node.properties = { self: node }
expect(violationsOf(node)).toEqual(['schema.properties.self is circular'])
})
it('accepts the same subschema object reused in two SIBLING positions (a DAG, not a cycle)', () => {
const leaf = { type: 'string' }
asserted({ type: 'object', properties: { a: leaf, b: leaf } })
})
it('required cannot be satisfied by INHERITED names — `toString` is not a declared property', () => {
// `'toString' in {}` is true via Object.prototype; the declared-property
// contract must be an own-property check.
expect(violationsOf({ type: 'object', properties: {}, required: ['toString'] }))
.toEqual(['schema.required names "toString" which is not in properties'])
})
it('rejects exotic host objects where the subset expects plain JSON structure', () => {
// A Map as `properties` has no own enumerable entries: structurally it
// would read as "no properties" and serialize to {} — lossy, not loud.
expect(violationsOf({ type: 'object', properties: new Map() }))
.toEqual(['schema.properties must be an object of schemas'])
// A Date node is not a schema object even though Object.values(date) is [].
expect(violationsOf({ type: 'object', properties: { at: new Date(0) } }))
.toEqual(['schema.properties.at must be a schema object'])
})
it('rejects exotic annotation payloads that would serialize lossily', () => {
expect(violationsOf({ type: 'object', default: new Date(0) }))
.toEqual(['schema.default annotation must be JSON data'])
expect(violationsOf({ type: 'object', examples: [new Map()] }))
.toEqual(['schema.examples annotation must be JSON data'])
})
})
describe('validateStructuredValue', () => {
const schema = asserted({
type: 'object',
properties: {
file: { type: 'string' },
line: { type: 'integer' },
score: { type: 'number' },
confirmed: { type: 'boolean' },
parent: { type: 'null' },
severity: { type: 'string', enum: ['low', 'high'] },
kind: { type: 'string', const: 'bug' },
tags: { type: 'array', items: { type: 'string' } },
free: { type: 'array' },
nested: { type: 'object', properties: { x: { type: 'number' } }, required: ['x'], additionalProperties: false },
},
required: ['file'],
})
it('accepts a fully valid value (empty violations)', () => {
expect(validateStructuredValue(schema, {
file: 'a.ts', line: 3, score: 0.5, confirmed: true, parent: null,
severity: 'high', kind: 'bug', tags: ['x'], free: [1, { any: true }], nested: { x: 1 },
})).toEqual([])
})
it('reports missing required and wrong root type', () => {
expect(validateStructuredValue(schema, {})).toEqual(['missing required property "value.file"'])
expect(validateStructuredValue(schema, 'nope')).toEqual(['"value" must be an object'])
expect(validateStructuredValue(schema, [])).toEqual(['"value" must be an object'])
})
it('type-checks every scalar branch with path-qualified messages', () => {
expect(validateStructuredValue(schema, { file: 1 })).toEqual(['"value.file" must be a string'])
expect(validateStructuredValue(schema, { file: 'a', line: 1.5 })).toEqual(['"value.line" must be an integer'])
expect(validateStructuredValue(schema, { file: 'a', line: 'x' })).toEqual(['"value.line" must be an integer'])
expect(validateStructuredValue(schema, { file: 'a', score: 'x' })).toEqual(['"value.score" must be a finite number'])
expect(validateStructuredValue(schema, { file: 'a', score: Number.NaN })).toEqual(['"value.score" must be a finite number'])
expect(validateStructuredValue(schema, { file: 'a', confirmed: 'yes' })).toEqual(['"value.confirmed" must be a boolean'])
expect(validateStructuredValue(schema, { file: 'a', parent: 0 })).toEqual(['"value.parent" must be null'])
})
it('enforces enum membership and const equality', () => {
expect(validateStructuredValue(schema, { file: 'a', severity: 'mid' }))
.toEqual(['"value.severity" must be one of ["low","high"]'])
expect(validateStructuredValue(schema, { file: 'a', kind: 'feature' }))
.toEqual(['"value.kind" must be "bug"'])
})
it('checks arrays per index; an items-less array accepts anything', () => {
expect(validateStructuredValue(schema, { file: 'a', tags: 'x' })).toEqual(['"value.tags" must be an array'])
expect(validateStructuredValue(schema, { file: 'a', tags: ['ok', 2] })).toEqual(['"value.tags[1]" must be a string'])
expect(validateStructuredValue(schema, { file: 'a', free: [{ deep: [1] }, null] })).toEqual([])
})
it('recurses into nested objects: required + additionalProperties: false', () => {
expect(validateStructuredValue(schema, { file: 'a', nested: {} }))
.toEqual(['missing required property "value.nested.x"'])
expect(validateStructuredValue(schema, { file: 'a', nested: { x: 1, y: 2 } }))
.toEqual(['"value.nested.y" is not a declared property (additionalProperties: false)'])
expect(validateStructuredValue(schema, { file: 'a', nested: 3 }))
.toEqual(['"value.nested" must be an object'])
})
it('a required key present-but-undefined counts as missing', () => {
expect(validateStructuredValue(schema, { file: undefined })).toEqual(['missing required property "value.file"'])
})
it('inherited properties satisfy nothing: required, additionalProperties, and recursion are own-property only', () => {
// required: ['toString'] must NOT be satisfied by Object.prototype.toString.
expect(validateStructuredValue(
asserted({ type: 'object', properties: { toString: { type: 'string' } }, required: ['toString'] }),
{},
)).toEqual(['missing required property "value.toString"'])
// additionalProperties: false must flag an OWN `toString` key even though
// `'toString' in properties` is true via the prototype.
expect(validateStructuredValue(
asserted({ type: 'object', additionalProperties: false }),
{ toString: 1 },
)).toEqual(['"value.toString" is not a declared property (additionalProperties: false)'])
// A declared property the value does NOT carry must not be validated
// against the value's INHERITED member (constructor is a function on
// every plain object's prototype, not a carried property).
expect(validateStructuredValue(
asserted({ type: 'object', properties: { constructor: { type: 'string' } } }),
{},
)).toEqual([])
})
it('a non-plain object value is not an object in the JSON sense', () => {
expect(validateStructuredValue(asserted({ type: 'object' }), new Date(0)))
.toEqual(['"value" must be an object'])
})
it('collects multiple violations across branches in one pass', () => {
expect(validateStructuredValue(schema, { line: 'x', severity: 'mid' })).toEqual([
'missing required property "value.file"',
'"value.line" must be an integer',
'"value.severity" must be one of ["low","high"]',
])
})
it('null-typed const/enum work through the scalar path', () => {
const nullish = asserted({ type: 'object', properties: { a: { type: 'null', const: null } } })
expect(validateStructuredValue(nullish, { a: null })).toEqual([])
})
it('rejects a non-object properties value in the schema walk', () => {
expect(violationsOf({ type: 'object', properties: [] }))
.toEqual(['schema.properties must be an object of schemas'])
})
it('an object schema without properties/required only type-checks its value', () => {
const bare = asserted({ type: 'object' })
expect(validateStructuredValue(bare, { any: ['thing'] })).toEqual([])
expect(validateStructuredValue(bare, 7)).toEqual(['"value" must be an object'])
})
it('validateStructuredValue throws on a type the assert would never let through (assertNever backstop)', () => {
const forged = { type: 'tuple' } as unknown as StructuredOutputSchema
expect(() => validateStructuredValue(forged, 1)).toThrow(/tuple/)
})
})

View File

@@ -8,7 +8,7 @@ A SQLite durable session-persistence backend — a second `SessionPersistence` i
Each `SessionEvent` maps 1:1 onto a row in an `events` table `(session_id, seq, type, time, data, source_event_seqs, surface_op)``data` is the event payload as JSON text, so the row shape is the event verbatim (including `assistant/chunk`, keeping `seq` contiguous). The two `TEXT` columns `source_event_seqs` and `surface_op` are nullable; they store the event's optional surface-metadata fields (see [session surface](../../../docs/rfc/implemented/architecture/2026-06-18-session-surface.md)). Out-of-log metadata (`SessionHeader`) lives in a `sessions` row. A `sessions` row is written only by the first `append` — its existence is the lazy-materialization signal (`list` reports exactly the sessions that have a row), so no separate column is needed.
The repo targets Node ≥ 24 (the root `engines` field), which includes the stable `node:sqlite` module. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and the configured `journal_mode` (default `wal`; pick a rollback-journal mode like `delete` on filesystems where WAL's shared-memory files do not work, e.g. network mounts). The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software).
The repo's `engines.node` is `^22.19.0 || >=24.0.0` (Node 22.19+ or 24+), matching the LTS floor required by the installed Pi adapter dependency; `node:sqlite` itself ships without the `--experimental-sqlite` flag from Node 22.13 (LTS) and 23.4 / 24 (Current) on. The range deliberately excludes Node 23 because that line is non-LTS/EOL and still has flagged runtime features before 23.6. The database opens with `foreign_keys = ON` (so `ON DELETE CASCADE` drops a session's events with its row) and the configured `journal_mode` (default `wal`; pick a rollback-journal mode like `delete` on filesystems where WAL's shared-memory files do not work, e.g. network mounts). The table-layout version is stored in `PRAGMA user_version` and checked on open: a fresh database is stamped with the current `SCHEMA_VERSION`; a database written by any other, incompatible build (a non-current `user_version`, older or newer) is rejected rather than opened against an unknown layout — there is no migration (unreleased software).
## Contract semantics over rows

View File

@@ -12,7 +12,7 @@ The seam this rides on: `CreateAgentOptions.seed` (added on `dsh-agent`, threade
## Capabilities
`{ outputSchema: false, depthLimit: true, toolFilter: false }` — identical to spawn (the depth/model/output behavior is the shared driver's).
`{ outputSchema: true, depthLimit: true, toolFilter: false }` — identical to spawn (the depth/model/structured-output behavior is the shared driver's).
## Config

View File

@@ -28,6 +28,10 @@ import type { SubagentCapabilities, SubagentProvider, SubagentStartRequest } fro
import { startInProcessRun } from '@deepseek-ai/dsh-subagent-inprocess'
export const name = 'subagent-fork'
// `tools` is deliberately NOT injected — same rationale as subagent-spawn: the
// per-run structured runtime gates its capture-tool registration on `tools`
// itself, so this backend's apply timing (and the delegation tool's position
// in the model-visible tool list) is unchanged by structured output.
export const inject = ['subagents', 'agents']
/** Config: the registry name to register the provider under. */
@@ -59,11 +63,12 @@ export function completedTurnPrefix(parent: Agent): SessionEvent[] {
}
/**
* The fork provider. Supports `depthLimit`; NOT `outputSchema`/`toolFilter` this
* cut (the service rejects a request needing either before `start` runs).
* The fork provider. Supports `depthLimit` and `outputSchema` (via the shared
* in-process structured runtime); NOT `toolFilter` this cut (the service
* rejects a request needing it before `start` runs).
*/
class ForkProvider implements SubagentProvider {
readonly capabilities: SubagentCapabilities = { outputSchema: false, depthLimit: true, toolFilter: false }
readonly capabilities: SubagentCapabilities = { outputSchema: true, depthLimit: true, toolFilter: false }
// Context contract: a forked child IS seeded with the parent's completed-turn prefix.
readonly inheritsParentContext = true

View File

@@ -9,9 +9,10 @@ import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
import * as Invariants from '@deepseek-ai/dsh-invariants'
import SubagentService from '@deepseek-ai/dsh-subagent'
import { MockAdapter, textResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import type { StreamChunk } from '@deepseek-ai/dsh-llm'
import * as fork from '../src/index.ts'
import { STRUCTURED_OUTPUT_TOOL } from '@deepseek-ai/dsh-subagent-inprocess'
import { completedTurnPrefix } from '../src/index.ts'
type Script = ConstructorParameters<typeof MockAdapter>[0]
@@ -141,6 +142,26 @@ describe('dsh-subagent-fork', () => {
await run.dispose()
})
it('captures structured output through the shipped plugin (seeded child, driver runtime)', async () => {
const { ctx, parent } = await setup([
textResponse('parent turn'),
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 9 }),
])
parent.send([{ type: 'text', text: 'warm up' }])
await parent.whenIdle()
const run = ctx.subagents.start('fork', {
prompt: [{ type: 'text', text: 'report structured' }],
parent,
outputSchema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] },
})
const result = await run.result
expect(result.stopReason).toBe('completed')
expect(result.structured).toEqual({ answer: 9 })
// Run-scoped runtime: nothing stays registered after the settle.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
await run.dispose()
})
it('does NOT return the seeded parent output when the child produces no message of its own', async () => {
// Regression: readResult must scope to the child's OWN events (after the
// seed). The parent completes a turn with a distinctive assistant message,
@@ -161,9 +182,9 @@ describe('dsh-subagent-fork', () => {
await run.dispose()
})
it('advertises depthLimit but not outputSchema/toolFilter', async () => {
it('advertises depthLimit and outputSchema but not toolFilter', async () => {
const { ctx } = await setup([])
expect(ctx.subagents.getProvider('fork')!.capabilities).toEqual({ outputSchema: false, depthLimit: true, toolFilter: false })
expect(ctx.subagents.getProvider('fork')!.capabilities).toEqual({ outputSchema: true, depthLimit: true, toolFilter: false })
})
it('unregisters the provider when its fiber is disposed (HMR safety)', async () => {

View File

@@ -8,10 +8,10 @@ The shared **in-process subagent run driver**. A pure library (no provider, no r
Runs a child as a child [`Agent`](../../core/agent) on the same cordis context (`ctx.agents`):
1. computes child depth = `depthOf(parent) + 1`; if `request.maxDepth` is set and exceeded, throws `SubagentDepthError` (the `depthLimit` capability);
2. creates a child via `ctx.agents.create` with a fresh `AgentId`/`SessionId`, the parent's `cwd` + `parentSession` lineage, the optional `options.seed` (fork's completed-turn prefix; omitted for a fresh child), and `agentOptions` (the child inherits the **parent's model** by default — a child with no model can't run — overridable via `request.agentOptions.model`; the system prompt is NOT inherited);
3. drives the one-shot: `child.send(prompt)` then `await child.whenIdle()` (ordering matters — `send` enqueues synchronously, so `whenIdle` observes the queued work and resolves on the child's `running → idle` transition, never before the turn starts);
4. reads the result, scoped to the child's OWN events (everything at or after `seedLength`, so a seeded child that produced no message of its own never returns the seeded parent's last message): the last `assistant/message` content (deep-cloned — the log is frozen) and the last `turn/end.reason` mapped to a `SubagentStopReason`.
1. computes child depth = `depthOf(parent) + 1`; if `request.maxDepth` is set and exceeded, throws `SubagentDepthError` (the `depthLimit` capability); a `request.outputSchema` is asserted against the supported subset (`assertSupportedOutputSchema` from [dsh-tools](../../core/tools/README.md)) and then snapshotted with `structuredClone` before any child exists — assertion first so a hostile value fails as `OutputSchemaError` (never a raw clone error), the snapshot so a post-`start()` caller mutation cannot drift the enforced schema;
2. creates a child via `ctx.agents.create` with a fresh `AgentId`/`SessionId`, the parent's `cwd` + `parentSession` lineage, the optional `options.seed` (fork's completed-turn prefix; omitted for a fresh child), and `agentOptions` (the child inherits the **parent's model** by default — a child with no model can't run — overridable via `request.agentOptions.model`; the deployment persona needs no inheritance — it is a context-wide prompt section);
3. drives the one-shot: `child.send(prompt)` then `await child.whenIdle()` (ordering matters — `send` enqueues synchronously, so `whenIdle` observes the queued work and resolves on the child's `running → idle` transition, never before the turn starts); there is deliberately NO re-prompt for a structured child that finished cleanly without calling `structured_output` — the shortfall maps to an `error` result for the parent;
4. reads the result, scoped to the child's OWN events (everything at or after `seedLength`, so a seeded child that produced no message of its own never returns the seeded parent's last message): the last `assistant/message` content (deep-cloned — the log is frozen) and the last `turn/end.reason` mapped to a `SubagentStopReason`. A structured run surfaces the captured value as `result.structured`; a structured child that finished cleanly WITHOUT ever capturing settles `error` (a clean finish without the demanded result is a failure, not a success with a missing field).
`dispose()` delegates to `AgentHandle.dispose()` (stop loop → await quiescence → remove session); `cancel()` cancels the child's in-flight turn. A cancel landing before any `turn/end` (the pre-turn window) still settles `aborted`, honoring the cancel contract rather than the generic no-turn `error`.
@@ -19,6 +19,19 @@ Runs a child as a child [`Agent`](../../core/agent) on the same cordis context (
`{ providerName: string; seed?: SessionEvent[] }` — the per-backend inputs: the provider name (for error context) and the optional child-session seed.
### Structured output (package-internal runtime)
The mechanism behind `outputSchema` for in-process children — acquired per structured RUN inside `startInProcessRun` (nothing is registered on a context that never runs a structured child; only the model-facing constants `STRUCTURED_OUTPUT_TOOL`/`STRUCTURED_OUTPUT_INSTRUCTION` are exported). One globally registered `structured_output` capture tool (its registered parameters are a placeholder) plus four listeners:
- a `system-prompt/assemble` waterfall listener registered `prepend: true` that post-processes `await next()`**final-assembly enforcement**: the assembly the loop renders never carries `structured_output` for an agent without a structured run, and for one that has it always carries the run's OWN schema (as the tool's `parameters`) plus the calling instruction as a trailing prompt section (the demand travels with the tool — `AgentOptions` has no per-agent prompt field to carry it). The loop logs the rendered assembly as the step's `request/header`, so the injection is reconstructable log state, never a wire-only mutation. Per-agent shaping lives here because the tool registry and prompt assembly are context-global while schemas differ per concurrent child (FIXME in the module doc: per-agent/per-session scoping would dissolve this); cooperative mutate-then-`next()` would not survive a downstream listener returning a replacement assembly.
- a `tools/post-execute` listener (`prepend: true` = outermost, so `await next()` yields the composed final decision) that COMMITS the capture: the tool body only stages the validated value, and it becomes the run's result only when the final decision accepts the call — a downstream block (a PostToolUse hook) turns the logged result into `isError`, and the run must not report `structured` success for a call the model and session log saw fail.
- a `tools/pre-execute` deny for any call arriving after the agent's capture — terminal means terminal WITHIN the step: a response listing `structured_output` before further tool calls cannot run side effects after the final answer was accepted.
- an `agent/turn-continuation` listener (also `prepend: true` — an earlier-registered force-continue listener returning without `next()` must not decide the turn before the veto runs) that stops a child's turn once its output is captured, so a successful capture doesn't buy a wasted extra model step.
The capture tool validates each call against the run's schema (`validateStructuredValue`) — violations become an `INVALID_ARGS` isError result the model retries in-turn; a valid call stages the value for the post-execute commit.
Lifetime is refcounted by live structured runs: each acquires at start and releases at settle, so the registrations exist exactly while at least one structured child is live, a backend hot-reload mid-run cannot unregister the capture tool under a live child, and the last settle disposes everything. `release()` is idempotent per acquisition.
### `depthOf(agent): number`
Delegation depth rides on a merge-extensible `AgentOptions.subagentDepth` field (0 for a top-level agent, parent + 1 for a child), so a nested spawn reads its parent's depth from `parent.options.subagentDepth`. `depthOf` reads it (absent ⇒ 0).

View File

@@ -26,6 +26,8 @@
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-subagent": "^0.0.1",
"@deepseek-ai/dsh-system-prompt": "^0.0.1",
"@deepseek-ai/dsh-tools": "^0.0.1",
"cordis": "^4.0.0-rc.6"
},
"devDependencies": {

View File

@@ -18,7 +18,20 @@ import type { Context } from 'cordis'
import { AgentId, type Agent, type AgentHandle, type AgentOptions } from '@deepseek-ai/dsh-agent'
import { SessionId, type SessionEvent, type TurnEndReason } from '@deepseek-ai/dsh-session'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { assertSupportedOutputSchema } from '@deepseek-ai/dsh-tools'
import type { SubagentResult, SubagentRun, SubagentStartRequest, SubagentStopReason } from '@deepseek-ai/dsh-subagent'
import {
acquireStructuredRuntime,
type StructuredAcquisition,
} from './structured.ts'
// The runtime itself (acquire/attach/release) is package-internal: runs
// acquire it inside startInProcessRun, and no other package drives it. Only
// the model-facing vocabulary is public.
export {
STRUCTURED_OUTPUT_TOOL,
STRUCTURED_OUTPUT_INSTRUCTION,
} from './structured.ts'
declare module '@deepseek-ai/dsh-agent' {
interface AgentOptions {
@@ -109,6 +122,18 @@ export function startInProcessRun(
if (request.maxDepth !== undefined && childDepth > request.maxDepth) {
throw new SubagentDepthError(childDepth, request.maxDepth)
}
// Assert, then snapshot, the schema subset BEFORE any child exists (the
// service has already capability-gated; this rejects a schema outside the
// enforced subset loud). Assertion comes FIRST so a hostile value fails as
// OutputSchemaError, never as structuredClone's raw DataCloneError — the
// asserted subset is plain JSON data, which always clones. The snapshot is
// load-bearing: the caller keeps its reference, so attaching the ORIGINAL
// would let a post-start() mutation drift the enforced schema away from the
// asserted one — the clone (taken synchronously with the assertion, no
// interleaving possible) pins assertion, the model-visible parameters, and
// validateStructuredValue to one isolation-immutable value.
if (request.outputSchema !== undefined) assertSupportedOutputSchema(request.outputSchema)
const schema = request.outputSchema === undefined ? undefined : structuredClone(request.outputSchema)
const childId = AgentId(randomUUID())
// The child's OWN events begin after the seed (fork seeds the parent's
@@ -120,13 +145,20 @@ export function startInProcessRun(
// Inherit the parent's model by default (a child with no model cannot run);
// an explicit `request.agentOptions.model` overrides it. The persona needs
// no inheritance: the deployment persona is a context-wide prompt section,
// so parent and child render the same one.
// so parent and child render the same one. A structured run's
// structured_output instruction is NOT prompt state either — the structured
// runtime's final-request listener appends it per request (see structured.ts).
const agentOptions: AgentOptions = {
...request.parent.options.model !== undefined ? { model: request.parent.options.model } : {},
...request.agentOptions,
subagentDepth: childDepth,
}
// The structured runtime is held for the WHOLE run (acquired before the child
// exists, released when the result settles), so a backend hot-reload mid-run
// cannot unregister the capture tool out from under this live child.
const structured: StructuredAcquisition | undefined = schema !== undefined ? acquireStructuredRuntime(ctx) : undefined
const handle: AgentHandle = ctx.agents.create({
agentId: childId,
sessionId: SessionId(randomUUID()),
@@ -141,6 +173,7 @@ export function startInProcessRun(
agentOptions,
})
const child = handle.agent
if (structured && schema !== undefined) structured.attach(child, schema)
// Bridge the request's abort signal to the child (the consumer also bridges
// its own exec.signal, but a backend-level bridge keeps the contract local).
@@ -149,6 +182,10 @@ export function startInProcessRun(
// `turn/end` is logged — settles as `aborted` (honoring the cancel contract)
// rather than falling through to the no-turn `error` mapping.
let cancelled = false
// An accessor, not an inline read: `cancelled` mutates from closures (the
// abort listener, run.cancel), which control-flow narrowing cannot see — an
// inline read at the result mapping would narrow to the initializer.
const isCancelled = (): boolean => cancelled
const requestCancel = (reason: string): void => {
cancelled = true
child.cancel(reason)
@@ -165,9 +202,16 @@ export function startInProcessRun(
if (request.signal?.aborted) return { output: [], stopReason: 'aborted' }
child.send(request.prompt)
await child.whenIdle()
return readResult(child, seedLength, cancelled)
// Deliberately NO re-prompt when a structured child finishes cleanly
// without calling structured_output: readResult maps that to `error` —
// the shortfall goes to the parent instead of buying extra model turns.
return readResult(child, seedLength, isCancelled(), structured ? { captured: structured.captured(child) } : undefined)
} finally {
request.signal?.removeEventListener('abort', onAbort)
if (structured) {
structured.detach(child)
structured.release()
}
}
})()
@@ -195,12 +239,32 @@ export function startInProcessRun(
* logged (a cancel landed in the pre-turn window, before any turn ran), the
* run settles `aborted` per the {@link SubagentRun.cancel} contract rather than
* the generic no-turn `error`.
*
* A structured run (`structured` present) additionally reports the captured
* value on {@link SubagentResult.structured}. A structured child that finished
* CLEANLY without ever capturing (the nudges ran out) settles `error` — a clean
* finish without the demanded structured result is a failure, not a success
* with a missing field; a non-`completed` reason keeps its own honest mapping.
*/
function readResult(child: Agent, seedLength: number, cancelled: boolean): SubagentResult {
function readResult(
child: Agent,
seedLength: number,
cancelled: boolean,
structured?: { captured?: { value: unknown } | undefined },
): SubagentResult {
const own = child.session.events.slice(seedLength)
const lastMessage = own.findLast((e): e is SessionEvent<'assistant/message'> => e.type === 'assistant/message')
const lastEnd = own.findLast((e): e is SessionEvent<'turn/end'> => e.type === 'turn/end')
const output: ContentBlock[] = lastMessage ? structuredClone(lastMessage.data.content) : []
if (lastEnd === undefined && cancelled) return { output, stopReason: 'aborted' }
return { output, stopReason: toStopReason(lastEnd?.data.reason) }
const stopReason: SubagentStopReason = lastEnd === undefined && cancelled
? 'aborted'
: toStopReason(lastEnd?.data.reason)
if (structured) {
if (structured.captured) return { output, structured: structured.captured.value, stopReason }
// No capture on a cleanly-completed turn: an ERROR when the run was left
// to finish (the nudges ran out), but ABORTED when a cancel is why the
// nudging stopped — the cancel contract outranks the schema shortfall.
if (stopReason === 'completed') return { output, stopReason: cancelled ? 'aborted' : 'error' }
}
return { output, stopReason }
}

View File

@@ -0,0 +1,312 @@
/**
* Structured-output support for the in-process subagent backends: the mechanism
* behind `SubagentStartRequest.outputSchema` for children that run as agents on
* the same context.
*
* The model-facing surface is one globally registered `structured_output` tool
* whose REGISTERED parameters are a placeholder — the real schema is per run.
* Because the tool registry and prompt assembly are context-global while
* schemas differ per child (two concurrent structured runs may carry different
* schemas), per-agent shaping happens on the `system-prompt/assemble`
* waterfall with a `prepend: true` listener that post-processes `await next()`
* — FINAL-ASSEMBLY enforcement: whatever downstream listeners mutated or
* replaced, the assembly the loop renders never carries `structured_output`
* for an agent without a structured run, and for one that has it always
* carries the run's OWN schema plus a trailing
* {@link STRUCTURED_OUTPUT_INSTRUCTION} section (the demand travels with the
* tool). The loop logs what the assembly produced as the request header, so
* the injection is a reconstructable fact of the session log, never a
* wire-only mutation (the reconstructability RFC).
* (Cooperative mutate-then-`next()` would not survive a downstream listener
* returning a replacement assembly — see the waterfall composition caveat in
* docs/architecture.md.)
*
* FIXME: the whole enforcement dance above exists because the tool registry
* and prompt assembly are context-global. If they become per-agent or
* per-session scoped, a structured run just registers its own schema'd tool on
* the child's scope and this module reduces to the capture tool plus the
* turn-stop — no placeholder, no final-assembly swap, no strip-for-everyone-
* else, no global-registration lifetime dance.
*
* A companion `agent/turn-continuation` listener stops a child's turn once its
* output is captured — without it, the loop's default "had tool calls ⇒
* continue" buys a wasted extra model step per structured child. It is also
* `prepend: true`: the veto must run before any earlier-registered listener
* that could short-circuit the chain into a forced continue. A third listener
* closes the within-step window the continuation veto cannot: a
* `tools/pre-execute` deny for any call arriving after the agent's capture, so
* a response that lists `structured_output` before further tool calls cannot
* run side effects after the final answer was accepted. A fourth,
* `tools/post-execute`, is the capture COMMIT: the tool body only stages the
* validated value, and it becomes the run's captured result only when the
* final post-execute decision accepts the call — a blocking hook downstream
* yields `isError` in the log, and the run must not report success for it.
*
* Lifetime is refcounted by structured RUNS: each acquires from start to
* settle, so the registrations exist exactly while at least one structured
* child is live — a plain deployment that never passes `outputSchema` carries
* no always-on global state, and a backend hot-reload mid-run cannot
* unregister the capture tool out from under a live child (the run holds its
* own acquisition). Registrations land on the ROOT context and the refcount
* disposes them when the last run settles; the next structured run
* re-registers them.
*
* @module @deepseek-ai/dsh-subagent-inprocess/structured
*/
import type { Context } from 'cordis'
import type { Agent } from '@deepseek-ai/dsh-agent'
import type { ContentBlock, ToolSchema } from '@deepseek-ai/dsh-llm'
import type { ContinuationDecision } from '@deepseek-ai/dsh-agent'
import type { AssembleContext, PromptAssembly } from '@deepseek-ai/dsh-system-prompt'
import type { PostToolDecision, PreToolDecision, ToolExecution, ToolExecutionResult } from '@deepseek-ai/dsh-tools'
import { ToolArgsError, validateStructuredValue, type StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
/** The model-facing tool name a structured child must call to finish. */
export const STRUCTURED_OUTPUT_TOOL = 'structured_output'
/**
* The instruction the assembly listener appends to a structured child's
* system prompt as a trailing section on every assembly. Per-assembly state,
* NOT agent prompt state: `AgentOptions` has no prompt field (the persona is
* deployment config on the system-prompt plugin), so the same final-assembly
* enforcement that injects the schema'd tool carries the instruction that
* demands calling it.
*/
export const STRUCTURED_OUTPUT_INSTRUCTION
= 'When you have your final answer, you MUST report it by calling the '
+ `\`${STRUCTURED_OUTPUT_TOOL}\` tool with arguments matching its parameter schema exactly. `
+ 'Do not finish with a plain text answer: only the tool call counts as your result.'
/** One structured run's state: the schema to enforce and the captured value, once recorded. */
interface RunState {
readonly schema: StructuredOutputSchema
/**
* A validated value awaiting the post-execute verdict on ITS OWN call. Set
* by the capture tool's body, promoted to {@link RunState.captured} only
* when the final `tools/post-execute` decision accepts the call — a
* downstream block turns the logged result into `isError`, and a value
* committed at body time would let the run report success for a call the
* model saw fail.
*/
pending?: { value: unknown }
captured?: { value: unknown }
}
/** The per-root-context runtime: run states plus the shared registrations. */
interface StructuredRuntime {
refs: number
readonly states: WeakMap<Agent, RunState>
readonly disposers: (() => void)[]
}
/** One root context ⇒ one runtime (multi-app test isolation). */
const runtimes = new WeakMap<Context, StructuredRuntime>()
/**
* One holder's handle on the shared structured runtime. `release()` is
* idempotent per acquisition; the runtime's registrations are disposed when the
* LAST holder (backend plugin or live run) releases.
*/
export interface StructuredAcquisition {
/** Enforce `schema` on `agent`'s requests and start capturing its `structured_output` call. */
attach(agent: Agent, schema: StructuredOutputSchema): void
/** The captured value, once the child called the tool with valid arguments. */
captured(agent: Agent): { value: unknown } | undefined
/** Stop enforcing/capturing for `agent` (WeakMap-backed; safe to call twice). */
detach(agent: Agent): void
/** Drop this holder's reference (idempotent); the last release unregisters everything. */
release(): void
}
/**
* Acquire the per-root-context structured runtime, registering the capture tool
* and the runtime's listeners on the FIRST acquisition. See the module doc
* for the enforcement and lifetime design.
* @param ctx - any context of the app; the runtime keys off `ctx.root`.
* @returns this holder's handle (attach/captured/detach + idempotent release).
*/
export function acquireStructuredRuntime(ctx: Context): StructuredAcquisition {
const root: Context = ctx.root
let runtime = runtimes.get(root)
if (!runtime) {
runtime = { refs: 0, states: new WeakMap(), disposers: [] }
runtimes.set(root, runtime)
registerRuntime(root, runtime)
}
runtime.refs += 1
let released = false
return {
attach(agent: Agent, schema: StructuredOutputSchema): void {
runtime.states.set(agent, { schema })
},
captured(agent: Agent): { value: unknown } | undefined {
return runtime.states.get(agent)?.captured
},
detach(agent: Agent): void {
runtime.states.delete(agent)
},
release(): void {
if (released) return
released = true
runtime.refs -= 1
if (runtime.refs > 0) return
runtimes.delete(root)
for (const dispose of runtime.disposers.splice(0)) dispose()
},
}
}
/** Register the capture tool + the two listeners on the root context (first acquire). */
function registerRuntime(root: Context, runtime: StructuredRuntime): void {
// The registered parameters are a PLACEHOLDER: the request listener below
// swaps in the run's real schema per child, and strips the tool entirely for
// every agent without a structured run — so this shape is never model-visible.
//
// Registration does NOT ride on the acquiring backend's plugin-level
// `inject`: a backend that waited on `tools` would apply later than it did
// before this module existed, shifting when its PROVIDER registers — and the
// delegation tool mirrors provider lifecycle, so that shift would reorder
// the model-visible tool list of every existing prompt. Instead the capture
// tool registers synchronously when `tools` is already live (the common
// case), and through a scoped inject fiber when the Loader happens to start
// the backend first. Either way the registration lands on root and is
// disposed by the runtime's refcount; disposing the fiber also covers the
// never-activated case.
let disposeTool: (() => void) | undefined
const registerCapture = (tools: Context['tools']): void => {
disposeTool = tools.register({
name: STRUCTURED_OUTPUT_TOOL,
description:
'Report your final structured result. Call this exactly once, when your answer is complete; '
+ 'the arguments must match this tool\'s parameter schema exactly.',
parameters: { type: 'object', properties: {} },
execute(args: unknown, exec: ToolExecution): Promise<ContentBlock[]> {
const state = exec.agent ? runtime.states.get(exec.agent) : undefined
if (!state) {
// Reachable only if a non-structured agent somehow calls the tool (the
// request listener strips it, so the model never sees it) — fail loud
// rather than capture into nowhere.
throw new Error(`${STRUCTURED_OUTPUT_TOOL} is only available to subagents started with an output schema`)
}
const violations = validateStructuredValue(state.schema, args)
// ToolArgsError → isError result with INVALID_ARGS: the model retries
// within the same turn, exactly like a schema-validated defineTool call.
if (violations.length > 0) throw new ToolArgsError(violations)
// Two-phase commit: the body only STAGES the value; the post-execute
// listener below promotes it once the final decision accepts the call.
state.pending = { value: args }
return Promise.resolve([{ type: 'text', text: 'Structured output recorded.' }])
},
})
}
const liveTools = root.get('tools')
const toolsFiber = liveTools ? undefined : root.inject(['tools'], (childCtx: Context) => {
registerCapture(childCtx.root.tools)
})
if (liveTools) registerCapture(liveTools)
runtime.disposers.push(() => {
disposeTool?.()
void toolsFiber?.dispose()
})
// FINAL-ASSEMBLY enforcement (prepend: true = first registered = OUTERMOST
// wrapper): post-process whatever the downstream listeners and the registry
// produced, so a downstream listener returning a replacement assembly cannot
// leak the tool to other agents or erase the child's schema. The loop logs
// the rendered assembly as the step's request header, so the swap is
// reconstructable log state, never a wire-only mutation.
runtime.disposers.push(root.on('system-prompt/assemble', async function (
this: unknown, _assembly: PromptAssembly, context: AssembleContext, next: () => Promise<PromptAssembly>,
): Promise<PromptAssembly> {
const final = await next()
const state = context.agent ? runtime.states.get(context.agent) : undefined
if (state) {
const schemaEntry: ToolSchema = {
name: STRUCTURED_OUTPUT_TOOL,
description:
'Report your final structured result. Call this exactly once, when your answer is complete; '
+ 'the arguments must match this tool\'s parameter schema exactly.',
// ToolSchema.parameters is the wire-level JSON Schema object; the
// asserted subset type is structurally exactly that.
parameters: state.schema as unknown as Record<string, unknown>,
}
final.tools = [...final.tools.filter(tool => tool.name !== STRUCTURED_OUTPUT_TOOL), schemaEntry]
// The demand travels WITH the tool: a trailing section in the
// tool-guidance order band, appended after next() so it renders last
// (renderPrompt joins in array order).
final.sections = [...final.sections, { name: `tool:${STRUCTURED_OUTPUT_TOOL}`, order: 190, text: STRUCTURED_OUTPUT_INSTRUCTION }]
return final
}
// No structured run: strip the placeholder so it is never model-visible.
// An empty tools array canonicalizes to an absent header/wire field
// (canonicalHeader pins empty ≡ absent), so no re-shaping is needed here.
final.tools = final.tools.filter(tool => tool.name !== STRUCTURED_OUTPUT_TOOL)
return final
}, { prepend: true }))
// Stop a structured child's turn once its output is captured: the default
// "had tool calls ⇒ continue" would otherwise buy a wasted extra model step
// after every successful capture. `prepend: true` puts the veto OUTERMOST —
// an earlier-registered listener that short-circuits the chain (a goal-style
// force-continue returning without `next()`) would otherwise decide the turn
// before this listener ever ran, and no downstream decision may resurrect a
// structured turn that is already finished.
runtime.disposers.push(root.on('agent/turn-continuation', function (
this: unknown, agent: Agent, _turn: number, _decision: ContinuationDecision, next: () => Promise<ContinuationDecision>,
): Promise<ContinuationDecision> {
if (runtime.states.get(agent)?.captured) return Promise.resolve({ action: 'stop' })
return next()
}, { prepend: true }))
// The capture COMMIT: promote the staged value only when the final
// post-execute decision accepts the call. The capture tool's body cannot
// decide — `tools/post-execute` runs after it, and a blocking listener (a
// PostToolUse hook) turns the logged result into `isError` feedback; a value
// committed at body time would make readResult report `structured` success
// for a call whose result the model and session log saw fail. `prepend:
// true` = outermost at registration time, so `await next()` returns the
// COMPOSED downstream decision — the same final verdict the registry maps
// onto the result. (A later-registered outer listener that blocks without
// delegating skips this commit entirely: the staged value is dropped and the
// run errors — failure-safe in the same direction.) The staging slot clears
// on every path, including a rejecting downstream listener.
runtime.disposers.push(root.on('tools/post-execute', async function (
this: unknown, exec: ToolExecution, _result: ToolExecutionResult, next: () => Promise<PostToolDecision>,
): Promise<PostToolDecision> {
const state = exec.agent ? runtime.states.get(exec.agent) : undefined
if (!state || exec.name !== STRUCTURED_OUTPUT_TOOL || state.pending === undefined) return next()
const pending = state.pending
try {
const decision = await next()
if (decision.kind === 'accept') state.captured = pending
return decision
} finally {
delete state.pending
}
}, { prepend: true }))
// Terminal means terminal WITHIN the step, not only at its end: the
// turn-continuation veto above runs after every call in the current model
// response has executed, so a response that puts `structured_output` before
// further tool calls would still perform those side effects after the final
// answer was accepted. Deny every later call for a captured agent at the
// allow/deny gate — dispatch is skipped and the model sees an `isError`
// result naming the contract. Calls that PRECEDE the capture in the same
// response ran before `captured` was set and are untouched; a second
// `structured_output` is denied like any other call. `prepend: true` for the
// same reason as the continuation veto: no earlier-registered allow may
// short-circuit past the terminal contract.
runtime.disposers.push(root.on('tools/pre-execute', function (
this: unknown, exec: ToolExecution, next: () => Promise<PreToolDecision>,
): Promise<PreToolDecision> {
if (exec.agent && runtime.states.get(exec.agent)?.captured) {
return Promise.resolve({
kind: 'deny',
reason: `structured output already recorded: the run is complete, so \`${exec.name}\` is not executed`,
})
}
return next()
}, { prepend: true }))
}

View File

@@ -0,0 +1,606 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import LlmService, { CallId, type ContentBlock, type GenerateOptions } from '@deepseek-ai/dsh-llm'
import SessionStore from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry from '@deepseek-ai/dsh-tools'
import AgentRegistry, { AgentId } from '@deepseek-ai/dsh-agent'
import type { Agent, ContinuationDecision } from '@deepseek-ai/dsh-agent'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
import * as Invariants from '@deepseek-ai/dsh-invariants'
import SubagentService, { type SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
import { MockAdapter, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import { startInProcessRun } from '../src/index.ts'
import {
acquireStructuredRuntime,
STRUCTURED_OUTPUT_INSTRUCTION,
STRUCTURED_OUTPUT_TOOL,
} from '../src/structured.ts'
type Script = ConstructorParameters<typeof MockAdapter>[0]
const SCHEMA: StructuredOutputSchema = {
type: 'object',
properties: { answer: { type: 'number' }, note: { type: 'string' } },
required: ['answer'],
}
/**
* Real loop + scripted mock model + an INLINE spawn-shaped provider over the
* shared driver. The concrete backend plugins are deliberately NOT loaded —
* they would devDep-cycle this package (spawn/fork already depend on the
* driver), and the runtime under test is the driver's; plugin-level structured
* coverage lives in the spawn/fork specs. The mock model script drives the
* child's structured_output calls.
*/
async function setup(script: Script) {
const ctx = new Context()
const adapter = new MockAdapter(script)
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await ctx.plugin(AgentRegistry)
await ctx.plugin(Invariants)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(SubagentService)
const disposeProvider = ctx.subagents.registerProvider({
name: 'spawn',
capabilities: { outputSchema: true, depthLimit: true, toolFilter: false },
inheritsParentContext: false,
start: (request: SubagentStartRequest) => startInProcessRun(ctx, request, { providerName: 'spawn' }),
})
ctx.llm.registerAdapter(['mock'], adapter)
const parent = ctx.agentLoop.create(AgentId('parent'), { model: 'mock' })
return { ctx, parent, adapter, disposeProvider }
}
function structuredRequest(parent: SubagentStartRequest['parent'], extra?: Partial<SubagentStartRequest>): SubagentStartRequest {
return { prompt: [{ type: 'text', text: 'produce the answer' }], parent, outputSchema: SCHEMA, ...extra }
}
/** The tool names of one recorded model request. */
function toolNames(request: GenerateOptions): string[] {
return (request.tools ?? []).map(tool => tool.name)
}
describe('in-process structured output', () => {
it('captures a valid structured_output call and surfaces result.structured', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42, note: 'done' }),
])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.stopReason).toBe('completed')
expect(result.structured).toEqual({ answer: 42, note: 'done' })
await run.dispose()
})
it('stops the turn after a successful capture — no extra model step is spent', async () => {
const { ctx, parent, adapter } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }),
textResponse('MUST NOT BE CONSUMED'),
])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
await run.result
// Default continuation would run a second step after the tool call; the
// structured runtime's turn-continuation veto stops the turn instead.
expect(adapter.requests.length).toBe(1)
await run.dispose()
})
it('denies tool calls that FOLLOW the capture in the same response — terminal means terminal', async () => {
// One model response carrying structured_output FIRST and a side-effecting
// call after it: the continuation veto only fires at step end, so without
// the pre-execute deny the trailing call would still run after the final
// answer was accepted.
const response = [
...toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 5 }).slice(0, -2),
{ type: 'block-start', index: 1, blockType: 'tool-call' },
{ type: 'block-end', index: 1, block: { type: 'tool-call', id: CallId('c2'), name: 'side_effect', arguments: '{}' } },
{ type: 'usage', usage: { inputTokens: 10, outputTokens: 5 } },
{ type: 'finish', reason: { kind: 'tool-calls' } },
] as Script[number]
const { ctx, parent } = await setup([response])
let sideEffectRan = false
ctx.tools.register({
name: 'side_effect',
description: 'probe',
parameters: { type: 'object', properties: {} },
execute(): Promise<ContentBlock[]> {
sideEffectRan = true
return Promise.resolve([{ type: 'text', text: 'ran' }])
},
})
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.stopReason).toBe('completed')
expect(result.structured).toEqual({ answer: 5 })
// The deny skipped dispatch entirely: the probe body never ran.
expect(sideEffectRan).toBe(false)
await run.dispose()
})
it('leaves tool calls that PRECEDE the capture in the same response untouched', async () => {
const response = [
{ type: 'block-start', index: 0, blockType: 'tool-call' },
{ type: 'block-end', index: 0, block: { type: 'tool-call', id: CallId('c1'), name: 'side_effect', arguments: '{}' } },
...toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 6 }).map(chunk =>
'index' in chunk ? { ...chunk, index: 1 } : chunk),
] as Script[number]
const { ctx, parent } = await setup([response])
let sideEffectRan = false
ctx.tools.register({
name: 'side_effect',
description: 'probe',
parameters: { type: 'object', properties: {} },
execute(): Promise<ContentBlock[]> {
sideEffectRan = true
return Promise.resolve([{ type: 'text', text: 'ran' }])
},
})
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
// The call ran BEFORE captured was set: the deny gate only guards the
// window after the terminal answer landed.
expect(sideEffectRan).toBe(true)
expect(result.structured).toEqual({ answer: 6 })
await run.dispose()
})
it('snapshots the schema at start(): caller mutation after start cannot drift enforcement', async () => {
const mutable: StructuredOutputSchema = {
type: 'object',
properties: { answer: { type: 'number' } },
required: ['answer'],
additionalProperties: false,
}
const pristine = structuredClone(mutable)
const { ctx, parent, adapter } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 3 }),
])
const run = ctx.subagents.start('spawn', structuredRequest(parent, { outputSchema: mutable }))
// Mutate the caller's object AFTER start() returned but before the child's
// first request assembles: with a live reference this would reach both the
// model-visible parameters and validateStructuredValue.
;(mutable.properties as Record<string, unknown>).answer = { type: 'string' }
const result = await run.result
expect(result.structured).toEqual({ answer: 3 })
// The child's request carried the PRISTINE schema, not the mutated one.
const childRequest = adapter.requests.at(-1)
const captureTool = (childRequest?.tools ?? []).find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)
expect(captureTool?.parameters).toEqual(pristine)
await run.dispose()
})
it('the captured-turn veto is prepend: an EARLIER force-continue listener cannot short-circuit it', async () => {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
// Registered BEFORE the structured runtime exists — without prepend, this
// goal-style listener would decide the turn first (returning WITHOUT
// calling next()) and the veto would never run.
ctx.on('agent/turn-continuation', () => Promise.resolve<ContinuationDecision>({ action: 'continue' }))
const acquisition = acquireStructuredRuntime(ctx)
const agent = { id: AgentId('structured-child') } as unknown as Agent
acquisition.attach(agent, SCHEMA)
const captured = await ctx.tools.execute({
callId: 'call-1' as never,
name: STRUCTURED_OUTPUT_TOOL,
arguments: { answer: 1 },
agent,
})
expect(captured.isError).toBeFalsy()
const decision = await ctx.waterfall(
'agent/turn-continuation', agent, 1,
{ action: 'continue' },
() => Promise.resolve<ContinuationDecision>({ action: 'continue' }),
)
expect(decision).toEqual({ action: 'stop' })
acquisition.detach(agent)
acquisition.release()
})
it('an invalid call gets an INVALID_ARGS isError result and the model retries in-turn', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 'not-a-number' }),
toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 7 }),
])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.structured).toEqual({ answer: 7 })
expect(result.stopReason).toBe('completed')
// The child's log carries the isError tool/result for the invalid call.
const child = ctx.agents.get(run.id)!
const results = child.session.events.filter(e => e.type === 'tool/result')
expect(results.length).toBe(2)
expect((results[0]!.data as { isError?: boolean }).isError).toBe(true)
await run.dispose()
})
it('a clean finish without a capture is an immediate error to the parent — deliberately NO re-prompt', async () => {
const { ctx, parent, adapter } = await setup([
textResponse('here is my answer in prose'),
textResponse('MUST NOT BE CONSUMED'),
])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.stopReason).toBe('error')
expect(result.structured).toBeUndefined()
// Exactly one model request and one user message: no nudge turn exists.
expect(adapter.requests.length).toBe(1)
const child = ctx.agents.get(run.id)!
expect(child.session.events.filter(e => e.type === 'user/message').length).toBe(1)
await run.dispose()
})
it('an errored child keeps its honest error result (no capture expected)', async () => {
// Script exhaustion on the first call → the child turn errors.
const { ctx, parent, adapter } = await setup([])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.stopReason).toBe('error')
expect(adapter.requests.length).toBe(1)
await run.dispose()
})
it('a cancel landing after a clean capture-less turn settles aborted, not error', async () => {
const { ctx, parent } = await setup([textResponse('prose, no capture')])
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const child = ctx.agents.get(run.id)!
// Cancel synchronously inside the turn's end recording: the cancel
// contract outranks the schema shortfall, so the result maps to aborted.
ctx.on('session/event', (session, event) => {
if (session === child.session && event.type === 'turn/end') run.cancel('cancelled at turn end')
})
const result = await run.result
expect(result.stopReason).toBe('aborted')
await run.dispose()
})
it('rejects a schema outside the subset loud, before any child exists', async () => {
const { ctx, parent } = await setup([])
expect(() => ctx.subagents.start('spawn', structuredRequest(parent, {
outputSchema: { type: 'object', oneOf: [] } as unknown as StructuredOutputSchema,
}))).toThrow(/unsupported output schema/)
expect(ctx.agents.get(AgentId('parent'))).toBeDefined()
})
it('a schema carrying non-JSON values fails as OutputSchemaError, never as a raw clone error', async () => {
const { ctx, parent } = await setup([])
// Assertion runs BEFORE the defensive structuredClone: a function-valued
// annotation must surface as the subset violation it is, not escape as
// structuredClone's DataCloneError.
expect(() => ctx.subagents.start('spawn', structuredRequest(parent, {
outputSchema: { type: 'object', default: () => {} } as unknown as StructuredOutputSchema,
}))).toThrow(/unsupported output schema.*annotation must be JSON data/)
})
it('a post-execute BLOCK on the capture call denies the capture: log and result agree on failure', async () => {
const { ctx, parent, adapter } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 7 }),
textResponse('continues after the blocked capture'),
])
// A PostToolUse-style hook, registered AFTER the runtime (so the runtime's
// prepend commit listener stays outermost and composes this verdict).
ctx.on('tools/post-execute', (exec, _result, next) => {
if (exec.name === STRUCTURED_OUTPUT_TOOL) {
return Promise.resolve({ kind: 'block' as const, feedback: [{ type: 'text' as const, text: 'capture rejected by hook' }] })
}
return next()
})
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
// No capture was committed: the run reports the schema shortfall...
expect(result.structured).toBeUndefined()
expect(result.stopReason).toBe('error')
// ...the logged tool result is the blocked isError with the feedback...
const child = ctx.agents.get(run.id)!
const results = child.session.events.filter(e => e.type === 'tool/result')
expect((results[0]!.data as { isError?: boolean }).isError).toBe(true)
expect(JSON.stringify((results[0]!.data as { content: unknown }).content)).toContain('capture rejected by hook')
// ...and the turn CONTINUED past the blocked call (no captured veto):
// the model got to react to the failure with a second step.
expect(adapter.requests.length).toBe(2)
await run.dispose()
})
it('a post-execute accept-with-replacement still commits the capture', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 8 }),
])
ctx.on('tools/post-execute', (exec, _result, next) => {
if (exec.name === STRUCTURED_OUTPUT_TOOL) {
return Promise.resolve({ kind: 'accept' as const, content: [{ type: 'text' as const, text: 'recorded (rewritten)' }] })
}
return next()
})
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.stopReason).toBe('completed')
expect(result.structured).toEqual({ answer: 8 })
await run.dispose()
})
it('appends the structured instruction to the child REQUEST\'s system text (base prompt preserved)', async () => {
const { ctx, parent, adapter } = await setup([toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 })])
// A context-wide section stands in for the deployment persona: the
// instruction must APPEND to whatever the prompt pipeline assembled, not
// replace it (AgentOptions has no prompt field — the instruction is
// per-request wire state added by the final-request listener).
ctx.systemPrompt.section({ name: 'test:persona', order: 10, text: 'You are a counter.' })
const run = ctx.subagents.start('spawn', structuredRequest(parent))
await run.result
const childRequest = adapter.requests.at(-1)!
expect(childRequest.system).toContain('You are a counter.')
expect(childRequest.system!.endsWith(STRUCTURED_OUTPUT_INSTRUCTION)).toBe(true)
expect(childRequest.system!.indexOf(STRUCTURED_OUTPUT_INSTRUCTION)).toBeGreaterThan(0)
await run.dispose()
})
it('the instruction rides ONLY structured requests: appended for the child, absent for a plain agent', async () => {
const { ctx, parent, adapter } = await setup([
textResponse('parent answer'),
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }),
])
parent.send([{ type: 'text', text: 'hello' }])
await parent.whenIdle()
expect(adapter.requests[0]!.system ?? '').not.toContain(STRUCTURED_OUTPUT_INSTRUCTION)
const run = ctx.subagents.start('spawn', structuredRequest(parent))
await run.result
// The loop always assembles a base prompt (the harness identity section),
// so the instruction APPENDS — never replaces.
const childSystem = adapter.requests.at(-1)!.system!
expect(childSystem.endsWith(STRUCTURED_OUTPUT_INSTRUCTION)).toBe(true)
expect(childSystem.length).toBeGreaterThan(STRUCTURED_OUTPUT_INSTRUCTION.length)
await run.dispose()
})
describe('final-request enforcement (the prepend agent/request listener)', () => {
it('a plain agent assembling while the runtime is LIVE gets the placeholder stripped', async () => {
// Run-scoped acquisition means a plain deployment never registers the
// tool at all; the strip branch exists for the CONCURRENT case — a plain
// agent taking a turn while some structured child holds the runtime open.
const { ctx, parent, adapter } = await setup([textResponse('parent answer')])
const hold = acquireStructuredRuntime(ctx)
parent.send([{ type: 'text', text: 'hello' }])
await parent.whenIdle()
// The placeholder IS in the registry during this turn; the assembly the
// loop rendered must not carry it for an agent without a structured run.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined()
expect(toolNames(adapter.requests[0]!)).not.toContain(STRUCTURED_OUTPUT_TOOL)
hold.release()
})
it('a structured child sees structured_output with ITS schema; a plain agent never sees the tool', async () => {
const { ctx, parent, adapter } = await setup([
// Parent turn (a plain agent): must NOT see the tool.
textResponse('parent answer'),
// Child turn: must see it, with the run's schema.
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42 }),
])
parent.send([{ type: 'text', text: 'hello' }])
await parent.whenIdle()
expect(toolNames(adapter.requests[0]!)).not.toContain(STRUCTURED_OUTPUT_TOOL)
const run = ctx.subagents.start('spawn', structuredRequest(parent))
await run.result
const childRequest = adapter.requests[1]!
expect(toolNames(childRequest)).toContain(STRUCTURED_OUTPUT_TOOL)
const entry = childRequest.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!
expect(entry.parameters).toEqual(SCHEMA)
await run.dispose()
})
it('two concurrent structured children each see their OWN schema', async () => {
const otherSchema: StructuredOutputSchema = {
type: 'object',
properties: { verdict: { type: 'string', enum: ['real', 'bogus'] } },
required: ['verdict'],
}
const { ctx, parent, adapter } = await setup([
(options: GenerateOptions) => {
// Answer with whatever schema this child was given — proves each
// request carried the right one regardless of scheduling order.
const entry = options.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!
const args = 'verdict' in (entry.parameters.properties as Record<string, unknown>)
? { verdict: 'real' }
: { answer: 1 }
return toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, args)
},
(options: GenerateOptions) => {
const entry = options.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!
const args = 'verdict' in (entry.parameters.properties as Record<string, unknown>)
? { verdict: 'real' }
: { answer: 1 }
return toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, args)
},
])
const runA = ctx.subagents.start('spawn', structuredRequest(parent))
const runB = ctx.subagents.start('spawn', structuredRequest(parent, { outputSchema: otherSchema }))
const [a, b] = await Promise.all([runA.result, runB.result])
expect(a.structured).toEqual({ answer: 1 })
expect(b.structured).toEqual({ verdict: 'real' })
const schemas = adapter.requests.map(request =>
request.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!.parameters)
expect(schemas).toContainEqual(SCHEMA)
expect(schemas).toContainEqual(otherSchema)
await runA.dispose()
await runB.dispose()
})
it('wins against a downstream listener that REPLACES the assembly object', async () => {
const { ctx, parent, adapter } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 5 }),
])
// A downstream (non-prepend) listener that returns a brand-new assembly —
// the composition caveat that erases cooperative mutations. Registered
// AFTER the runtime's prepend listener, so it runs INSIDE it.
ctx.on('system-prompt/assemble', async (_assembly, _context, next) => {
const replaced = await next()
return { sections: [...replaced.sections], tools: [...replaced.tools], variables: { ...replaced.variables } }
})
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
expect(result.structured).toEqual({ answer: 5 })
const entry = adapter.requests[0]!.tools!.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)
expect(entry).toBeDefined()
expect(entry!.parameters).toEqual(SCHEMA)
await run.dispose()
})
it('a non-structured agent request keeps tools ABSENT when it had none (no tools: [] materialized)', async () => {
const { parent, adapter } = await setup([
// The registry contributes the placeholder via prompt assembly, so
// tools is an array in the raw request — but after stripping the
// placeholder (its ONLY entry), the field must not be re-added as a
// different shape.
textResponse('plain'),
])
parent.send([{ type: 'text', text: 'q' }])
await parent.whenIdle()
const request = adapter.requests[0]!
expect(toolNames(request)).not.toContain(STRUCTURED_OUTPUT_TOOL)
await new Promise(resolve => setTimeout(resolve, 0))
})
it('shapes a bare assembly on the waterfall: no-agent context strips the placeholder; a structured agent gains schema + trailing instruction section', async () => {
// Drive ctx.systemPrompt.assemble directly — the enforcement listener
// must tolerate a context with NO agent (a bare diagnostic assemble)
// and shape a structured agent's assembly on the same path the loop
// renders and logs as the request header.
const { ctx, parent } = await setup([])
const acquisition = acquireStructuredRuntime(ctx)
// Bare assemble WHILE the runtime is live: the no-agent branch must
// strip the registered placeholder (before the acquisition there is
// nothing to strip — run-scoped registration).
const bare = await ctx.systemPrompt.assemble({})
expect(bare.tools.map(tool => tool.name)).not.toContain(STRUCTURED_OUTPUT_TOOL)
acquisition.attach(parent, SCHEMA)
const shaped = await ctx.systemPrompt.assemble({ agent: parent })
expect(shaped.tools.map(tool => tool.name)).toContain(STRUCTURED_OUTPUT_TOOL)
expect(shaped.tools.find(tool => tool.name === STRUCTURED_OUTPUT_TOOL)!.parameters).toEqual(SCHEMA)
// The demand travels with the tool: the instruction renders LAST
// (appended post-next(); renderPrompt joins in array order).
expect(shaped.sections.at(-1)).toMatchObject({ name: `tool:${STRUCTURED_OUTPUT_TOOL}`, text: STRUCTURED_OUTPUT_INSTRUCTION })
acquisition.detach(parent)
acquisition.release()
})
})
describe('runtime lifetime (refcount: live structured runs)', () => {
it('the runtime exists exactly while structured runs are live: nothing before, nothing after', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 4 }),
])
// No always-on global state: a context that has run no structured child
// carries no capture tool.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
const run = ctx.subagents.start('spawn', structuredRequest(parent))
const result = await run.result
// The capture succeeded — the registrations existed while the run lived.
expect(result.structured).toEqual({ answer: 4 })
// The run's settle released the last acquisition.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
await run.dispose()
})
it('concurrent structured runs share one runtime; the last settle disposes it', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 1 }),
toolCallResponse('c2', STRUCTURED_OUTPUT_TOOL, { answer: 2 }),
])
const first = ctx.subagents.start('spawn', structuredRequest(parent))
const second = ctx.subagents.start('spawn', structuredRequest(parent))
const [a, b] = await Promise.all([first.result, second.result])
expect([a.structured, b.structured].sort()).toEqual([{ answer: 1 }, { answer: 2 }].sort())
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
await first.dispose()
await second.dispose()
})
it('acquisition release is idempotent (double release cannot underflow the refcount)', async () => {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
const first = acquireStructuredRuntime(ctx)
const second = acquireStructuredRuntime(ctx)
first.release()
first.release()
// The second holder still keeps the tool registered.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined()
second.release()
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
})
it('registers the capture tool through the scoped fiber when tools loads after the acquisition', async () => {
// The Loader starts sibling plugins concurrently, so a backend can
// acquire the runtime before dsh-tools has applied. The capture tool
// must then register as soon as `tools` exists — via the inject fiber,
// not by deferring the backend (which would reorder the prompt's tools).
const ctx = new Context()
const acquisition = acquireStructuredRuntime(ctx)
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
// Fiber activation completes asynchronously after the service appears.
await new Promise(resolve => setImmediate(resolve))
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeDefined()
acquisition.release()
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
})
it('releasing before tools ever loads disposes the pending fiber without registering', async () => {
const ctx = new Context()
const acquisition = acquireStructuredRuntime(ctx)
acquisition.release()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await new Promise(resolve => setImmediate(resolve))
// The disposed fiber never fires: nothing registers after the fact.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
})
it('attach/captured/detach manage per-agent state through the acquisition surface', async () => {
const { ctx, parent } = await setup([])
const acquisition = acquireStructuredRuntime(ctx)
expect(acquisition.captured(parent)).toBeUndefined()
acquisition.attach(parent, SCHEMA)
expect(acquisition.captured(parent)).toBeUndefined()
acquisition.detach(parent)
acquisition.detach(parent)
acquisition.release()
// That manual acquisition was the ONLY holder - release disposes.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
})
})
it('a direct structured_output call from an agent WITHOUT a structured run is an isError', async () => {
const { ctx, parent } = await setup([])
// Hold the runtime open (run-scoped: nothing is registered otherwise) so
// the call reaches the capture tool's own fail-loud guard, not UNKNOWN_TOOL.
const hold = acquireStructuredRuntime(ctx)
const result = await ctx.tools.execute({
callId: 'x' as never,
name: STRUCTURED_OUTPUT_TOOL,
arguments: { answer: 1 },
agent: parent,
})
expect(result.isError).toBe(true)
expect(JSON.stringify(result.content)).toContain('only available to subagents')
hold.release()
})
it('a structured_output call with NO calling agent at all is an isError', async () => {
const { ctx } = await setup([])
const hold = acquireStructuredRuntime(ctx)
const result = await ctx.tools.execute({
callId: 'x' as never,
name: STRUCTURED_OUTPUT_TOOL,
arguments: { answer: 1 },
})
expect(result.isError).toBe(true)
hold.release()
})
})

View File

@@ -25,6 +25,12 @@
},
{
"path": "../subagent"
},
{
"path": "../../core/system-prompt"
},
{
"path": "../../core/tools"
}
]
}

View File

@@ -10,7 +10,7 @@ The run mechanics live in the shared [`@deepseek-ai/dsh-subagent-inprocess`](../
## Capabilities
`{ outputSchema: false, depthLimit: true, toolFilter: false }`. It constructs the child, so it enforces a recursion cap; structured output and tool-scoping are deferred (the service rejects a request needing either before `start` runs).
`{ outputSchema: true, depthLimit: true, toolFilter: false }`. It constructs the child, so it enforces a recursion cap, and it supports structured output via the driver's [structured runtime](../subagent-inprocess/README.md) (acquired per structured run inside the driver — this backend registers nothing at apply). Tool-scoping is deferred (the service rejects a request needing it before `start` runs).
## Config

View File

@@ -9,6 +9,11 @@
* ({@link startInProcessRun}); this backend just passes NO seed (a fresh
* child). The fork backend is an independent peer over the same driver.
*
* Structured output (`outputSchema`) is supported via the driver's shared
* structured runtime: the backend acquires it for its plugin lifetime (so the
* capture tool and request-shaping listeners exist before any run), and each
* structured run holds its own acquisition until it settles.
*
* Plugin export shape: named `name`/`inject`/`Config`/`apply`, NO default.
*
* @module @deepseek-ai/dsh-subagent-spawn
@@ -20,6 +25,11 @@ import type { SubagentCapabilities, SubagentProvider, SubagentStartRequest } fro
import { startInProcessRun } from '@deepseek-ai/dsh-subagent-inprocess'
export const name = 'subagent-spawn'
// `tools` is deliberately NOT injected: the shared driver's structured runtime
// (acquired per structured RUN, not at apply) gates its own capture-tool
// registration on `tools` availability, so this backend's apply timing — and
// with it the provider-mirroring delegation tool's position in the
// model-visible tool list — stays what it was before structured output existed.
export const inject = ['subagents', 'agents']
/** Config: the registry name to register the provider under. */
@@ -34,11 +44,12 @@ export const Config: z<Config> = z.object({
/**
* The spawn provider. Supports `depthLimit` (it constructs the child, so it can
* enforce a recursion cap) but NOT `outputSchema` or `toolFilter` in this cut —
* a request that needs either is rejected by the service before `start` runs.
* enforce a recursion cap) and `outputSchema` (via the shared in-process
* structured runtime); NOT `toolFilter` in this cut — a request that needs it
* is rejected by the service before `start` runs.
*/
class SpawnProvider implements SubagentProvider {
readonly capabilities: SubagentCapabilities = { outputSchema: false, depthLimit: true, toolFilter: false }
readonly capabilities: SubagentCapabilities = { outputSchema: true, depthLimit: true, toolFilter: false }
// Context contract: a spawned child starts fresh — it never sees the parent conversation.
readonly inheritsParentContext = false
@@ -46,7 +57,8 @@ class SpawnProvider implements SubagentProvider {
start(request: SubagentStartRequest) {
// Fresh child: no seed. The shared driver mints ids, stamps cwd/lineage/
// depth, drives the one-shot, and maps the result.
// depth, drives the one-shot (including the structured capture when the
// request carries an outputSchema), and maps the result.
return startInProcessRun(this.ctx, request, { providerName: this.name })
}
}

View File

@@ -10,9 +10,9 @@ import { SessionId } from '@deepseek-ai/dsh-session'
import AgentLoop from '@deepseek-ai/dsh-agent-loop'
import * as Invariants from '@deepseek-ai/dsh-invariants'
import SubagentService from '@deepseek-ai/dsh-subagent'
import { MockAdapter, maxTokensResponse, textResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import { MockAdapter, maxTokensResponse, textResponse, toolCallResponse } from '../../../core/agent-loop/tests/mock-adapter.ts'
import * as spawn from '../src/index.ts'
import { depthOf, SubagentDepthError } from '@deepseek-ai/dsh-subagent-inprocess'
import { depthOf, STRUCTURED_OUTPUT_TOOL, SubagentDepthError } from '@deepseek-ai/dsh-subagent-inprocess'
type Script = ConstructorParameters<typeof MockAdapter>[0]
@@ -241,10 +241,10 @@ describe('dsh-subagent-spawn', () => {
await parentHandle.dispose()
})
it('advertises depthLimit but not outputSchema/toolFilter', async () => {
it('advertises depthLimit and outputSchema but not toolFilter', async () => {
const { ctx } = await setup([])
const provider = ctx.subagents.getProvider('spawn')!
expect(provider.capabilities).toEqual({ outputSchema: false, depthLimit: true, toolFilter: false })
expect(provider.capabilities).toEqual({ outputSchema: true, depthLimit: true, toolFilter: false })
})
it('unregisters the provider when its fiber is disposed (HMR safety)', async () => {
@@ -257,6 +257,54 @@ describe('dsh-subagent-spawn', () => {
expect(ctx.subagents.list()).toEqual([])
})
it('captures structured output through the shipped plugin (driver runtime, plugin wiring)', async () => {
const { ctx, parent } = await setup([
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { answer: 42 }),
])
const run = ctx.subagents.start('spawn', {
prompt: [{ type: 'text', text: 'produce the answer' }],
parent,
outputSchema: { type: 'object', properties: { answer: { type: 'number' } }, required: ['answer'] },
})
const result = await run.result
expect(result.stopReason).toBe('completed')
expect(result.structured).toEqual({ answer: 42 })
// Run-scoped runtime: the settle released the last acquisition.
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
await run.dispose()
})
it('a backend unload mid-structured-run settles the run and releases the runtime', async () => {
// Rebuild the stack by hand so we hold the backend's fiber.
const ctx = new Context()
const adapter = new MockAdapter(['hang'])
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await ctx.plugin(AgentRegistry)
await ctx.plugin(Invariants)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(SubagentService)
const fiber = await ctx.plugin(spawn, { providerName: 'spawn' })
ctx.llm.registerAdapter(['mock'], adapter)
const parent = ctx.agentLoop.create(AgentId('parent'), { model: 'mock' })
const run = ctx.subagents.start('spawn', {
prompt: [{ type: 'text', text: 'q' }],
parent,
outputSchema: { type: 'object', properties: { a: { type: 'number' } } },
})
// Let the child's step start streaming, then unload the backend. The
// backend owns the child agent, so the unload tears the child down and
// the run settles — releasing its own runtime acquisition on the way out.
await new Promise(resolve => setTimeout(resolve, 30))
await fiber.dispose()
const result = await run.result
expect(result.stopReason).toBe('error')
expect(ctx.tools.get(STRUCTURED_OUTPUT_TOOL)).toBeUndefined()
await run.dispose()
})
it('has the namespace-plugin export shape (no stray default)', () => {
expect('default' in spawn).toBe(false)
expect(spawn.name).toBe('subagent-spawn')

View File

@@ -8,7 +8,7 @@
import type { Agent, AgentId, AgentOptions } from '@deepseek-ai/dsh-agent'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { SchemaSpec } from '@deepseek-ai/dsh-tools'
import type { StructuredOutputSchema } from '@deepseek-ai/dsh-tools'
/**
* Which START-TIME features a provider supports. Checked by the service
@@ -56,12 +56,16 @@ export interface SubagentStartRequest {
/** Per-child agent options (model, system prompt). */
agentOptions?: AgentOptions
/**
* Optional structured-output schema. When set AND the provider's
* {@link SubagentCapabilities.outputSchema} is `true`, the child's final
* answer is shaped to this schema and surfaced as {@link SubagentResult.structured}.
* Optional structured-output schema — an object-rooted JSON Schema within the
* enforced subset (see `assertSupportedOutputSchema` in dsh-tools; a schema
* outside the subset is rejected loud at start). When set AND the provider's
* {@link SubagentCapabilities.outputSchema} is `true`, the child is driven to
* report a value matching this schema, surfaced as
* {@link SubagentResult.structured}. The schema must be plain host-realm JSON
* data — a caller holding foreign-realm data materializes it first.
* Requesting it against a provider that lacks the capability is rejected at start.
*/
outputSchema?: SchemaSpec
outputSchema?: StructuredOutputSchema
/**
* Optional recursion cap (max delegation depth below this child). Requires
* {@link SubagentCapabilities.depthLimit}; rejected at start otherwise.

View File

@@ -178,7 +178,7 @@ describe('SubagentService', () => {
describe('start-time capability validation (fail loud, before any child)', () => {
it.each([
{ field: 'outputSchema', request: baseRequest({ outputSchema: { x: { type: 'string' } } }) },
{ field: 'outputSchema', request: baseRequest({ outputSchema: { type: 'object', properties: { x: { type: 'string' } } } }) },
{ field: 'maxDepth', request: baseRequest({ maxDepth: 2 }) },
{ field: 'toolFilter', request: baseRequest({ toolFilter: { deny: ['bash'] } }) },
])('rejects $field against a provider that lacks the capability — before start() runs', ({ request }) => {
@@ -203,7 +203,7 @@ describe('SubagentService', () => {
await ctx.plugin(SubagentService)
const provider = new StubProvider('strong', ALL_CAPS)
ctx.subagents.registerProvider(provider)
ctx.subagents.start('strong', baseRequest({ outputSchema: { x: { type: 'string' } }, maxDepth: 1 }))
ctx.subagents.start('strong', baseRequest({ outputSchema: { type: 'object', properties: { x: { type: 'string' } } }, maxDepth: 1 }))
expect(provider.startCount).toBe(1)
})
})

View File

@@ -4,8 +4,9 @@ Packages that exist to serve development, testing, and the examples rather than
| Package | Role | ctx key |
|---|---|---|
| `acp-snapshot/` | ACP snapshot suite kit: subprocess scenario harness + golden normalizers + the `defineAcpSnapshotSuite` factory | (library — imported by example `*.snapshot.ts` suites) |
| `invariants/` | Dev-mode event-contract invariants + session-log freeze | (listens on `session/*`, `agent/*`) |
| `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) |
| `subagent-mock/` | Scripted `SubagentProvider` for deterministic seam/tool tests | (registers on `ctx.subagents`) |
`invariants` runs only in dev mode (contract checks, not runtime behavior). `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers.
`invariants` runs only in dev mode (contract checks, not runtime behavior). `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `acp-snapshot` carries the snapshot tier's harness/normalizer/suite machinery so every example's suite is a scenario table over one shared, gate-covered implementation. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers.

View File

@@ -0,0 +1,36 @@
# `@deepseek-ai/dsh-acp-snapshot`
The ACP snapshot suite kit: the shared machinery behind the keyless snapshot tier (`pnpm run test:snapshot`, [testing policy](../../../docs/testing.md)). An example gets a full snapshot suite from a scenario table plus a fixtures directory; every compare/guard mechanic lives here, under the per-file coverage gate, instead of being copied per example.
Three layers, importable separately:
- **`runScenario` (harness)** — boots the real agent bin as a subprocess via tsx (unbuilt, Loader path), drives it over ACP JSON-RPC stdio from a deterministic `input.json` script, tees raw stdout for the golden + purity check, and harvests every persisted session JSONL (parent + subagent children, primary-first) after a graceful stdin-EOF shutdown. Parameterized by `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath` — absolute paths; the subprocess cwd is a temp dir outside the repo).
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), and the composable `scrubRequestHeaders` (header bulk → `{{system}}`/`{{tools}}`, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record-mode fixture write-back, the per-suite header pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin, non-pinning fixtures header-scrubbed). Must be called at vitest collection time.
A consuming `*.snapshot.ts` is the scenario table plus one factory call:
```ts
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { defineAcpSnapshotSuite, type Scenario } from '@deepseek-ai/dsh-acp-snapshot'
const SCENARIOS: Scenario[] = [
{ name: 'text-turn', hasModelTurn: true, recorded: true, pinsHeader: true },
]
defineAcpSnapshotSuite({
agent: { // absolute paths, resolved from the suite's own location
binScript: fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)),
configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)),
tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)),
},
snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'),
scenarios: SCENARIOS, // exactly one entry sets pinsHeader
mode: process.env.DSH_SNAPSHOT === 'record' ? 'record' : 'replay',
})
```
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config RFC](../../../docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. Fixture roles, record/replay semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
Constraints: `suite.ts` imports vitest, so the package is importable only inside a vitest run (the harness and normalizers have no such dependency but ship from the same entry). ACP-specific by design — the harness speaks the SDK's `ClientSideConnection`. Permission round-trips are scriptable: `InputScript.permissionAnswers` is a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) the client maps to the agent-issued `optionId` at answer time; an absent or exhausted queue answers `cancelled`, and a kind the request never offered rejects the run (the agent is answered `cancelled`, so a tolerant agent cannot absorb the scenario bug).

View File

@@ -0,0 +1,35 @@
{
"name": "@deepseek-ai/dsh-acp-snapshot",
"description": "ACP snapshot suite kit: real-subprocess scenario harness, golden normalizers, and the suite factory behind the keyless snapshot tier",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"dependencies": {
"@agentclientprotocol/sdk": "0.25.1",
"tsx": "^4.22.4",
"vitest": "^4.1.8"
},
"peerDependencies": {
"cordis": "^4.0.0-rc.6"
},
"devDependencies": {
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,469 @@
/**
* Shared subprocess harness for ACP snapshot suites. A library module driven by
* the suite factory in ./suite.ts (and directly by harness-level specs); each
* example's `*.snapshot.ts` names its own agent-under-test paths.
*
* It boots the REAL agent bin subprocess via the cordis Loader (so the
* export-shape bug class stays guarded — see docs/postmortem/0001), drives it
* over real ACP JSON-RPC stdio with a deterministic input script, tees raw
* stdout (for the golden + a purity check) into an SDK `ClientSideConnection`,
* and — in record mode — harvests the persisted session JSONL after a graceful
* shutdown flush. The pure normalizers in ./normalize.ts turn the captured
* stdout frames and the session-log events into stable, snapshot-able text.
*
* See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md.
*
* @module @deepseek-ai/dsh-acp-snapshot/harness
*/
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
import { cp, mkdtemp, readFile, readdir, rm } from 'node:fs/promises'
import { existsSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join, delimiter } from 'node:path'
import { fileURLToPath } from 'node:url'
import { Readable, Writable } from 'node:stream'
import {
ClientSideConnection,
ndJsonStream,
PROTOCOL_VERSION,
type Agent as AcpAgent,
type Client,
type RequestPermissionRequest,
type RequestPermissionResponse,
type SessionNotification,
} from '@agentclientprotocol/sdk'
// Resolve tsx's ESM loader to an ABSOLUTE path once: the child runs with its
// cwd in a temp dir OUTSIDE the repo, where a bare `--import tsx` would not
// resolve from node_modules. import.meta.resolve gives this package's tsx
// regardless of the child cwd.
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
/**
* The agent composition a scenario runs against: which bin to boot and which
* leaf config it loads. All paths are ABSOLUTE — the subprocess cwd is a temp
* dir outside the repo, so relative resolution would miss; a suite resolves
* them from its own `import.meta.url`.
*/
export interface AgentUnderTest {
/** The agent bin entry (e.g. `packages/ui/acp-agent/src/bin.ts`), run unbuilt via tsx. */
binScript: string
/**
* The example's live `cordis.yml`. Under `DSH_SNAPSHOT=replay` the bin swaps
* it for the sibling `cordis.snapshot.yml` (the keyless replay overlay), so
* one path serves both modes.
*/
configPath: string
/**
* The repo-root tsconfig whose `paths` map resolves the unbuilt workspace
* imports. Passed to the child as `TSX_TSCONFIG_PATH`: tsx finds a tsconfig
* by searching UP from the child's cwd — a temp dir outside the repo — so
* without the explicit pin the dsh-* imports fail before the bin writes a
* byte.
*/
tsconfigPath: string
}
/**
* One step of a scenario's deterministic input script (`input.json`). The
* harness interprets these in order. `newSession` captures the server-issued
* (random) session id into a `{{sessionId}}` variable that later steps
* reference, since a committed file cannot know the id in advance.
*
* `promptAndCancel` sends a prompt WITHOUT awaiting its response, waits until
* the client observes the first streamed `agent_message_chunk` (so the emitted
* frames deterministically precede the cancellation), then cancels the turn —
* the only way to exercise a cancel deterministically (a plain `prompt` step
* awaits the response, which a cancel/hang scenario would block on forever).
*/
export type InputStep =
| { op: 'initialize'; terminalOutput?: boolean }
| { op: 'newSession' }
| { op: 'newSessionExpectError'; additionalDirectories?: string[] }
| { op: 'prompt'; text: string }
| { op: 'promptExpectError'; text: string }
| { op: 'promptAndCancel'; text: string }
| { op: 'cancel' }
/** A scenario's `input.json`: an ordered list of input steps. */
export interface InputScript {
steps: InputStep[]
/**
* Ordered answers for the agent's `session/request_permission` round-trips,
* consumed FIFO — the Nth request gets the Nth answer. Each answer selects
* by option KIND: option ids are agent-issued randoms a committed script
* cannot know, while kinds are the ACP-stable vocabulary, so the client maps
* kind → the offered `optionId` at answer time. A request beyond the queue
* (or with no queue at all) is answered `cancelled` — the stub behavior a
* scenario without approvals relies on. A scripted kind the request does
* not offer REJECTS the run: the scenario scripted an impossible click,
* and {@link runScenario} throws once the in-flight step settles (the
* agent itself just sees `cancelled`, so it cannot absorb the bug).
*/
permissionAnswers?: PermissionAnswer[]
}
/** One scripted answer to a permission request: which offered option kind to select. */
export interface PermissionAnswer {
/** The `PermissionOption.kind` to select (`allow_once`, `reject_always`, …). */
kind: 'allow_once' | 'allow_always' | 'reject_once' | 'reject_always'
}
/** One harvested session log plus the identifying facts off its header line. */
export interface HarvestedLog {
/** The recorded session id (header `id`). */
id: string
/** Session creation time (header `createdAt`) — the child-ordering key. */
createdAt: number
/** The parent session id, if this log is a subagent child (header `parentSession`). */
parentSession?: string
/** The full `.jsonl` file content. */
content: string
}
/** The result of running a scenario: raw stdout + the harvested session log(s). */
export interface RunResult {
/** Raw stdout bytes (decoded utf8), every newline-delimited JSON-RPC frame. */
rawStdout: string
/** stderr (for diagnostics on failure). */
stderr: string
/** The session id the server issued (undefined if no session was created). */
sessionId?: string
/** The temp cwd the session ran in (the bash workspace). */
cwd: string
/**
* Every persisted session log harvested after the run, ordered primary-first:
* the top-level (parent) session — the one with no `parentSession` — then each
* subagent child by ascending `createdAt`. A single-session scenario harvests
* exactly one; a nested-agent scenario harvests the parent plus one per child.
*/
sessionLogs: HarvestedLog[]
}
/** How to run one scenario: the agent to boot, the mode, and the fixture wiring. */
export interface RunOptions {
/** The agent composition to boot. */
agent: AgentUnderTest
/** `replay` (default, keyless) or `record` (real API, harvests the log). */
mode: 'replay' | 'record'
/** The recorded session JSONL fixture path (replay reads it; record writes near it). */
fixtureFile: string
/** Optional sidecar override path (replay). */
overrideFile?: string
/**
* Recorded SUBAGENT child-session fixture paths (replay). A nested-agent
* scenario ships one per child (`session.1.jsonl`, …); the harness forwards
* them to `dsh-llm-replay` via `$DSH_SNAPSHOT_CHILD_FILES` so each child
* session replays from its own recorded script. Empty for single-session
* scenarios. Ignored in record mode (children are harvested, not replayed).
*/
childFiles?: string[]
/**
* Optional `<scenario>/workspace/` directory whose contents are copied into
* the temp cwd BEFORE the run — the standard way to seed files the agent
* operates on (a file to read, edit, or grep). Absent for scenarios that
* start from an empty workspace.
*/
workspaceDir?: string
}
/**
* Run a scenario end-to-end against a freshly-spawned subprocess. Owns the
* child and its temp dirs; always tears them down. Returns the captured stdout
* and (record mode) the harvested session-log path.
*
* @param input The scenario's input script (steps + optional permission answers).
* @param opts The agent to boot, the mode, and the fixture wiring.
* @returns The captured stdout/stderr, session id, temp cwd, and harvested logs.
*/
export async function runScenario(input: InputScript, opts: RunOptions): Promise<RunResult> {
const cwd = await mkdtemp(join(tmpdir(), 'acp-snap-cwd-'))
const sessionsRoot = await mkdtemp(join(tmpdir(), 'acp-snap-sessions-'))
// Everything past the temp-dir creation runs under a try/finally that always
// removes both dirs — so a failure in workspace seeding, spawn, or any step
// never leaks them (the "e2e tests own their resources" rule).
let child: ChildProcessWithoutNullStreams | undefined
let sessionId: string | undefined
let sessionLogs: HarvestedLog[] = []
const rawBuffers: Buffer[] = []
const stderrChunks: string[] = []
try {
// Seed the workspace if the scenario ships one (a file the agent reads/edits).
// Copied into the temp cwd so the agent's bash tools see it; the goldens
// normalize the cwd, so the seeded paths stay stable across runs.
if (opts.workspaceDir !== undefined && existsSync(opts.workspaceDir)) {
await cp(opts.workspaceDir, cwd, { recursive: true })
}
const env: NodeJS.ProcessEnv = {
...process.env,
TSX_TSCONFIG_PATH: opts.agent.tsconfigPath,
DSH_SNAPSHOT: opts.mode,
DSH_SNAPSHOT_FILE: opts.fixtureFile,
DSH_SNAPSHOT_SESSIONS_ROOT: sessionsRoot,
...opts.overrideFile !== undefined ? { DSH_SNAPSHOT_OVERRIDE: opts.overrideFile } : {},
...opts.childFiles !== undefined && opts.childFiles.length > 0
? { DSH_SNAPSHOT_CHILD_FILES: opts.childFiles.join(delimiter) }
: {},
}
child = spawn(
process.execPath,
['--import', tsxLoader, opts.agent.binScript, opts.agent.configPath],
{ cwd, env, stdio: ['pipe', 'pipe', 'pipe'] },
)
child.stderr.setEncoding('utf8')
child.stderr.on('data', (c: string) => stderrChunks.push(c))
// Tee raw stdout: accumulate the bytes for the golden + purity check, and ALSO
// feed the same bytes to the SDK client through a passthrough. Buffer the raw
// bytes (not per-chunk utf8 strings) and decode once at the end, so a
// multibyte sequence split across two 'data' events can't corrupt the golden.
const passthrough = new Readable({ read() {} })
child.stdout.on('data', (buf: Buffer) => {
rawBuffers.push(buf)
passthrough.push(buf)
})
child.stdout.on('end', () => passthrough.push(null))
const stream = ndJsonStream(
Writable.toWeb(child.stdin) as WritableStream<Uint8Array>,
Readable.toWeb(passthrough) as ReadableStream<Uint8Array>,
)
// Watcher so a step can block until the client OBSERVES a particular
// session/update — used by promptAndCancel to pin frame order (send cancel
// only after the streamed agent_message_chunk has arrived, so those frames
// deterministically precede the cancelled prompt response).
const updateWaiters: { match: (u: SessionNotification['update']) => boolean; resolve: () => void }[] = []
const waitForUpdate = (match: (u: SessionNotification['update']) => boolean): Promise<void> =>
new Promise<void>(resolve => updateWaiters.push({ match, resolve }))
// Permission answers are consumed FIFO across the whole run; exhaustion
// falls back to `cancelled` so approval-free scenarios keep the plain stub.
const permissionQueue = [...input.permissionAnswers ?? []]
// A scenario bug detected inside a client callback (a scripted permission
// kind the agent never offered). It cannot fail the run from in there: a
// callback throw only becomes a JSON-RPC error RESPONSE to the agent, and
// a tolerant agent treats that as a denial and carries on — the run (or
// worse, a record) would absorb the impossible click silently. So the
// callback answers `cancelled` (a well-defined path for the agent),
// captures the error here, and the step loop fails the run on it.
let scriptError: Error | undefined
const makeClient = (_agent: AcpAgent): Client => ({
sessionUpdate(params: SessionNotification): Promise<void> {
for (let i = updateWaiters.length - 1; i >= 0; i--) {
const waiter = updateWaiters[i]
// The index is always in-bounds (i only decreases; splice removes at
// i, so lower entries stay valid); the guard satisfies
// noUncheckedIndexedAccess.
/* v8 ignore next 1 -- unreachable in-bounds guard, see above */
if (waiter === undefined) continue
if (waiter.match(params.update)) {
updateWaiters.splice(i, 1)
waiter.resolve()
}
}
return Promise.resolve()
},
requestPermission(params: RequestPermissionRequest): Promise<RequestPermissionResponse> {
const answer = permissionQueue.shift()
if (answer === undefined) return Promise.resolve({ outcome: { outcome: 'cancelled' } })
const option = params.options.find(o => o.kind === answer.kind)
if (option === undefined) {
// The scenario scripted a click the agent never offered — a scenario
// bug. Captured (last one wins; same bug class either way) and
// answered `cancelled`; the step loop rejects the run on it.
scriptError = new Error(
`snapshot-harness: scripted permission answer ${answer.kind} not among `
+ `the offered options [${params.options.map(o => o.kind).join(', ')}]`,
)
return Promise.resolve({ outcome: { outcome: 'cancelled' } })
}
return Promise.resolve({ outcome: { outcome: 'selected', optionId: option.optionId } })
},
})
const client = new ClientSideConnection(makeClient, stream)
for (const step of input.steps) {
await runStep(client, step, cwd, waitForUpdate, () => sessionId, (id) => { sessionId = id })
// A permission exchange happens while a step's request is in flight, so
// by the time the step settles any script bug it exposed is captured —
// fail the run HERE, as a harness error, rather than hoping the agent's
// reaction to the answer perturbs the transcript.
if (scriptError !== undefined) throw scriptError
}
// Done driving: close stdin so the server disposes gracefully (flushing
// persistence) and exits. Then await exit so the harvested log is complete.
child.stdin.end()
await waitForExit(child)
// Harvest EVERY persisted log (parent + any subagent children) while the
// temp dirs still exist, ordered primary-first.
sessionLogs = await harvestSessionLogs(sessionsRoot)
} finally {
// Failure-safe teardown: kill a still-running child and drop the temp dirs
// even if seeding/spawn/a step/harvest threw, so a flaky run never leaks a
// process or dir. `child` is undefined only if spawn itself threw.
if (child !== undefined && child.exitCode === null && child.signalCode === null) {
child.kill('SIGKILL')
await waitForExit(child)
}
await rm(cwd, { recursive: true, force: true })
await rm(sessionsRoot, { recursive: true, force: true })
}
return {
rawStdout: Buffer.concat(rawBuffers).toString('utf8'),
stderr: stderrChunks.join(''),
cwd,
...sessionId !== undefined ? { sessionId } : {},
sessionLogs,
}
}
/** Drive one input step over the client connection. */
async function runStep(
client: ClientSideConnection,
step: InputStep,
cwd: string,
waitForUpdate: (match: (u: SessionNotification['update']) => boolean) => Promise<void>,
getSessionId: () => string | undefined,
setSessionId: (id: string) => void,
): Promise<void> {
switch (step.op) {
case 'initialize':
await client.initialize({
protocolVersion: PROTOCOL_VERSION,
clientCapabilities: step.terminalOutput === true ? { _meta: { terminal_output: true } } : {},
})
return
case 'newSession': {
const { sessionId } = await client.newSession({ cwd, mcpServers: [] })
setSessionId(sessionId)
return
}
case 'newSessionExpectError': {
// The bridge rejects a session/new that widens the workspace scope
// (non-empty additionalDirectories / mcpServers — unimplemented). The SDK
// surfaces that as a rejected RPC; swallow it so the run completes and the
// error frame is captured in the transcript.
await client.newSession({
cwd,
mcpServers: [],
...step.additionalDirectories !== undefined ? { additionalDirectories: step.additionalDirectories } : {},
}).then(
() => { throw new Error('snapshot-harness: expected session/new to be rejected but it succeeded') },
() => { /* expected: the bridge rejected the unsupported workspace scope */ },
)
return
}
case 'prompt': {
const sessionId = getSessionId()
if (sessionId === undefined) throw new Error('snapshot-harness: prompt before newSession')
await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] })
return
}
case 'promptExpectError': {
const sessionId = getSessionId()
if (sessionId === undefined) throw new Error('snapshot-harness: promptExpectError before newSession')
// The model fails this turn (a recorded provider error), so the bridge
// answers the prompt with a JSON-RPC error and the SDK rejects. That
// rejection IS the expected editor experience — swallow it so the run
// completes and the stdout transcript (the error frame) is captured.
await client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] })
.then(() => { throw new Error('snapshot-harness: expected the prompt to fail but it succeeded') },
() => { /* expected: the turn failed and the bridge returned an error */ })
return
}
case 'promptAndCancel': {
const sessionId = getSessionId()
if (sessionId === undefined) throw new Error('snapshot-harness: promptAndCancel before newSession')
// Dispatch the prompt WITHOUT awaiting (a hang fixture never resolves on
// its own). To pin frame order deterministically, wait until the client
// has OBSERVED the hang's streamed agent_message_chunk before cancelling —
// so those update frames always precede the cancelled prompt response in
// the transcript (without this, the late chunk and the response race).
// Then cancel and await the prompt, which the bridge settles as
// `cancelled` once the abort propagates.
const promptDone = client.prompt({ sessionId, prompt: [{ type: 'text', text: step.text }] })
await waitForUpdate(u => u.sessionUpdate === 'agent_message_chunk')
await client.cancel({ sessionId })
await promptDone
return
}
case 'cancel': {
const sessionId = getSessionId()
if (sessionId === undefined) throw new Error('snapshot-harness: cancel before newSession')
await client.cancel({ sessionId })
return
}
default:
throw new Error(`snapshot-harness: unknown input op ${JSON.stringify(step)}`)
}
}
/** Resolve once the child process exits (any code/signal). */
function waitForExit(child: ChildProcessWithoutNullStreams): Promise<void> {
// Race guard: both call sites run within one synchronous frame of
// stdin.end()/kill(), so the exit event cannot have been delivered yet;
// kept for any future caller that awaits in between.
/* v8 ignore next 1 -- unreachable race guard, see above */
if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve()
return new Promise<void>(resolve => child.once('exit', () => { resolve() }))
}
/**
* Harvest EVERY persisted `.jsonl` session log under a sessions root, parse each
* header line, and return them ordered primary-first: the top-level session (no
* `parentSession`) leads, then each subagent child by ascending `createdAt`.
*
* The JSONL backend lays sessions out as `<root>/<cwd-bucket>/<encoded-id>.jsonl`
* (one bucket per cwd), so a parent and its same-cwd in-process child land in
* the SAME bucket — collecting all files across all buckets catches both (a
* first-match short-circuit would silently drop the child). Returns `[]` if no
* log was produced (a no-session scenario).
*/
async function harvestSessionLogs(root: string): Promise<HarvestedLog[]> {
let cwdDirs: string[]
try {
cwdDirs = await readdir(root)
} catch {
return []
}
const logs: HarvestedLog[] = []
for (const dir of cwdDirs) {
const sub = join(root, dir)
let files: string[]
try {
files = await readdir(sub)
} catch {
continue
}
for (const f of files) {
if (!f.endsWith('.jsonl')) continue
const content = await readFile(join(sub, f), 'utf8')
const firstLine = content.split('\n').find(line => line.trim().length > 0) ?? '{}'
const header = JSON.parse(firstLine) as { id?: unknown; createdAt?: unknown; parentSession?: unknown }
logs.push({
id: typeof header.id === 'string' ? header.id : '',
createdAt: typeof header.createdAt === 'number' ? header.createdAt : 0,
...typeof header.parentSession === 'string' ? { parentSession: header.parentSession } : {},
content,
})
}
}
// Primary (no parentSession) first, then children by ascending createdAt. A
// scenario has exactly one top-level session. In the synchronous cut sibling
// children are created strictly sequentially, so their createdAt values are
// strictly ordered; the recordedId tiebreak only keeps a degenerate
// same-millisecond collision (unreachable here) deterministic. This harvest
// order must match the replay load order in dsh-llm-replay's loadSessionScripts
// so session.<n>.jsonl maps to the same child on record and replay — replay
// re-sorts childFiles by the same key, so the two stay consistent.
logs.sort((a, b) => {
const ap = a.parentSession === undefined ? 0 : 1
const bp = b.parentSession === undefined ? 0 : 1
return ap - bp || a.createdAt - b.createdAt || a.id.localeCompare(b.id)
})
return logs
}

View File

@@ -0,0 +1,38 @@
/**
* ACP snapshot suite kit — the shared machinery behind the keyless snapshot
* tier (`pnpm run test:snapshot`). Three layers, composable per example:
* the subprocess scenario harness ({@link runScenario}), the pure golden
* normalizers ({@link normalizeStdout} / {@link normalizeSessionLog} /
* {@link scrubRequestHeaders}), and the suite factory
* ({@link defineAcpSnapshotSuite}) that registers a scenario table as a full
* describe/it tree. An example's `*.snapshot.ts` supplies only its
* {@link AgentUnderTest} paths, its snapshots directory, and its
* {@link Scenario} table.
*
* NOTE: ./suite.ts imports vitest, so this package is importable only inside a
* vitest run — a support-tier constraint stated in the README.
*
* @module @deepseek-ai/dsh-acp-snapshot
*/
export {
runScenario,
type AgentUnderTest,
type HarvestedLog,
type InputScript,
type InputStep,
type PermissionAnswer,
type RunOptions,
type RunResult,
} from './harness.ts'
export {
normalizeSessionLog,
normalizeStdout,
scrubRequestHeaders,
type NormalizeContext,
} from './normalize.ts'
export {
defineAcpSnapshotSuite,
type Scenario,
type SnapshotSuiteOptions,
} from './suite.ts'

View File

@@ -0,0 +1,199 @@
/**
* Pure normalizers for the ACP snapshot goldens. They replace the
* non-deterministic values in the two captured surfaces — the stdout JSON-RPC
* transcript and the persisted session JSONL — with stable tokens, so a golden
* compare reflects behavior, not run-to-run noise. Kept dependency-free and
* side-effect-free so they unit-test trivially.
*
* Scrubbed: `randomUUID()` session ids → `{{sessionId}}`; the temp `mkdtemp`
* cwd → `{{cwd}}` (it appears in terminal-card `_meta` and the log header);
* JSON-RPC request `id` → a stable per-transcript sequence; the log's per-event
* `time` (epoch ms) and header `createdAt` → 0; a `hook/result` event's
* `durationMs` (wall-clock hook runtime) → 0. NOT scrubbed: the log's `seq`
* (deterministic — `seq = log.length`, part of the event-log contract).
*
* A separate, composable normalizer — {@link scrubRequestHeaders} — replaces
* the bulky request-header CONTENT (the composed system prompt and the tool
* schema list) with `{{system}}`/`{{tools}}` tokens. It is deliberately NOT
* folded into {@link normalizeSessionLog}: each suite's one header-pinning
* scenario compares that content verbatim, every other scenario composes the
* scrub in (the `pinsHeader` flag on the scenario table, consumed by the suite
* factory in ./suite.ts; see the pinned-header RFC,
* docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md).
*
* See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md.
*
* @module @deepseek-ai/dsh-acp-snapshot/normalize
*/
const SESSION_ID = '{{sessionId}}'
const CWD = '{{cwd}}'
const SYSTEM = '{{system}}'
const TOOLS = '{{tools}}'
/** A UUID v4 string, the shape `randomUUID()` produces for session ids. */
const UUID_RE = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi
/** Inputs the normalizers need to recognize a run's volatile values. */
export interface NormalizeContext {
/** The session id(s) the run issued — replaced with `{{sessionId}}`. */
sessionIds: string[]
/** The temp cwd the run used — replaced with `{{cwd}}`. */
cwd: string
}
/** Replace cwd, session ids, and any stray UUID with stable tokens in a string. */
function scrubString(value: string, ctx: NormalizeContext): string {
let out = value
// cwd first (longest, most specific), then explicit session ids, then any
// residual UUID (covers ids that appear in places we didn't enumerate).
out = out.split(ctx.cwd).join(CWD)
for (const id of ctx.sessionIds) out = out.split(id).join(SESSION_ID)
out = out.replace(UUID_RE, SESSION_ID)
return out
}
/** Recursively scrub a parsed JSON value (strings replaced; structure kept). */
function scrubValue(value: unknown, ctx: NormalizeContext): unknown {
if (typeof value === 'string') return scrubString(value, ctx)
if (Array.isArray(value)) return value.map(v => scrubValue(v, ctx))
if (value !== null && typeof value === 'object') {
const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(value)) out[k] = scrubValue(v, ctx)
return out
}
return value
}
/**
* Normalize a raw stdout transcript (newline-delimited JSON-RPC frames) into a
* stable golden in the SAME shape as the wire: one compact JSON frame per line
* (NDJSON), with the JSON-RPC `id` rewritten to a per-transcript sequence
* (1, 2, 3, …) and all volatile strings scrubbed. Throws if any non-empty line
* is not valid JSON — that doubles as the stdout-purity check (no logger leaked
* onto the protocol).
*
* @param rawStdout The captured stdout bytes, decoded utf8.
* @param ctx The run's volatile values to scrub.
* @returns The normalized NDJSON transcript, one frame per line.
*/
export function normalizeStdout(rawStdout: string, ctx: NormalizeContext): string {
const lines = rawStdout.split('\n').filter(line => line.trim().length > 0)
// Map each distinct JSON-RPC id (request/response correlate by id) to a stable
// sequence number, in first-seen order, so id churn doesn't perturb the golden.
const idSeq = new Map<string, number>()
const stableId = (id: unknown): number => {
const key = JSON.stringify(id)
let n = idSeq.get(key)
if (n === undefined) { n = idSeq.size + 1; idSeq.set(key, n) }
return n
}
const frames = lines.map((line) => {
const frame = JSON.parse(line) as Record<string, unknown>
if ('id' in frame && frame.id !== undefined && frame.id !== null) {
frame.id = stableId(frame.id)
}
return scrubValue(frame, ctx) as Record<string, unknown>
})
return frames.map(f => JSON.stringify(f)).join('\n') + '\n'
}
/**
* Normalize a session JSONL log into a stable golden: the header line's
* volatile fields (`createdAt`, `id`, `cwd`) and every event's `time` are
* zeroed/scrubbed, all volatile strings scrubbed, and `seq` is LEFT INTACT
* (deterministic by contract). Output is JSONL in the same shape as the input —
* one compact record per line.
*
* @param rawLog The raw session `.jsonl` content.
* @param ctx The run's volatile values to scrub.
* @returns The normalized JSONL log, one record per line.
*/
export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): string {
const lines = rawLog.split('\n').filter(line => line.trim().length > 0)
const records = lines.map((line) => {
const record = JSON.parse(line) as Record<string, unknown>
// Header line: { type: 'session', createdAt, id, cwd, … }.
if (record.type === 'session') {
if ('createdAt' in record) record.createdAt = 0
} else if ('time' in record) {
// Event line: zero the epoch-ms timestamp; keep seq (deterministic).
record.time = 0
// A hook/result carries the hook's wall-clock runtime (`data.durationMs`),
// which is run-to-run noise like `time` — zero it so the golden reflects
// the hook's decision/exit, not how long the shell took.
if (record.type === 'hook/result' && record.data !== null && typeof record.data === 'object') {
const data = record.data as Record<string, unknown>
if ('durationMs' in data) data.durationMs = 0
}
}
return scrubValue(record, ctx) as Record<string, unknown>
})
return records.map(r => JSON.stringify(r)).join('\n') + '\n'
}
/**
* Replace request-header CONTENT in a session JSONL with stable tokens,
* keeping its structure: a `request/header` event's `data.header.system` →
* `{{system}}` and `data.header.tools` → `{{tools}}`; a
* `request/header-delta` event keeps every structural fact — the system
* delta's `keepStart`/`keepEnd` line positions and inserted-line COUNT (one
* `{{system}}` token per inserted line), the tools delta's
* added/removed/changed tool NAMES — and tokenizes only the bulk (prompt
* text; each added/changed schema's fields other than `name` → `{{tools}}`),
* so two different deltas still compare different.
* Absent fields stay absent — WHETHER a header carried a system prompt or
* tools is behavior and stays visible; `config` and `reason` are small and
* stable, so they stay verbatim (a model swap churns every fixture by design
* — it invalidates the recorded responses; a prompt/schema edit churns none —
* replay never reads this content, see dsh-llm-replay).
*
* Only lines with something to scrub are re-serialized; every other line
* passes through byte-for-byte, so the transform is idempotent and applying
* it to an already-scrubbed fixture is a no-op — the on-disk-fixtures guard
* in ./suite.ts relies on exactly that.
*
* @param rawLog The raw session `.jsonl` content.
* @returns The JSONL with header content tokenized, other lines byte-identical.
*/
export function scrubRequestHeaders(rawLog: string): string {
const lines = rawLog.split('\n')
const out = lines.map((line) => {
if (line.trim().length === 0) return line
const record = JSON.parse(line) as Record<string, unknown>
const data = record.data as Record<string, unknown> | null | undefined
if (data === null || typeof data !== 'object') return line
if (record.type === 'request/header') {
const header = data.header as Record<string, unknown> | null | undefined
if (header === null || typeof header !== 'object') return line
if (!('system' in header) && !('tools' in header)) return line
if ('system' in header) header.system = SYSTEM
if ('tools' in header) header.tools = TOOLS
return JSON.stringify(record)
}
if (record.type === 'request/header-delta') {
let touched = false
const system = data.system as Record<string, unknown> | null | undefined
if (system !== null && typeof system === 'object' && Array.isArray(system.insert)) {
system.insert = system.insert.map(() => SYSTEM)
touched = true
}
const tools = data.tools as Record<string, unknown> | null | undefined
if (tools !== null && typeof tools === 'object') {
if (Array.isArray(tools.added)) { tools.added = tools.added.map(scrubToolSchema); touched = true }
if (Array.isArray(tools.changed)) { tools.changed = tools.changed.map(scrubToolSchema); touched = true }
}
return touched ? JSON.stringify(record) : line
}
return line
})
return out.join('\n')
}
/** Tokenize one tool schema's bulk (description, parameters, anything else), keeping its identifying `name`. */
function scrubToolSchema(tool: unknown): unknown {
if (tool === null || typeof tool !== 'object' || Array.isArray(tool)) return tool
const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(tool)) out[k] = k === 'name' ? v : TOOLS
return out
}

View File

@@ -0,0 +1,373 @@
/**
* The ACP snapshot suite factory (REPLAY by default, keyless). A suite is a
* scenario table plus a snapshots directory: each scenario under
* `<snapshotsDir>/<name>/` ships an `input.json` (the client stdin script) and
* a `session.jsonl` fixture; replay boots the real agent subprocess
* (./harness.ts), drives it, and diffs the normalized stdout transcript
* against the committed `stdout.golden.jsonl`. For model scenarios it ALSO
* checks the re-persisted session log — against the `session.jsonl` fixture
* itself, not a separate golden: the fixture doubles as the replay source
* (recorded scenarios) and the expected produced log (both sides normalized
* before comparing).
*
* Request-header content (the composed system prompt + tool schemas riding on
* `request/header` events) is pinned by exactly ONE scenario per suite — the
* one with `pinsHeader` — and scrubbed to `{{system}}`/`{{tools}}` tokens in
* every other fixture and compare, so a prompt or tool-schema edit churns one
* committed line instead of every fixture. A per-run uniformity guard keeps
* the single pin sound: every live header must equal the pinned one, and no
* header-delta may appear outside the pinning scenario (see the
* pinned-header RFC,
* docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md).
*
* `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the
* `session.jsonl` fixtures against the real API and refreshes the stdout golden
* in one pass; the caller resolves that env into {@link SnapshotSuiteOptions}
* (env reading stays at the suite edge, not in this library).
*
* @module @deepseek-ai/dsh-acp-snapshot/suite
*/
import { readFile, readdir, writeFile } from 'node:fs/promises'
import { existsSync } from 'node:fs'
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import { type AgentUnderTest, type HarvestedLog, type InputScript, runScenario } from './harness.ts'
import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from './normalize.ts'
/** A snapshot scenario and how its fixtures are produced. */
export interface Scenario {
name: string
/** Whether the scenario drives at least one model turn (so a JSONL golden applies). */
hasModelTurn: boolean
/**
* Whether the run persists a comparable session log to diff against the
* `session.jsonl` fixture. Defaults to {@link hasModelTurn} (a model turn
* always produces a log worth comparing). Set it independently for a scenario
* that produces a non-trivial log WITHOUT a model turn — e.g. a prompt blocked
* by a `UserPromptSubmit` hook, which opens a `rejected` turn carrying `hook/*`
* events but never calls the model.
*/
comparesLog?: boolean
/**
* Whether `test:snapshot:record` regenerates this scenario's `session.jsonl`
* from the LIVE API. `recorded` scenarios are model-driven and reproducible;
* `authored` scenarios (a hand-written `replay.override.json` sidecar drives
* replay — e.g. a provider error or a cancel, which the live API can't be
* coaxed into deterministically — or a deterministic hook scenario whose
* derived empty script needs no sidecar) are NEVER re-recorded.
*/
recorded: boolean
/**
* How many SUBAGENT child sessions this scenario records beyond the top-level
* one (0 for a single-session scenario). Each child rides in a sibling fixture
* `session.<n>.jsonl` (1-based); replay forwards them to `dsh-llm-replay` so
* each child session replays from its own script, and record mode writes the
* harvested child logs back to those files. Defaults to 0.
*/
childSessions?: number
/**
* Whether THIS scenario's fixtures keep the full request-header content (the
* composed system prompt and tool schema list on `request/header` /
* `request/header-delta` events) and compare it verbatim. Exactly one
* scenario per suite pins it; every other scenario stores and compares that
* content as `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}),
* so a system prompt or tool-schema change shows up as ONE committed-fixture
* diff, not one per scenario. One pin suffices because header composition is
* suite-uniform (parent, spawn child, and fork child all compose the same
* prompt-modulo-cwd and the same tools) — and that premise is ASSERTED, not
* assumed: every non-pinning run's live headers must equal the pinned
* fixture's (normalized), so a session-dependent header (say, a restricted
* subagent toolset) fails loud until it gets its own pinning scenario.
* Defaults to false.
*/
pinsHeader?: boolean
}
/** One suite's inputs: the agent to boot, where its fixtures live, and its scenario table. */
export interface SnapshotSuiteOptions {
/** The agent composition every scenario boots. */
agent: AgentUnderTest
/** Absolute path of the suite's `snapshots/` directory (one subdir per scenario). */
snapshotsDir: string
/** The scenario table; exactly one entry must set `pinsHeader`. */
scenarios: Scenario[]
/**
* `replay` (keyless, the default tier) or `record` (live API; re-records the
* `recorded` scenarios' fixtures and refreshes the vitest goldens under
* `--update`). The caller derives this from `$DSH_SNAPSHOT` — env reading
* stays outside this library.
*/
mode: 'replay' | 'record'
}
/**
* The sibling child-fixture paths for a scenario (`session.1.jsonl` …).
*
* @param dir The scenario's snapshots directory (`<snapshotsDir>/<name>`).
* @param childSessions How many subagent child sessions the scenario records.
* @returns One path per child, 1-based, in fixture order.
*/
export function childFixturePaths(dir: string, childSessions: number): string[] {
return Array.from({ length: childSessions }, (_, i) => join(dir, `session.${i + 1}.jsonl`))
}
/**
* Derive the {@link NormalizeContext} for a `session.jsonl` fixture from its own
* header line (`{ type: 'session', id, cwd }`). A committed fixture carries the
* session id and cwd of the run that harvested it — different from the live
* replay run — so normalizing it against the live run's ctx would leave those
* recorded values unscrubbed. Reading them from the header scrubs the fixture's
* own id/cwd to the same `{{sessionId}}`/`{{cwd}}` tokens the replay output gets.
* An authored fixture whose header is already normalized (`id:'{{sessionId}}'`,
* `cwd:'{{cwd}}'`) yields those tokens as the volatile values, so scrubbing them
* is an idempotent no-op. A header with no `cwd` falls back to a sentinel that
* cannot occur in a log (NOT `''`, which `String.split` would match on every
* character boundary and corrupt the output).
*
* @param fixture The committed `session.jsonl` content.
* @returns The fixture's own volatile values, ready for {@link normalizeSessionLog}.
*/
export function fixtureContext(fixture: string): NormalizeContext {
const firstLine = fixture.split('\n').find(line => line.trim().length > 0) ?? '{}'
const header = JSON.parse(firstLine) as { id?: unknown; cwd?: unknown }
return {
sessionIds: typeof header.id === 'string' ? [header.id] : [],
cwd: typeof header.cwd === 'string' ? header.cwd : '\0no-cwd\0',
}
}
/**
* The `data.header` payload of every `request/header` event in a session
* JSONL, in log order, with the log's volatile values scrubbed first
* ({@link normalizeSessionLog}) so headers harvested from different runs —
* each embedding its own temp cwd in the composed prompt — compare on equal
* footing.
*
* @param rawLog The session `.jsonl` content to extract headers from.
* @param ctx The volatile values of the run that produced it.
* @returns The normalized `data.header` payloads, in log order.
*/
export function normalizedHeaders(rawLog: string, ctx: NormalizeContext): unknown[] {
return normalizeSessionLog(rawLog, ctx)
.split('\n')
.filter(line => line.trim().length > 0)
.map(line => JSON.parse(line) as { type?: unknown; data?: { header?: unknown } })
.filter(record => record.type === 'request/header')
.map(record => record.data?.header)
}
/**
* Count the `request/header-delta` events in a session JSONL.
*
* @param rawLog The session `.jsonl` content.
* @returns How many `request/header-delta` events the log carries.
*/
export function headerDeltaCount(rawLog: string): number {
return rawLog.split('\n')
.filter(line => line.trim().length > 0)
.filter(line => (JSON.parse(line) as { type?: unknown }).type === 'request/header-delta')
.length
}
/**
* Register the suite: one `describe` per scenario (the golden/log compares and
* the header-uniformity guard) plus the fixture guard block (no orphan
* scenario dirs, required files present, exactly one pin, non-pinning fixtures
* header-scrubbed). Must run at vitest collection time — it calls
* `describe`/`it`. Throws immediately if no scenario pins the header (the
* uniformity guard would have nothing to compare against).
*
* @param options The agent, snapshots directory, scenario table, and mode.
*/
export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
const { agent, snapshotsDir, scenarios, mode } = options
const RECORDING = mode === 'record'
/** The suite's single header-pinning scenario. Guarded here (and by a meta-test) so the pin cannot silently vanish. */
const pinningScenario = scenarios.find(s => s.pinsHeader === true)
if (pinningScenario === undefined) throw new Error('acp-snapshot: no scenario pins the request-header content')
for (const scenario of scenarios) {
describe(`snapshot: ${scenario.name}`, () => {
// In RECORD mode, only re-run the `recorded` (live-API) scenarios; the
// `authored` ones (sidecar-driven errors/cancel) are never re-recorded.
it.skipIf(RECORDING && !scenario.recorded)('matches the goldens', async () => {
const dir = join(snapshotsDir, scenario.name)
const input = JSON.parse(await readFile(join(dir, 'input.json'), 'utf8')) as InputScript
const overrideFile = join(dir, 'replay.override.json')
const workspaceDir = join(dir, 'workspace')
const childSessions = scenario.childSessions ?? 0
const result = await runScenario(input, {
agent,
mode,
fixtureFile: join(dir, 'session.jsonl'),
...existsSync(overrideFile) ? { overrideFile } : {},
// In REPLAY, forward the recorded child fixtures so each subagent session
// replays from its own script. In RECORD they are harvested, not read.
...!RECORDING && childSessions > 0 ? { childFiles: childFixturePaths(dir, childSessions) } : {},
...existsSync(workspaceDir) ? { workspaceDir } : {},
})
// Scrub every volatile id the run produced: the ACP server-issued session
// id plus every harvested log's recorded id (a subagent child id never
// surfaces over ACP, but it appears in the child's own log header). The
// normalizer's UUID catch-all covers any we don't enumerate.
const ctx: NormalizeContext = {
sessionIds: [
...result.sessionId !== undefined ? [result.sessionId] : [],
...result.sessionLogs.map(l => l.id),
],
cwd: result.cwd,
}
// RECORD mode (recorded model scenarios only): persist the freshly-harvested
// logs back to their fixtures — the primary to session.jsonl, each child to
// session.<n>.jsonl in harvest order. `--update` refreshes the Vitest
// goldens but NOT these fixtures, so write them here. A non-pinning
// scenario's fixtures are written header-scrubbed, so a re-record can
// never smuggle the full prompt/schema content back into every fixture.
const scrub = scenario.pinsHeader === true
? (log: string): string => log
: scrubRequestHeaders
if (RECORDING && scenario.recorded && scenario.hasModelTurn) {
expect(result.sessionLogs.length, 'record produced no session log to harvest').toBeGreaterThan(0)
expect(result.sessionLogs.length, `expected ${childSessions + 1} session logs (parent + children)`)
.toBe(childSessions + 1)
await writeFile(join(dir, 'session.jsonl'), scrub((result.sessionLogs[0] as HarvestedLog).content))
for (let i = 1; i < result.sessionLogs.length; i++) {
await writeFile(join(dir, `session.${i}.jsonl`), scrub((result.sessionLogs[i] as HarvestedLog).content))
}
}
await expect(normalizeStdout(result.rawStdout, ctx))
.toMatchFileSnapshot(join(dir, 'stdout.golden.jsonl'))
// A model turn always produces a log worth comparing; a hook scenario can
// produce one without a model turn (a `rejected` turn carrying `hook/*`).
const comparesLog = scenario.comparesLog ?? scenario.hasModelTurn
if (comparesLog) {
// The harvested logs (primary-first) must match their committed fixtures
// 1:1. Each side passes through normalizeSessionLog, scrubbed against ITS
// OWN volatile values — the live run's via `ctx`, the committed fixture's
// via its own header (a committed file cannot share the live run's ids).
// Unless this scenario pins the header, both sides ALSO pass through
// scrubRequestHeaders: the live log carries the real prompt/schemas, the
// fixture carries the `{{system}}`/`{{tools}}` tokens, and the scrub is
// idempotent — so the compare checks the header's presence, position,
// reason, and config, but not its bulk content (pinned once, in the
// `pinsHeader` scenario).
expect(result.sessionLogs.length, 'this scenario must persist a session log').toBe(childSessions + 1)
const fixtureFiles = ['session.jsonl', ...Array.from({ length: childSessions }, (_, i) => `session.${i + 1}.jsonl`)]
for (let i = 0; i < fixtureFiles.length; i++) {
const harvested = scrub((result.sessionLogs[i] as HarvestedLog).content)
const fixture = scrub(await readFile(join(dir, fixtureFiles[i] as string), 'utf8'))
expect(normalizeSessionLog(harvested, ctx), `${fixtureFiles[i]} mismatch`)
.toEqual(normalizeSessionLog(fixture, fixtureContext(fixture)))
}
}
// Header-uniformity guard: the single pin is sound only while every
// session in the suite composes the SAME header and keeps it for the
// whole run. Assert both halves live. (1) Every request/header the run
// produced (parent, spawn child, fork child, initial or resume) must
// equal the pinned fixture's header after each side is normalized
// against its own volatile values. (2) No request/header-delta may
// appear at all — a mid-run header change diverges from the pin by
// construction, and its content would be invisible under the scrub. If
// either fails, either the header changed (update the pin: re-record or
// hand-edit the pinning scenario's fixture) or composition became
// session-dependent by design (give the divergent shape its own
// pinning scenario).
if (scenario.pinsHeader !== true) {
const pinnedFixture = await readFile(join(snapshotsDir, pinningScenario.name, 'session.jsonl'), 'utf8')
const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture))
expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`)
.toBe(1)
for (const log of result.sessionLogs) {
expect(headerDeltaCount(log.content), `session ${log.id}: a request/header-delta in a non-pinning scenario`)
.toBe(0)
const headers = normalizedHeaders(log.content, ctx)
for (const [k, header] of headers.entries()) {
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
.toEqual(pinned[0])
}
}
}
})
})
}
describe('snapshot fixtures', () => {
it('every scenario directory is registered (no orphans)', async () => {
// toMatchFileSnapshot does not prune orphaned golden/fixture files, so a
// renamed/removed scenario could leave a stale dir that nothing exercises.
// Fail loud on any snapshots/<dir> not present in the scenario table.
const entries = await readdir(snapshotsDir, { withFileTypes: true })
const onDisk = entries.filter(e => e.isDirectory()).map(e => e.name).sort()
const registered = scenarios.map(s => s.name).sort()
expect(onDisk).toEqual(registered)
})
it('every registered scenario has its required fixture files', () => {
// Every scenario has an input script and an stdout golden. EVERY scenario
// also needs `session.jsonl`: the suite boots `llm-replay` with that path
// as the replay source for ALL scenarios (the factory passes
// `fixtureFile: <dir>/session.jsonl` unconditionally), and `loadReplayScript`
// throws "fixture not found" when it is absent and no override replaces it.
// A no-model scenario ships a header-only `session.jsonl` (it derives to an
// empty script — no model call is made); a model scenario's fixture also
// doubles as the expected-log artifact the run is diffed against. An authored
// (non-`recorded`) model scenario additionally ships a `replay.override.json`
// sidecar for the throw/hang cases a derived script cannot express.
for (const { name, hasModelTurn, recorded, childSessions } of scenarios) {
const dir = join(snapshotsDir, name)
expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true)
expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true)
expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true)
if (hasModelTurn && !recorded) {
expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json`).toBe(true)
}
// A nested-agent scenario ships one child fixture per recorded subagent
// session (`session.1.jsonl` …), the replay source for that child session.
for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) {
expect(existsSync(childFixture), childFixture).toBe(true)
}
}
})
it('exactly one scenario pins the request-header content', () => {
// Zero pins would drop the prompt/schema surface from the suite entirely;
// two would split it. One pin per suite is the design (pinned-header RFC);
// WHICH scenario pins is the scenario table's reviewable choice.
expect(scenarios.filter(s => s.pinsHeader === true).map(s => s.name)).toEqual([pinningScenario.name])
})
it('committed fixtures carry request-header content ONLY in the pinning scenario', async () => {
// The whole point of the pin: a system-prompt or tool-schema change must
// churn exactly one committed line. A non-pinning fixture that carries the
// full header (a hand-recorded file, or a header line hand-edited out of
// its canonical JSON form) silently reopens the suite-wide churn, so fail
// loud here: every non-pinning session*.jsonl must be a fixed point of
// scrubRequestHeaders (apply the scrub to fix a violation), and the
// pinning scenario's fixtures must NOT be (their content IS the pin).
for (const scenario of scenarios) {
const dir = join(snapshotsDir, scenario.name)
const files = [
'session.jsonl',
...Array.from({ length: scenario.childSessions ?? 0 }, (_, i) => `session.${i + 1}.jsonl`),
]
for (const file of files) {
const fixture = await readFile(join(dir, file), 'utf8')
if (scenario.pinsHeader === true) {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must PIN the full header content`)
.not.toEqual(fixture)
} else {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} carries unscrubbed header content`)
.toEqual(fixture)
}
}
}
})
})
}

View File

@@ -0,0 +1,232 @@
/**
* Scripted fake ACP agent bin for `dsh-acp-snapshot`'s unit specs. Speaks
* newline-delimited JSON-RPC on stdio like the real `dsh-acp-agent` bin, but
* every behavior — how prompts settle, whether session/new rejects, which
* session logs get persisted, what filesystem noise to leave — comes from a
* `behavior.json` sitting NEXT to the `$DSH_SNAPSHOT_FILE` fixture, so a spec
* scripts a whole subprocess run from data. The specs launch it through the
* REAL `runScenario` spawn path (tsx loader, temp cwd, env plumbing), so the
* harness plumbing is exercised for real; only the agent behind the protocol
* is scripted.
*
* The specs (not the golden tier) own this bin: it asserts nothing, echoes
* observable facts into `session/update` text chunks (env probe, permission
* outcome, seeded-workspace listing) for the spec to read off `rawStdout`, and
* exits 0 on stdin EOF after writing the scripted logs — mirroring the real
* bin's dispose-flush-exit shape.
*/
import { mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { readdirSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { randomUUID } from 'node:crypto'
import { createInterface } from 'node:readline'
/** One scripted session log: a file path under the sessions root plus its JSONL lines. */
interface ScriptedLog {
/** Path relative to `$DSH_SNAPSHOT_SESSIONS_ROOT`, e.g. `bucket/a.jsonl` (an empty dir segment is invalid). */
file: string
/**
* The JSONL records. String templates `{{CWD}}` and `{{SID}}` are replaced
* with the run's real cwd and the ACP session id this bin issued, so a
* written log carries genuine volatile values for the normalizers to scrub.
*/
lines: unknown[]
}
/** The whole scripted behavior for one run. Every field defaults to the least surprising choice. */
interface Behavior {
/** Reject every `session/new` (exercises the expect-error step without extra dirs). */
rejectNewSession?: boolean
/** Reject `session/new` only when `additionalDirectories` is non-empty (the real bridge's rule). */
rejectExtraDirs?: boolean
/** How `session/prompt` settles: a clean response, a JSON-RPC error, or a hang until `session/cancel`. */
prompt?: 'respond' | 'error' | 'hang-until-cancel'
/** Before responding to a prompt, send a `session/request_permission` request and echo its outcome as a chunk. */
permissionProbe?: boolean
/** Echo the `DSH_SNAPSHOT_*` env the harness set as a chunk (spec-side env-plumbing assertions). */
echoEnv?: boolean
/** Echo the sorted cwd listing as a chunk (spec-side workspace-seeding assertions). */
echoWorkspace?: boolean
/** Write a line to stderr on boot (spec-side stderr-capture assertions). */
stderrNote?: string
/** Session logs to persist on stdin EOF. */
logs?: ScriptedLog[]
/** Leave a stray FILE directly under the sessions root (harvest must skip it). */
strayRootFile?: boolean
/** Leave a stray non-`.jsonl` file inside a bucket (harvest must skip it). */
strayBucketFile?: boolean
/** Delete the sessions root entirely (harvest must yield no logs). */
deleteSessionsRoot?: boolean
}
const sessionsRoot = process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? ''
const fixtureFile = process.env.DSH_SNAPSHOT_FILE ?? ''
const behavior: Behavior = fixtureFile === ''
? {}
: JSON.parse(readFileSync(join(dirname(fixtureFile), 'behavior.json'), 'utf8')) as Behavior
if (behavior.stderrNote !== undefined) process.stderr.write(`${behavior.stderrNote}\n`)
let nextOutboundId = 1000
let sessionId = ''
/**
* The cwd the client passed to `session/new` — used verbatim for `{{CWD}}`
* substitution, mirroring the real bin (whose persisted header carries the
* session cwd as given, NOT `process.cwd()`, which the OS realpaths — on
* macOS `/var/folders/…` vs `/private/var/folders/…`).
*/
let sessionCwd = ''
/** The parked prompt request id while `hang-until-cancel` waits for the cancel notification. */
let parkedPromptId: number | string | null = null
/** Resolvers for permission-probe responses, keyed by outbound request id. */
const pendingPermission = new Map<number, (outcome: unknown) => void>()
function send(frame: Record<string, unknown>): void {
process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', ...frame })}\n`)
}
function respond(id: number | string, result: unknown): void {
send({ id, result })
}
function respondError(id: number | string, message: string): void {
send({ id, error: { code: -32603, message } })
}
function chunk(text: string): void {
send({
method: 'session/update',
params: { sessionId, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text } } },
})
}
/** Substitute the `{{CWD}}`/`{{SID}}` templates through a scripted log record. */
function instantiate(value: unknown): unknown {
if (typeof value === 'string') return value.split('{{CWD}}').join(sessionCwd).split('{{SID}}').join(sessionId)
if (Array.isArray(value)) return value.map(instantiate)
if (value !== null && typeof value === 'object') {
const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(value)) out[k] = instantiate(v)
return out
}
return value
}
async function handlePrompt(id: number | string): Promise<void> {
if ((behavior.prompt ?? 'respond') === 'hang-until-cancel') {
// A thought chunk BEFORE any message chunk: a promptAndCancel waiter
// watches for agent_message_chunk, so this exercises its non-matching
// update path while the waiter is armed.
send({
method: 'session/update',
params: { sessionId, update: { sessionUpdate: 'agent_thought_chunk', content: { type: 'text', text: 'mulling' } } },
})
}
chunk('thinking about it')
if (behavior.echoEnv === true) {
chunk(`env:${JSON.stringify({
mode: process.env.DSH_SNAPSHOT,
override: process.env.DSH_SNAPSHOT_OVERRIDE ?? null,
childFiles: process.env.DSH_SNAPSHOT_CHILD_FILES ?? null,
})}`)
}
if (behavior.echoWorkspace === true) {
chunk(`workspace:${readdirSync(process.cwd()).sort().join(',')}`)
}
if (behavior.permissionProbe === true) {
const requestId = nextOutboundId++
const outcome = await new Promise<unknown>((resolve) => {
pendingPermission.set(requestId, resolve)
send({
id: requestId,
method: 'session/request_permission',
params: {
sessionId,
toolCall: { toolCallId: 'call_fake_1', title: 'fake tool', kind: 'execute', status: 'pending' },
options: [
{ optionId: 'opt-allow', name: 'Allow once', kind: 'allow_once' },
{ optionId: 'opt-reject', name: 'Reject once', kind: 'reject_once' },
],
},
})
})
chunk(`permission:${JSON.stringify(outcome)}`)
}
switch (behavior.prompt ?? 'respond') {
case 'respond':
respond(id, { stopReason: 'end_turn' })
return
case 'error':
respondError(id, 'model exploded')
return
case 'hang-until-cancel':
parkedPromptId = id
return
}
}
function handleFrame(frame: Record<string, unknown>): void {
const id = frame.id as number | string | undefined
const method = frame.method as string | undefined
const params = (frame.params ?? {}) as Record<string, unknown>
// A response to one of OUR outbound requests (the permission probe).
if (method === undefined && id !== undefined && typeof id === 'number' && pendingPermission.has(id)) {
const resolve = pendingPermission.get(id) as (outcome: unknown) => void
pendingPermission.delete(id)
resolve((frame.result as { outcome?: unknown } | undefined)?.outcome ?? null)
return
}
switch (method) {
case 'initialize':
respond(id as number | string, { protocolVersion: 1, agentCapabilities: { loadSession: false } })
return
case 'session/new': {
const extra = params.additionalDirectories as unknown[] | undefined
if (behavior.rejectNewSession === true || (behavior.rejectExtraDirs === true && extra !== undefined && extra.length > 0)) {
respondError(id as number | string, 'unsupported workspace scope')
return
}
sessionId = randomUUID()
sessionCwd = typeof params.cwd === 'string' ? params.cwd : process.cwd()
respond(id as number | string, { sessionId })
return
}
case 'session/prompt':
void handlePrompt(id as number | string)
return
case 'session/cancel':
if (parkedPromptId !== null) {
const parked = parkedPromptId
parkedPromptId = null
respond(parked, { stopReason: 'cancelled' })
}
return
default:
// Unknown method: a notification is ignored; a request gets an error so
// the SDK never waits forever on a frame this fake doesn't model.
if (id !== undefined) respondError(id, `unhandled method ${String(method)}`)
}
}
function flushLogsAndExit(): void {
for (const log of behavior.logs ?? []) {
const target = join(sessionsRoot, log.file)
mkdirSync(dirname(target), { recursive: true })
writeFileSync(target, log.lines.map(l => JSON.stringify(instantiate(l))).join('\n') + '\n')
}
if (behavior.strayRootFile === true) writeFileSync(join(sessionsRoot, 'stray.txt'), 'not a bucket\n')
if (behavior.strayBucketFile === true) {
mkdirSync(join(sessionsRoot, 'bucket-noise'), { recursive: true })
writeFileSync(join(sessionsRoot, 'bucket-noise', 'notes.txt'), 'not a session log\n')
}
if (behavior.deleteSessionsRoot === true) rmSync(sessionsRoot, { recursive: true, force: true })
process.exit(0)
}
const rl = createInterface({ input: process.stdin })
rl.on('line', (line) => {
if (line.trim().length === 0) return
handleFrame(JSON.parse(line) as Record<string, unknown>)
})
rl.on('close', () => { flushLogsAndExit() })

View File

@@ -0,0 +1,13 @@
{
"prompt": "respond",
"logs": [
{ "file": "b/parent.jsonl", "lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 700, "cwd": "{{CWD}}" },
{ "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }
]},
{ "file": "b/child.jsonl", "lines": [
{ "type": "session", "id": "abababab-cdcd-4efe-8ada-badabadabada", "createdAt": 800, "cwd": "{{CWD}}", "parentSession": "{{SID}}" },
{ "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }
]}
]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "rec child" }] }

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"abababab-cdcd-4efe-8ada-badabadabada","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88"}
{"type":"request/header","seq":0,"time":2,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88","createdAt":700,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW"}
{"type":"request/header","seq":0,"time":3,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}

View File

@@ -0,0 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}

View File

@@ -0,0 +1,10 @@
{
"prompt": "respond",
"logs": [{
"file": "b/main.jsonl",
"lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 600, "cwd": "{{CWD}}" },
{ "type": "request/header", "seq": 0, "time": 4, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }
]
}]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "rec pin" }] }

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"ccdc749f-56f3-4267-9750-598b5c60b7b2","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy"}
{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}

View File

@@ -0,0 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }] }

View File

@@ -0,0 +1 @@
[{ "kind": "hang" }]

View File

@@ -0,0 +1 @@
{"type":"session","id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"}

View File

@@ -0,0 +1 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}

View File

@@ -0,0 +1,10 @@
{
"prompt": "error",
"logs": [{
"file": "b/main.jsonl",
"lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 500, "cwd": "{{CWD}}" },
{ "type": "turn/end", "seq": 1, "time": 9, "data": { "error": "model exploded" } }
]
}]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "promptExpectError", "text": "boom" }] }

View File

@@ -0,0 +1 @@
[{ "kind": "throw", "chunks": [], "message": "model exploded", "code": "PROVIDER" }]

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"44444444-3333-4222-8111-000000000000","createdAt":17,"cwd":"/rec/authored-cwd"}
{"type":"turn/end","seq":1,"time":17,"data":{"error":"model exploded"}}

View File

@@ -0,0 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"model exploded"}}

View File

@@ -0,0 +1,10 @@
{
"prompt": "error",
"logs": [{
"file": "b/main.jsonl",
"lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 400, "cwd": "{{CWD}}" },
{ "type": "hook/result", "seq": 1, "time": 8, "data": { "decision": "block", "durationMs": 37 } }
]
}]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "promptExpectError", "text": "blocked" }] }

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"99999999-8888-4777-8666-555555555555","createdAt":13,"cwd":"/rec/blocked-cwd"}
{"type":"hook/result","seq":1,"time":13,"data":{"decision":"block","durationMs":99}}

View File

@@ -0,0 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","id":3,"error":{"code":-32603,"message":"model exploded"}}

View File

@@ -0,0 +1 @@
{}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }] }

View File

@@ -0,0 +1 @@
{"type":"session","id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}"}

View File

@@ -0,0 +1 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}

View File

@@ -0,0 +1,11 @@
{
"prompt": "respond",
"logs": [{
"file": "b/main.jsonl",
"lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}" },
{ "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } },
{ "type": "turn/start", "seq": 1, "time": 100, "data": { "turn": 1 } }
]
}]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "pin" }] }

View File

@@ -0,0 +1,3 @@
{"type":"session","id":"12121212-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/pin-cwd"}
{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"turn/start","seq":1,"time":7,"data":{"turn":1}}

View File

@@ -0,0 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}

View File

@@ -0,0 +1,15 @@
{
"prompt": "respond",
"echoWorkspace": true,
"logs": [
{ "file": "b/parent.jsonl", "lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 200, "cwd": "{{CWD}}" },
{ "type": "request/header", "seq": 0, "time": 5, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } },
{ "type": "assistant/chunk", "seq": 1, "time": 5, "data": { "turn": 1, "step": 1, "chunk": { "type": "text-delta", "index": 0, "text": "hi" } } }
]},
{ "file": "b/child.jsonl", "lines": [
{ "type": "session", "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}" },
{ "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }
]}
]
}

View File

@@ -0,0 +1 @@
{ "steps": [{ "op": "initialize" }, { "op": "newSession" }, { "op": "prompt", "text": "plain" }] }

View File

@@ -0,0 +1,2 @@
{"type":"session","id":"eeeeeeee-1111-4222-8333-444444444444","createdAt":12,"cwd":"/rec/plain-cwd","parentSession":"56565656-7878-4989-8a9a-9b9b9b9b9b9b"}
{"type":"request/header","seq":0,"time":12,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}

View File

@@ -0,0 +1,3 @@
{"type":"session","id":"56565656-7878-4989-8a9a-9b9b9b9b9b9b","createdAt":11,"cwd":"/rec/plain-cwd"}
{"type":"request/header","seq":0,"time":11,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
{"type":"assistant/chunk","seq":1,"time":11,"data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"hi"}}}

View File

@@ -0,0 +1,5 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":false}}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"thinking about it"}}}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"workspace:seed.txt"}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}

View File

@@ -0,0 +1 @@
seeded

View File

@@ -0,0 +1,272 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { delimiter, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { afterAll, describe, expect, it } from 'vitest'
import { runScenario, type AgentUnderTest, type InputStep } from '../src/harness.ts'
/**
* Unit tests for the subprocess harness, driven through the REAL spawn path
* (tsx loader, temp cwd, env plumbing) against the scripted fake ACP bin in
* ./fixtures/fake-acp-agent.ts. Each case writes a `behavior.json` next to a
* throwaway fixture path; the fake bin echoes observable facts (env, seeded
* workspace, permission outcomes) into `agent_message_chunk` text, so the
* assertions read plain `rawStdout`.
*/
const AGENT: AgentUnderTest = {
binScript: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)),
// The fake bin ignores its config argv; any real path documents the shape.
configPath: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)),
tsconfigPath: fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)),
}
/** Temp scenario dirs to drop after the suite. */
const tempDirs: string[] = []
afterAll(async () => {
for (const dir of tempDirs) await rm(dir, { recursive: true, force: true })
})
/** Write a behavior.json into a fresh temp dir; return the sibling fixture path the harness points the bin at. */
async function scenario(behavior: object): Promise<{ dir: string; fixtureFile: string }> {
const dir = await mkdtemp(join(tmpdir(), 'acp-snap-spec-'))
tempDirs.push(dir)
await writeFile(join(dir, 'behavior.json'), JSON.stringify(behavior))
return { dir, fixtureFile: join(dir, 'session.jsonl') }
}
const boot: InputStep[] = [{ op: 'initialize' }, { op: 'newSession' }]
describe('runScenario', () => {
it('drives a full turn: initialize (terminal caps), session, prompt, permission stub, harvest', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({
permissionProbe: true,
logs: [{
file: 'bucket/main.jsonl',
lines: [
{ type: 'session', id: '{{SID}}', createdAt: 42, cwd: '{{CWD}}' },
{ type: 'turn/start', seq: 1, time: 9, data: { turn: 1 } },
],
}],
})
const result = await runScenario(
{ steps: [{ op: 'initialize', terminalOutput: true }, { op: 'newSession' }, { op: 'prompt', text: 'go' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.sessionId).toBeDefined()
// The harness's client answers a permission request with `cancelled`; the
// fake bin echoes the outcome it received back as a chunk.
expect(result.rawStdout).toContain('permission:{\\"outcome\\":\\"cancelled\\"}')
expect(result.sessionLogs).toHaveLength(1)
expect(result.sessionLogs[0]?.id).toBe(result.sessionId)
expect(result.sessionLogs[0]?.createdAt).toBe(42)
expect(result.sessionLogs[0]?.content).toContain('turn/start')
// The harvested log embeds the run's REAL temp cwd (template-substituted).
expect(result.sessionLogs[0]?.content).toContain(result.cwd)
})
it('forwards override/child fixture paths into the child env and captures stderr', { timeout: 20_000 }, async () => {
const { dir, fixtureFile } = await scenario({ echoEnv: true, stderrNote: 'fake bin booted' })
const childFiles = [join(dir, 'session.1.jsonl'), join(dir, 'session.2.jsonl')]
const result = await runScenario(
{ steps: [...boot, { op: 'prompt', text: 'env?' }] },
{
agent: AGENT,
mode: 'replay',
fixtureFile,
overrideFile: join(dir, 'replay.override.json'),
childFiles,
// A workspaceDir that does not exist is skipped, not an error.
workspaceDir: join(dir, 'no-such-workspace'),
},
)
expect(result.stderr).toContain('fake bin booted')
expect(result.rawStdout).toContain('replay.override.json')
// Child paths ride one env var, joined with the platform delimiter.
expect(result.rawStdout).toContain(JSON.stringify(childFiles.join(delimiter)).slice(1, -1))
})
it('seeds the workspace dir into the temp cwd before the run', { timeout: 20_000 }, async () => {
const { dir, fixtureFile } = await scenario({ echoWorkspace: true })
const workspaceDir = join(dir, 'workspace')
await writeFile(join(dir, 'behavior.json'), JSON.stringify({ echoWorkspace: true }))
const { mkdir } = await import('node:fs/promises')
await mkdir(workspaceDir, { recursive: true })
await writeFile(join(workspaceDir, 'seeded.txt'), 'hello')
const result = await runScenario(
{ steps: [...boot, { op: 'prompt', text: 'ls' }] },
{ agent: AGENT, mode: 'replay', fixtureFile, workspaceDir },
)
expect(result.rawStdout).toContain('workspace:seeded.txt')
})
it('promptAndCancel waits for the streamed chunk, cancels, and settles the prompt', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ prompt: 'hang-until-cancel' })
const result = await runScenario(
{ steps: [...boot, { op: 'promptAndCancel', text: 'hang' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.rawStdout).toContain('"stopReason":"cancelled"')
// The streamed chunk deterministically precedes the cancelled response.
expect(result.rawStdout.indexOf('thinking about it')).toBeLessThan(result.rawStdout.indexOf('cancelled'))
})
it('promptExpectError swallows a model-error response as the expected outcome', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ prompt: 'error' })
const result = await runScenario(
{ steps: [...boot, { op: 'promptExpectError', text: 'boom' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.rawStdout).toContain('model exploded')
})
it('promptExpectError throws when the prompt unexpectedly succeeds (and teardown kills the live child)', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ prompt: 'respond' })
await expect(runScenario(
{ steps: [...boot, { op: 'promptExpectError', text: 'fine' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)).rejects.toThrow(/expected the prompt to fail/)
})
it('newSessionExpectError swallows the rejection, with and without extra dirs', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ rejectExtraDirs: true })
const result = await runScenario(
{ steps: [{ op: 'initialize' }, { op: 'newSessionExpectError', additionalDirectories: ['/elsewhere'] }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
// No session was created, so no id and no logs.
expect(result.sessionId).toBeUndefined()
expect(result.sessionLogs).toHaveLength(0)
const rejectAll = await scenario({ rejectNewSession: true })
const second = await runScenario(
{ steps: [{ op: 'initialize' }, { op: 'newSessionExpectError' }] },
{ agent: AGENT, mode: 'replay', fixtureFile: rejectAll.fixtureFile },
)
expect(second.rawStdout).toContain('unsupported workspace scope')
})
it('newSessionExpectError throws when session/new unexpectedly succeeds', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({})
await expect(runScenario(
{ steps: [{ op: 'initialize' }, { op: 'newSessionExpectError' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)).rejects.toThrow(/expected session\/new to be rejected/)
})
it('a plain cancel step is forwarded (and ignored by an idle agent)', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({})
const result = await runScenario(
{ steps: [...boot, { op: 'cancel' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.sessionId).toBeDefined()
})
it.each([
[{ op: 'prompt', text: 'x' }, /prompt before newSession/],
[{ op: 'promptExpectError', text: 'x' }, /promptExpectError before newSession/],
[{ op: 'promptAndCancel', text: 'x' }, /promptAndCancel before newSession/],
[{ op: 'cancel' }, /cancel before newSession/],
] as [InputStep, RegExp][])('rejects %j before newSession', { timeout: 20_000 }, async (step, message) => {
const { fixtureFile } = await scenario({})
await expect(runScenario(
{ steps: [{ op: 'initialize' }, step] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)).rejects.toThrow(message)
})
it('rejects an unknown input op', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({})
const bogus = { op: 'reticulate' } as unknown as InputStep
await expect(runScenario(
{ steps: [bogus] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)).rejects.toThrow(/unknown input op/)
})
it('harvests all logs primary-first, children by createdAt then id, skipping filesystem noise', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({
strayRootFile: true,
strayBucketFile: true,
logs: [
// File names chosen so readdir feeds the sort children-first AND
// parent-in-the-middle: the comparator then sees a parent on both
// sides of a pair, plus the same-createdAt (localeCompare) tiebreak.
{ file: 'b1/aa-child-c.jsonl', lines: [{ type: 'session', id: 'cccccccc-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] },
{ file: 'b1/bb-parent.jsonl', lines: [{ type: 'session', id: '{{SID}}', createdAt: 900 }] },
{ file: 'b1/cc-child-a.jsonl', lines: [{ type: 'session', id: 'aaaaaaaa-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] },
// Missing id/createdAt fall back to ''/0; earliest child by createdAt.
{ file: 'b2/orphan-fields.jsonl', lines: [{ type: 'session', parentSession: '{{SID}}' }] },
],
})
const result = await runScenario(
{ steps: [...boot, { op: 'prompt', text: 'go' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.sessionLogs.map(l => [l.id, l.createdAt])).toEqual([
[result.sessionId, 900],
['', 0],
['aaaaaaaa-0000-4000-8000-000000000000', 500],
['cccccccc-0000-4000-8000-000000000000', 500],
])
expect(result.sessionLogs[1]?.parentSession).toBe(result.sessionId)
})
it('treats an empty log file as a header-less primary with default fields', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ logs: [{ file: 'b/empty.jsonl', lines: [] }] })
const result = await runScenario(
{ steps: boot },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.sessionLogs.map(l => [l.id, l.createdAt, l.parentSession])).toEqual([['', 0, undefined]])
})
it('yields no logs when the sessions root vanished', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ deleteSessionsRoot: true })
const result = await runScenario(
{ steps: boot },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.sessionLogs).toHaveLength(0)
})
it('answers permission requests from the scripted queue by option kind, falling back to cancelled', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ permissionProbe: true })
// Two prompts → two permission round-trips; one scripted answer, so the
// second request exercises the exhausted-queue fallback.
const result = await runScenario(
{
steps: [...boot, { op: 'prompt', text: 'one' }, { op: 'prompt', text: 'two' }],
permissionAnswers: [{ kind: 'allow_once' }],
},
{ agent: AGENT, mode: 'replay', fixtureFile },
)
const first = result.rawStdout.indexOf('permission:{\\"outcome\\":\\"selected\\",\\"optionId\\":\\"opt-allow\\"}')
const second = result.rawStdout.indexOf('permission:{\\"outcome\\":\\"cancelled\\"}')
expect(first).toBeGreaterThanOrEqual(0)
expect(second).toBeGreaterThan(first)
})
it('selects a non-first offered option by kind', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ permissionProbe: true })
const result = await runScenario(
{ steps: [...boot, { op: 'prompt', text: 'deny it' }], permissionAnswers: [{ kind: 'reject_once' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)
expect(result.rawStdout).toContain('permission:{\\"outcome\\":\\"selected\\",\\"optionId\\":\\"opt-reject\\"}')
})
it('rejects the run on a scripted permission kind the agent never offered', { timeout: 20_000 }, async () => {
const { fixtureFile } = await scenario({ permissionProbe: true })
// The fake bin offers allow_once/reject_once; scripting allow_always is a
// scenario bug. The agent is answered `cancelled` (it must not be able to
// absorb the bug as an error-means-denial), and the RUN fails: a callback
// throw would only reach the agent as a JSON-RPC error response, letting
// a tolerant agent carry on and the scenario pass — or record.
await expect(runScenario(
{ steps: [...boot, { op: 'prompt', text: 'impossible click' }], permissionAnswers: [{ kind: 'allow_always' }] },
{ agent: AGENT, mode: 'replay', fixtureFile },
)).rejects.toThrow(/allow_always not among the offered options \[allow_once, reject_once\]/)
})
})

View File

@@ -0,0 +1,230 @@
import { describe, expect, it } from 'vitest'
import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from '../src/normalize.ts'
/**
* Unit tests for the pure snapshot normalizers. Live as a *.spec.ts (runs in
* the default unit gate) and import the normalizers directly.
*/
const ctx: NormalizeContext = {
sessionIds: ['11111111-2222-3333-4444-555555555555'],
cwd: '/tmp/acp-snap-cwd-abc123',
}
describe('normalizeStdout', () => {
it('rewrites JSON-RPC ids to a stable first-seen sequence', () => {
const raw = [
JSON.stringify({ jsonrpc: '2.0', id: 42, method: 'initialize' }),
JSON.stringify({ jsonrpc: '2.0', id: 42, result: {} }),
JSON.stringify({ jsonrpc: '2.0', id: 99, method: 'session/new' }),
].join('\n')
const out = normalizeStdout(raw, ctx)
expect(out).toContain('"id":1')
expect(out).toContain('"id":2')
expect(out).not.toContain('42')
expect(out).not.toContain('99')
})
it('scrubs the cwd and session id anywhere they appear', () => {
const raw = JSON.stringify({
jsonrpc: '2.0', method: 'session/update',
params: { sessionId: ctx.sessionIds[0], cwd: ctx.cwd, note: `at ${ctx.cwd}/x` },
})
const out = normalizeStdout(raw, ctx)
expect(out).toContain('{{sessionId}}')
expect(out).toContain('{{cwd}}')
expect(out).not.toContain(ctx.cwd)
expect(out).not.toContain(ctx.sessionIds[0] as string)
})
it('scrubs a stray UUID not in the known list', () => {
const raw = JSON.stringify({ jsonrpc: '2.0', method: 'x', params: { id: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee' } })
expect(normalizeStdout(raw, ctx)).toContain('{{sessionId}}')
})
it('leaves notification frames without an id untouched in id-space', () => {
const raw = JSON.stringify({ jsonrpc: '2.0', method: 'session/update', params: {} })
const out = normalizeStdout(raw, ctx)
expect(out).not.toContain('"id"')
})
it('throws on a non-JSON stdout line (the purity check)', () => {
const raw = `${JSON.stringify({ jsonrpc: '2.0', id: 1 })}\noops a log leaked\n`
expect(() => normalizeStdout(raw, ctx)).toThrow()
})
it('ignores blank lines', () => {
const raw = `\n${JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'm' })}\n\n`
expect(() => normalizeStdout(raw, ctx)).not.toThrow()
})
})
describe('normalizeSessionLog', () => {
const header = (over: object) => JSON.stringify({ type: 'session', version: 0, id: 's', createdAt: 123, ...over })
const event = (over: object) => JSON.stringify({ type: 'turn/start', seq: 1, time: 999, data: { turn: 1 }, ...over })
it('zeroes the header createdAt', () => {
const out = normalizeSessionLog(`${header({})}\n`, ctx)
expect(out).toContain('"createdAt":0')
expect(out).not.toContain('123')
})
it('zeroes each event time but keeps seq', () => {
const out = normalizeSessionLog(`${header({})}\n${event({ seq: 7, time: 999 })}\n`, ctx)
expect(out).toContain('"time":0')
expect(out).toContain('"seq":7') // seq is deterministic — NOT scrubbed
expect(out).not.toContain('999')
})
it('scrubs cwd and session id deep inside event data', () => {
const ev = JSON.stringify({
type: 'tool/result', seq: 2, time: 5,
data: { content: [{ type: 'text', text: `wrote ${ctx.cwd}/proof.txt` }] },
})
const out = normalizeSessionLog(`${header({ cwd: ctx.cwd })}\n${ev}\n`, ctx)
expect(out).toContain('{{cwd}}')
expect(out).not.toContain(ctx.cwd)
})
it('scrubs the session id in the header', () => {
const out = normalizeSessionLog(`${header({ id: ctx.sessionIds[0] })}\n`, ctx)
expect(out).toContain('{{sessionId}}')
})
it('zeroes a hook/result durationMs (run-to-run noise) but keeps its decision', () => {
const ev = JSON.stringify({
type: 'hook/result', seq: 2, time: 5,
data: { turn: 1, point: 'UserPromptSubmit', handlerId: 'h', decision: 'block', exitCode: 2, durationMs: 37 },
})
const out = normalizeSessionLog(`${header({})}\n${ev}\n`, ctx)
expect(out).toContain('"durationMs":0')
expect(out).not.toContain('37')
expect(out).toContain('"decision":"block"') // the decision is the behavior — kept
})
it('leaves a non-hook event durationMs untouched (only hook/result is scrubbed)', () => {
const ev = JSON.stringify({ type: 'tool/result', seq: 2, time: 5, data: { durationMs: 88 } })
const out = normalizeSessionLog(`${header({})}\n${ev}\n`, ctx)
expect(out).toContain('"durationMs":88')
})
it('tolerates records missing the volatile fields it would zero', () => {
const bareHeader = JSON.stringify({ type: 'session', id: 's' })
const timeless = JSON.stringify({ type: 'note', seq: 1 })
const bareHook = JSON.stringify({ type: 'hook/result', seq: 2, time: 5, data: { decision: 'allow' } })
const nullDataHook = JSON.stringify({ type: 'hook/result', seq: 3, time: 6, data: null })
const out = normalizeSessionLog(`${bareHeader}\n${timeless}\n${bareHook}\n${nullDataHook}\n`, ctx)
expect(out).toContain('"type":"note","seq":1')
expect(out).toContain('"decision":"allow"')
expect(out).not.toContain('durationMs')
})
})
describe('scrubRequestHeaders', () => {
const headerLine = JSON.stringify({ type: 'session', version: 0, id: 's', createdAt: 1, cwd: '/w' })
const headerEvent = (header: object) =>
JSON.stringify({ type: 'request/header', seq: 3, time: 9, data: { header, reason: 'initial' } })
it('replaces header system and tools with tokens, keeping config and reason', () => {
const ev = headerEvent({
config: { model: 'm' },
system: 'You are an agent.\nBe brief.',
tools: [{ name: 'read', description: 'Read a file.', parameters: { type: 'object' } }],
})
const out = scrubRequestHeaders(`${headerLine}\n${ev}\n`)
expect(out).toContain('"system":"{{system}}"')
expect(out).toContain('"tools":"{{tools}}"')
expect(out).toContain('"config":{"model":"m"}')
expect(out).toContain('"reason":"initial"')
expect(out).not.toContain('You are an agent')
expect(out).not.toContain('Read a file')
})
it('keeps an absent system/tools absent (presence is behavior)', () => {
const out = scrubRequestHeaders(`${headerLine}\n${headerEvent({ config: { model: 'm' } })}\n`)
expect(out).not.toContain('{{system}}')
expect(out).not.toContain('{{tools}}')
})
it('scrubs a header carrying only one of system/tools, leaving the other absent', () => {
const systemOnly = scrubRequestHeaders(`${headerLine}\n${headerEvent({ system: 'secret prompt' })}\n`)
expect(systemOnly).toContain('"system":"{{system}}"')
expect(systemOnly).not.toContain('{{tools}}')
const toolsOnly = scrubRequestHeaders(`${headerLine}\n${headerEvent({ tools: [{ name: 't' }] })}\n`)
expect(toolsOnly).toContain('"tools":"{{tools}}"')
expect(toolsOnly).not.toContain('{{system}}')
})
it('leaves a delta with no scrubbable payload byte-identical (config-only, or non-array shapes)', () => {
const configOnly = JSON.stringify({ type: 'request/header-delta', seq: 8, time: 9, data: { config: { model: 'm2' } } })
const oddShapes = JSON.stringify({ type: 'request/header-delta', seq: 9, time: 9, data: { system: { insert: 'not-an-array' }, tools: null } })
const headerless = JSON.stringify({ type: 'request/header', seq: 10, time: 9, data: { reason: 'initial' } })
const nullData = JSON.stringify({ type: 'request/header', seq: 11, time: 9, data: null })
const raw = `${headerLine}\n${configOnly}\n${oddShapes}\n${headerless}\n${nullData}\n`
expect(scrubRequestHeaders(raw)).toBe(raw)
})
it('scrubs a one-sided tools delta and passes non-object schema entries through', () => {
const addedOnly = JSON.stringify({
type: 'request/header-delta', seq: 8, time: 9,
data: { tools: { added: [null, 'weird', { name: 'x', description: 'D' }] } },
})
const out = scrubRequestHeaders(`${headerLine}\n${addedOnly}\n`)
// Non-object entries survive untouched; the object entry keeps only name.
expect(out).toContain('"added":[null,"weird",{"name":"x","description":"{{tools}}"}]')
const changedOnly = JSON.stringify({
type: 'request/header-delta', seq: 8, time: 9,
data: { tools: { changed: [{ name: 'y', parameters: {} }] } },
})
expect(scrubRequestHeaders(`${headerLine}\n${changedOnly}\n`))
.toContain('"changed":[{"name":"y","parameters":"{{tools}}"}]')
})
it('scrubs a header-delta system payload but keeps its line positions and arity', () => {
const delta = JSON.stringify({
type: 'request/header-delta', seq: 8, time: 9,
data: { system: { keepStart: 1, keepEnd: 4, insert: ['leaked prompt line', 'second line'] }, config: { model: 'm2' } },
})
const out = scrubRequestHeaders(`${headerLine}\n${delta}\n`)
// One token PER inserted line: the edit's position AND extent survive.
expect(out).toContain('"insert":["{{system}}","{{system}}"]')
expect(out).toContain('"keepStart":1')
expect(out).toContain('"keepEnd":4')
expect(out).toContain('"config":{"model":"m2"}')
expect(out).not.toContain('leaked prompt line')
expect(out).not.toContain('{{tools}}') // no tools delta → none invented
})
it('scrubs a header-delta tools payload but keeps the added/removed/changed names', () => {
const delta = JSON.stringify({
type: 'request/header-delta', seq: 8, time: 9,
data: {
tools: {
added: [{ name: 'grep', description: 'Search files.', parameters: { type: 'object' } }],
removed: ['bash_kill'],
changed: [{ name: 'read', description: 'Read v2.', parameters: { type: 'object' } }],
},
},
})
const out = scrubRequestHeaders(`${headerLine}\n${delta}\n`)
// WHICH tools changed is behavior and survives; their bulk does not.
expect(out).toContain('"added":[{"name":"grep","description":"{{tools}}","parameters":"{{tools}}"}]')
expect(out).toContain('"removed":["bash_kill"]')
expect(out).toContain('"changed":[{"name":"read","description":"{{tools}}","parameters":"{{tools}}"}]')
expect(out).not.toContain('Search files')
expect(out).not.toContain('Read v2')
})
it('passes every other line through byte-for-byte and is idempotent', () => {
const other = JSON.stringify({ type: 'assistant/chunk', seq: 4, time: 9, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'hi' } } })
const delta = JSON.stringify({
type: 'request/header-delta', seq: 8, time: 9,
data: { system: { keepStart: 0, keepEnd: 0, insert: ['x'] }, tools: { added: [{ name: 't', description: 'd', parameters: {} }], removed: [], changed: [] } },
})
const raw = `${headerLine}\n${headerEvent({ config: { model: 'm' }, system: 's', tools: [] })}\n${delta}\n${other}\n`
const once = scrubRequestHeaders(raw)
expect(once.split('\n')[0]).toBe(headerLine)
expect(once.split('\n')[3]).toBe(other)
expect(scrubRequestHeaders(once)).toBe(once)
})
})

View File

@@ -0,0 +1,145 @@
import { cpSync, mkdtempSync } from 'node:fs'
import { rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { afterAll, describe, expect, it } from 'vitest'
import { defineAcpSnapshotSuite, type Scenario } from '../src/index.ts'
import { childFixturePaths, fixtureContext, headerDeltaCount, normalizedHeaders } from '../src/suite.ts'
/**
* Unit tests for the suite factory, by running it: two synthetic suites over
* the scripted fake ACP bin (./fixtures/fake-acp-agent.ts) register REAL
* describe/it trees at collection time, so every factory path — golden and log
* compares, the per-suite header pin and its uniformity guard, record-mode
* fixture write-back, skip semantics, and the fixture guard block — executes
* as an ordinary green test. The pure helpers get direct cases below.
*
* The replay suite runs against the committed fixtures in ./fixtures/suite.
* The record suite runs against a TEMP COPY of ./fixtures/record-suite
* (record mode writes session fixtures back into its snapshots dir; a run must
* never touch the committed tree). To re-bootstrap the record tree's goldens
* after changing the fake bin's output, run this spec once with
* `ACP_SNAPSHOT_SPEC_BOOTSTRAP=1` (points the record suite at the committed
* tree so vitest creates/updates the goldens and the write-back lands there),
* then commit the result.
*/
const AGENT = {
binScript: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)),
configPath: fileURLToPath(new URL('./fixtures/fake-acp-agent.ts', import.meta.url)),
tsconfigPath: fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url)),
}
const REPLAY_DIR = fileURLToPath(new URL('./fixtures/suite', import.meta.url))
const RECORD_SRC = fileURLToPath(new URL('./fixtures/record-suite', import.meta.url))
const REPLAY_SCENARIOS: Scenario[] = [
{ name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true },
{ name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1 },
{ name: 'no-model', hasModelTurn: false, recorded: false },
{ name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false },
{ name: 'authored-error', hasModelTurn: true, recorded: false },
]
const RECORD_SCENARIOS: Scenario[] = [
{ name: 'rec-pin', hasModelTurn: true, recorded: true, pinsHeader: true },
{ name: 'rec-child', hasModelTurn: true, recorded: true, childSessions: 1 },
// recorded:false in record mode → registered but skipped (never re-recorded).
{ name: 'rec-skip', hasModelTurn: true, recorded: false },
]
// Record mode mutates its snapshots dir, so run it on a throwaway copy —
// except under the documented bootstrap knob, which regenerates the committed
// fixtures/goldens in place.
const BOOTSTRAP = process.env.ACP_SNAPSHOT_SPEC_BOOTSTRAP === '1'
const recordDir = BOOTSTRAP ? RECORD_SRC : mkdtempSync(join(tmpdir(), 'acp-snap-record-suite-'))
if (!BOOTSTRAP) cpSync(RECORD_SRC, recordDir, { recursive: true })
afterAll(async () => {
if (!BOOTSTRAP) await rm(recordDir, { recursive: true, force: true })
})
describe('defineAcpSnapshotSuite: replay mode', () => {
defineAcpSnapshotSuite({ agent: AGENT, snapshotsDir: REPLAY_DIR, scenarios: REPLAY_SCENARIOS, mode: 'replay' })
})
// The record suite's tests run in registration order: rec-pin re-records the
// pinned fixture FIRST, so rec-child's uniformity guard reads the fresh pin.
describe('defineAcpSnapshotSuite: record mode', () => {
defineAcpSnapshotSuite({ agent: AGENT, snapshotsDir: recordDir, scenarios: RECORD_SCENARIOS, mode: 'record' })
})
describe('defineAcpSnapshotSuite: registration contract', () => {
it('throws when no scenario pins the request-header content', () => {
expect(() => {
defineAcpSnapshotSuite({
agent: AGENT,
snapshotsDir: REPLAY_DIR,
scenarios: [{ name: 'pinless', hasModelTurn: true, recorded: true }],
mode: 'replay',
})
}).toThrow(/no scenario pins/)
})
})
describe('childFixturePaths', () => {
it('yields one sibling path per child, 1-based', () => {
expect(childFixturePaths('/snap/s', 2)).toEqual(['/snap/s/session.1.jsonl', '/snap/s/session.2.jsonl'])
})
it('yields nothing for a single-session scenario', () => {
expect(childFixturePaths('/snap/s', 0)).toEqual([])
})
})
describe('fixtureContext', () => {
it('reads the fixture header id and cwd', () => {
const ctx = fixtureContext('{"type":"session","id":"abc","cwd":"/rec"}\n{"type":"turn/start"}\n')
expect(ctx).toEqual({ sessionIds: ['abc'], cwd: '/rec' })
})
it('yields no session ids for a header without a string id', () => {
expect(fixtureContext('{"type":"session","cwd":"/rec"}\n').sessionIds).toEqual([])
})
it('falls back to an impossible sentinel cwd (never the empty string)', () => {
const ctx = fixtureContext('{"type":"session","id":"abc"}\n')
expect(ctx.cwd).toBe('\0no-cwd\0')
expect(ctx.cwd).not.toBe('')
})
it('treats an empty fixture as an empty header', () => {
expect(fixtureContext('')).toEqual({ sessionIds: [], cwd: '\0no-cwd\0' })
})
})
describe('normalizedHeaders', () => {
const header = (system: string): string => JSON.stringify({
type: 'request/header', seq: 0, time: 9, data: { header: { config: { model: 'm' }, system }, reason: 'initial' },
})
it('extracts every request/header payload in log order, normalized', () => {
const id = '11111111-2222-4333-8444-555555555555'
const log = `${JSON.stringify({ type: 'session', id, createdAt: 5, cwd: '/w' })}\n${header('one')}\n`
+ `${JSON.stringify({ type: 'turn/start', seq: 1, time: 9, data: { turn: 1 } })}\n${header('two')}\n`
const headers = normalizedHeaders(log, { sessionIds: [id], cwd: '/w' })
expect(headers).toEqual([
{ config: { model: 'm' }, system: 'one' },
{ config: { model: 'm' }, system: 'two' },
])
})
it('yields nothing for a log without header events', () => {
const log = `${JSON.stringify({ type: 'session', id: 'a', createdAt: 5 })}\n`
expect(normalizedHeaders(log, { sessionIds: [], cwd: '/w' })).toEqual([])
})
})
describe('headerDeltaCount', () => {
it('counts request/header-delta events, ignoring blanks and other lines', () => {
const delta = JSON.stringify({ type: 'request/header-delta', seq: 2, time: 9, data: {} })
const other = JSON.stringify({ type: 'request/header', seq: 0, time: 9, data: {} })
expect(headerDeltaCount(`${other}\n\n${delta}\n${delta}\n`)).toBe(2)
expect(headerDeltaCount(`${other}\n`)).toBe(0)
})
})

View File

@@ -0,0 +1,11 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": []
}

View File

@@ -41,13 +41,13 @@ describe('dsh-subagent-mock', () => {
it('surfaces a structured result when the request carries an outputSchema', async () => {
const ctx = await mount({ reply: 'r', structured: { answer: 42 } })
const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { answer: { type: 'number' } } }))
const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { type: 'object', properties: { answer: { type: 'number' } } } }))
await expect(run.result).resolves.toMatchObject({ structured: { answer: 42 } })
})
it('defaults structured output to { reply } when outputSchema is requested but no structured value is configured', async () => {
const ctx = await mount({ reply: 'fallback reply' })
const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { answer: { type: 'number' } } }))
const run = ctx.subagents.start('mock', baseRequest({ outputSchema: { type: 'object', properties: { answer: { type: 'number' } } } }))
await expect(run.result).resolves.toMatchObject({ structured: { reply: 'fallback reply' } })
})

View File

@@ -76,9 +76,10 @@ async function makeConsumer(welcome: string, disabledBrokenEntry = false): Promi
await mkdir(dirname(target), { recursive: true })
await symlink(abs, target)
}
// The example's mock model + echo tool are example-local TS plugins (Node 24+
// strips types natively, so plain `node` loads them); they import the workspace
// packages the symlinked node_modules now provides.
// The example's mock model + echo tool are example-local TS plugins (Node
// 22.19+ — the engines floor — strips types natively, so plain `node` loads
// them); they import the workspace packages the symlinked node_modules now
// provides.
await cp(join(repoRoot, 'examples/echo-agent/src'), join(dir, 'src'), { recursive: true })
await writeFile(join(dir, 'cordis.yml'), [
'- id: mock-llm',

View File

@@ -1,6 +1,6 @@
/**
* `LocalFetchProvider`: a `WebFetchProvider` that retrieves a concrete public
* HTTP(S) URL with the platform-native `fetch` (Node 24) and returns a status
* HTTP(S) URL with platform-native `fetch` at the repo's Node floor and returns a status
* code plus bounded decoded content. It owns SAFE RESOURCE RETRIEVAL — URL
* validation, redirect policy, timeout, abort, byte caps, charset decoding,
* content-type classification, binary rejection — but NOT presentation

View File

@@ -12,7 +12,7 @@
* `web_search_tool_result` block (native search did not trigger), it throws
* `WEB_PROVIDER_ERROR` rather than degrading to prose-scraping.
*
* Network requests use platform-native `fetch` (Node 24), mirroring
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
* The Anthropic wire shape is a provider-private detail and does NOT make this
* provider depend on `ctx.llm`.

View File

@@ -6,7 +6,7 @@
* `title`, the first highlight as `snippet`, and `publishedDate` as
* `publishedAt`.
*
* Network requests use platform-native `fetch` (Node 24), mirroring
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter — not a cordis HTTP-client service.
*
* @module @deepseek-ai/dsh-web-search-exa/provider

View File

@@ -5,7 +5,7 @@
* structured `search_results[]` for `sources[]`, falling back to the URL-only
* `citations[]` when `search_results` is absent.
*
* Network requests use platform-native `fetch` (Node 24), mirroring
* Network requests use platform-native `fetch` at the repo's Node floor, mirroring
* `@deepseek-ai/dsh-llm-deepseek`'s adapter. The OpenAI-compatible request shape
* is a provider-private detail and does NOT make this provider depend on
* `ctx.llm`.