Merge remote-tracking branch 'origin/worktree-agent-scope-design' into codex/package-readme-limitations-audit-20260712

This commit is contained in:
Tianyi Cui
2026-07-12 01:46:59 +08:00
53 changed files with 1005 additions and 239 deletions

View File

@@ -453,7 +453,7 @@ export const EVENT_API: readonly EventApiEntry[] = [
name: 'workflow/agent-start',
mode: 'emit',
signature: '\'workflow/agent-start\'(info: WorkflowRunInfo, agent: WorkflowAgentInfo): void',
summary: 'One `agent()` call started a child run.',
summary: 'One `agent()` call established a ready child run.',
},
{
name: 'workflow/end',

View File

@@ -5,8 +5,8 @@ The ACP snapshot suite kit: the shared machinery behind the keyless snapshot tie
Three layers, importable separately:
- **`runScenario` (harness)** — boots the real agent bin as a subprocess via tsx (unbuilt, Loader path), drives it over ACP JSON-RPC stdio from a deterministic `input.json` script, tees raw stdout for the golden + purity check, and harvests every persisted session JSONL (parent + subagent children, primary-first) after a graceful stdin-EOF shutdown. Parameterized by `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath` — absolute paths; the subprocess cwd is a temp dir outside the repo).
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), and the composable `scrubRequestHeaders` (header bulk → `{{system}}`/`{{tools}}`, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record/refresh fixture write-back, the per-header-class pin with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, pinning fixtures well-formed, non-pinning fixtures header-scrubbed). Must be called at vitest collection time.
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), `scrubSystemPrompts` (prompt text → `{{system}}` in every JSONL), and `scrubRequestHeaders` (the remaining header bulk → `{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record/refresh fixture write-back, the per-header-class pin (`system-prompt.golden.md` plus the JSONL's full tool schemas) with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL prompt-scrubbed, non-pinning fixtures fully header-scrubbed). Must be called at vitest collection time.
A consuming `*.snapshot.ts` is the scenario table plus one factory call:
@@ -35,9 +35,9 @@ defineAcpSnapshotSuite({
})
```
A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode scenarios are the template.
A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode scenarios are the template. Each pinning directory's generated `system-prompt.golden.md` is the reviewable snapshot of the normalized composed prompt; `session.jsonl` stores `"system":"{{system}}"` while retaining the complete tool list.
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config RFC](../../../docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout plus comparable session-log goldens from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config RFC](../../../docs/rfc/implemented/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log goldens, and each pin's Markdown prompt snapshot from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
Constraints: `suite.ts` imports vitest, so the package is importable only inside a vitest run (the harness and normalizers have no such dependency but ship from the same entry). ACP-specific by design — the harness speaks the SDK's `ClientSideConnection`. Permission round-trips are scriptable: `InputScript.permissionAnswers` is a FIFO queue of option-kind selections (`allow_once`, `reject_once`, …) the client maps to the agent-issued `optionId` at answer time; an absent or exhausted queue answers `cancelled`, and a kind the request never offered rejects the run (the agent is answered `cancelled`, so a tolerant agent cannot absorb the scenario bug). Session config options are scriptable too: the `setConfigOption` step switches a knob over `session/set_config_option`, and `setConfigOptionExpectError` asserts the bridge rejects an unknown id or out-of-vocabulary value (the error frame stays in the transcript).

View File

@@ -3,7 +3,7 @@
* tier (`pnpm run test:snapshot`). Three layers, composable per example:
* the subprocess scenario harness ({@link runScenario}), the pure golden
* normalizers ({@link normalizeStdout} / {@link normalizeSessionLog} /
* {@link scrubRequestHeaders}), and the suite factory
* {@link scrubRequestHeaders} / {@link scrubSystemPrompts}), and the suite factory
* ({@link defineAcpSnapshotSuite}) that registers a scenario table as a full
* describe/it tree. An example's `*.snapshot.ts` supplies only its
* {@link AgentUnderTest} paths, its snapshots directory, and its
@@ -29,6 +29,7 @@ export {
normalizeSessionLog,
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
type NormalizeContext,
} from './normalize.ts'
export {

View File

@@ -12,14 +12,14 @@
* `durationMs` (wall-clock hook runtime) → 0. NOT scrubbed: the log's `seq`
* (deterministic — `seq = log.length`, part of the event-log contract).
*
* A separate, composable normalizer — {@link scrubRequestHeaders} — replaces
* the bulky request-header CONTENT (the composed system prompt, the tool
* schema list, and the session prefix) with
* `{{system}}`/`{{tools}}`/`{{messagePrefix}}` tokens. It is deliberately NOT
* folded into {@link normalizeSessionLog}: each suite's one header-pinning
* scenario compares that content verbatim, every other scenario composes the
* scrub in (the `pinsHeader` flag on the scenario table, consumed by the suite
* factory in ./suite.ts; see the pinned-header RFC,
* Separate, composable normalizers keep bulky request-header content out of
* session fixtures. {@link scrubSystemPrompts} replaces the composed system
* prompt in EVERY fixture; {@link scrubRequestHeaders} additionally replaces
* tool schemas and the session prefix outside each suite's header-pinning
* scenario. They are deliberately NOT folded into
* {@link normalizeSessionLog}: the suite factory composes the right scrub for
* each scenario and snapshots the pin's actual prompt as Markdown (see the
* pinned-header RFC,
* docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md).
*
* See docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md.
@@ -135,37 +135,37 @@ export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): stri
}
/**
* Replace request-header CONTENT in a session JSONL with stable tokens,
* keeping its structure: a `request/header` event's `data.header.system` →
* `{{system}}`, `data.header.tools` → `{{tools}}`, and
* `data.header.messagePrefix` → one `{{messagePrefix}}` token per message
* (the session prefix is model-visible bulk — an AGENTS digest, a skills
* catalog — so its COUNT stays a structural fact while its text never lands
* in a fixture); a
* `request/header-delta` event keeps every structural fact — the system
* delta's `keepStart`/`keepEnd` line positions and inserted-line COUNT (one
* `{{system}}` token per inserted line), the tools delta's
* added/removed/changed tool NAMES, the prefix replacement's message COUNT —
* and tokenizes only the bulk (prompt
* text; each added/changed schema's fields other than `name` → `{{tools}}`;
* each replacement prefix message → `{{messagePrefix}}`),
* so two different deltas still compare different.
* Absent fields stay absent — WHETHER a header carried a system prompt,
* tools, or a prefix is behavior and stays visible; `config` and `reason`
* are small and
* stable, so they stay verbatim (a model swap churns every fixture by design
* — it invalidates the recorded responses; a prompt/schema edit churns none —
* replay never reads this content, see dsh-llm-replay).
*
* Only lines with something to scrub are re-serialized; every other line
* passes through byte-for-byte, so the transform is idempotent and applying
* it to an already-scrubbed fixture is a no-op — the on-disk-fixtures guard
* in ./suite.ts relies on exactly that.
* Replace system-prompt content in request headers and header deltas with
* `{{system}}` tokens while retaining field presence and delta structure.
* Other header content stays verbatim, so a header-pinning fixture can keep
* its complete tool schemas while every JSONL fixture omits the prompt text.
* Lines without a system payload pass through byte-for-byte; the transform is
* idempotent.
*
* @param rawLog The raw session `.jsonl` content.
* @returns The JSONL with header content tokenized, other lines byte-identical.
* @returns The JSONL with system-prompt content tokenized.
*/
export function scrubSystemPrompts(rawLog: string): string {
return scrubHeaderContent(rawLog, false)
}
/**
* Replace all bulky request-header content in a session JSONL with stable
* tokens. This includes the system-prompt fields handled by
* {@link scrubSystemPrompts}, tool schemas, and session-prefix messages. It
* keeps system-delta line positions and arity, tool-delta names, prefix
* message counts, field presence, config, and reason. Lines without content
* to scrub pass through byte-for-byte, and the transform is idempotent.
*
* @param rawLog The raw session `.jsonl` content.
* @returns The JSONL with all header bulk tokenized, other lines byte-identical.
*/
export function scrubRequestHeaders(rawLog: string): string {
return scrubHeaderContent(rawLog, true)
}
/** Transform header content, optionally including tool schemas and the session prefix. */
function scrubHeaderContent(rawLog: string, scrubToolsAndPrefix: boolean): string {
const lines = rawLog.split('\n')
const out = lines.map((line) => {
if (line.trim().length === 0) return line
@@ -175,11 +175,14 @@ export function scrubRequestHeaders(rawLog: string): string {
if (record.type === 'request/header') {
const header = data.header as Record<string, unknown> | null | undefined
if (header === null || typeof header !== 'object') return line
if (!('system' in header) && !('tools' in header) && !('messagePrefix' in header)) return line
if ('system' in header) header.system = SYSTEM
if ('tools' in header) header.tools = TOOLS
if (Array.isArray(header.messagePrefix)) header.messagePrefix = header.messagePrefix.map(() => MESSAGE_PREFIX)
return JSON.stringify(record)
let touched = false
if ('system' in header) { header.system = SYSTEM; touched = true }
if (scrubToolsAndPrefix && 'tools' in header) { header.tools = TOOLS; touched = true }
if (scrubToolsAndPrefix && Array.isArray(header.messagePrefix)) {
header.messagePrefix = header.messagePrefix.map(() => MESSAGE_PREFIX)
touched = true
}
return touched ? JSON.stringify(record) : line
}
if (record.type === 'request/header-delta') {
let touched = false
@@ -189,11 +192,11 @@ export function scrubRequestHeaders(rawLog: string): string {
touched = true
}
const tools = data.tools as Record<string, unknown> | null | undefined
if (tools !== null && typeof tools === 'object') {
if (scrubToolsAndPrefix && tools !== null && typeof tools === 'object') {
if (Array.isArray(tools.added)) { tools.added = tools.added.map(scrubToolSchema); touched = true }
if (Array.isArray(tools.changed)) { tools.changed = tools.changed.map(scrubToolSchema); touched = true }
}
if (Array.isArray(data.messagePrefix)) {
if (scrubToolsAndPrefix && Array.isArray(data.messagePrefix)) {
data.messagePrefix = data.messagePrefix.map(() => MESSAGE_PREFIX)
touched = true
}

View File

@@ -10,15 +10,14 @@
* (recorded scenarios) and the expected produced log (both sides normalized
* before comparing).
*
* Request-header content (the composed system prompt + tool schemas riding on
* `request/header` events) is pinned by exactly ONE scenario per HEADER CLASS
* — scenarios that boot the same config compose the same header; each class's
* `pinsHeader` scenario commits it verbatim — and scrubbed to
* `{{system}}`/`{{tools}}` tokens in every other fixture and compare, so a
* prompt or tool-schema edit churns one committed line per class instead of
* every fixture. A per-run uniformity guard keeps each pin sound: every live
* header must equal its class's pinned one, and no header-delta may appear
* outside a pinning scenario (see the pinned-header RFC,
* Request-header content is pinned by exactly ONE scenario per HEADER CLASS —
* scenarios that boot the same config compose the same header. Every JSONL
* fixture scrubs the system prompt to `{{system}}`; each class's pinning
* scenario stores the readable prompt in `system-prompt.golden.md` and keeps its full
* tool schemas in `session.jsonl`, while every other fixture also scrubs tools
* to `{{tools}}`. A per-run uniformity guard compares both artifacts against
* every live header and forbids unrepresented header deltas (see the
* pinned-header RFC,
* docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md).
*
* `pnpm run test:snapshot:record` (DSH_SNAPSHOT=record + -u) re-records the
@@ -37,7 +36,16 @@ import { existsSync } from 'node:fs'
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import { type AgentUnderTest, type HarvestedLog, type InputScript, runScenario } from './harness.ts'
import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from './normalize.ts'
import {
type NormalizeContext,
normalizeSessionLog,
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
} from './normalize.ts'
/** The readable system-prompt snapshot beside each header-pinning fixture. */
const SYSTEM_PROMPT_SNAPSHOT = 'system-prompt.golden.md'
/** A snapshot scenario and how its fixtures are produced. */
export interface Scenario {
@@ -81,14 +89,13 @@ export interface Scenario {
*/
childSessions?: number
/**
* Whether THIS scenario's fixtures keep the full request-header content (the
* composed system prompt and tool schema list on `request/header` /
* `request/header-delta` events) and compare it verbatim. Exactly one
* scenario per HEADER CLASS ({@link headerClass}) pins it; every other
* scenario of that class stores and compares that content as
* `{{system}}`/`{{tools}}` tokens ({@link scrubRequestHeaders}),
* so a system prompt or tool-schema change shows up as ONE committed-fixture
* diff per class, not one per scenario. One pin per class suffices because
* Whether THIS scenario pins its header class's model-facing request-header
* content. Its actual composed prompt is maintained as a readable
* `system-prompt.golden.md`; its JSONL keeps full tool schemas but stores the prompt
* as `{{system}}`. Every other scenario of the class stores tools as
* `{{tools}}` too ({@link scrubRequestHeaders}). A prompt or tool-schema
* change therefore shows up in one focused artifact per class, not every
* session fixture. One pin per class suffices because
* header composition is class-uniform (parent, spawn child, and fork child
* all compose the same prompt-modulo-cwd and the same tools) — and that
* premise is ASSERTED, not assumed: every non-pinning run's live headers
@@ -102,7 +109,7 @@ export interface Scenario {
* How many `request/header-delta` events this PINNING scenario's fixture
* legitimately carries (default 0). A recorded mid-run header change — a
* config-option switch rewriting a prompt section — is part of the pinned
* surface, committed verbatim like the header itself; any OTHER count
* surface, with readable prompt text in Markdown; any OTHER count
* still fails, so fixture rot stays caught. Meaningless off the pin (the
* live uniformity guard keeps non-pinning scenarios delta-free).
*/
@@ -133,7 +140,7 @@ export interface SnapshotSuiteOptions {
agent: AgentUnderTest
/** Absolute path of the suite's `snapshots/` directory (one subdir per scenario). */
snapshotsDir: string
/** The scenario table; exactly one entry must set `pinsHeader`. */
/** The scenario table; exactly one entry per header class must set `pinsHeader`. */
scenarios: Scenario[]
/**
* `replay` (keyless, the default tier), `record` (live API; re-records the
@@ -201,6 +208,86 @@ export function normalizedHeaders(rawLog: string, ctx: NormalizeContext): unknow
.map(record => record.data?.header)
}
/**
* The normalized string-valued system prompts carried by request headers in a
* session JSONL, in log order. Headers without a string prompt are omitted so
* callers can assert one prompt per header explicitly.
*
* @param rawLog The session `.jsonl` content to inspect.
* @param ctx The volatile values of the run that produced it.
* @returns The normalized system prompts, in header order.
*/
export function normalizedSystemPrompts(rawLog: string, ctx: NormalizeContext): string[] {
return normalizedHeaders(rawLog, ctx).flatMap((header) => {
if (header === null || typeof header !== 'object') return []
const system = (header as { system?: unknown }).system
return typeof system === 'string' ? [system] : []
})
}
/** One normalized system-prompt edit carried by a `request/header-delta`. */
export interface SystemPromptDeltaSnapshot {
/** How many leading lines remain from the prior prompt. */
keepStart: number
/** How many trailing lines remain from the prior prompt. */
keepEnd: number
/** The normalized replacement lines inserted between the retained ranges. */
insert: string[]
}
/**
* Extract normalized system-prompt edits from request-header deltas in log
* order. Deltas without a well-formed system edit are omitted; their non-prompt
* structure remains pinned in JSONL.
*
* @param rawLog The session `.jsonl` content to inspect.
* @param ctx The volatile values of the run that produced it.
* @returns The normalized system-prompt edits, in event order.
*/
export function normalizedSystemPromptDeltas(rawLog: string, ctx: NormalizeContext): SystemPromptDeltaSnapshot[] {
return normalizeSessionLog(rawLog, ctx)
.split('\n')
.filter(line => line.trim().length > 0)
.map(line => JSON.parse(line) as { type?: unknown; data?: { system?: unknown } })
.filter(record => record.type === 'request/header-delta')
.flatMap((record) => {
const system = record.data?.system
if (system === null || typeof system !== 'object') return []
const { keepStart, keepEnd, insert } = system as { keepStart?: unknown; keepEnd?: unknown; insert?: unknown }
if (typeof keepStart !== 'number' || typeof keepEnd !== 'number' || !Array.isArray(insert)) return []
if (!insert.every(line => typeof line === 'string')) return []
return [{ keepStart, keepEnd, insert: insert }]
})
}
/**
* Render a normalized prompt as a repository-friendly Markdown snapshot.
* Prompt text is unchanged except that a missing terminal newline is added so
* the committed file follows the repository newline contract.
*
* @param prompt The normalized system prompt.
* @param deltas Normalized prompt edits to append as readable sections.
* @returns Markdown snapshot text ending in a newline.
*/
export function formatSystemPromptSnapshot(
prompt: string,
deltas: readonly SystemPromptDeltaSnapshot[] = [],
): string {
let snapshot = prompt.endsWith('\n') ? prompt : `${prompt}\n`
for (const [index, delta] of deltas.entries()) {
snapshot += `\n<!-- request/header-delta ${index + 1}: keepStart=${delta.keepStart}, keepEnd=${delta.keepEnd} -->\n\n`
const insert = delta.insert.join('\n')
snapshot += insert.endsWith('\n') ? insert : `${insert}\n`
}
return snapshot
}
/** Return the initial-prompt portion of a possibly delta-bearing snapshot. */
function initialSystemPromptSnapshot(snapshot: string): string {
const marker = snapshot.indexOf('\n<!-- request/header-delta ')
return marker < 0 ? snapshot : snapshot.slice(0, marker)
}
/**
* Count the `request/header-delta` events in a session JSONL.
*
@@ -298,7 +385,8 @@ export function stabilizeRefreshLog(fresh: string, existing: string, replacement
* Register the suite: one `describe` per scenario (the golden/log compares and
* the header-uniformity guard) plus the fixture guard block (no orphan
* scenario dirs, required files present, exactly one pin per header class,
* pinning fixtures well-formed, non-pinning fixtures header-scrubbed). Must
* pinning fixtures well-formed, every JSONL prompt-scrubbed, non-pinning
* fixtures fully header-scrubbed). Must
* run at vitest collection time — it calls `describe`/`it`. Throws
* immediately if any header class lacks a pinning scenario or carries two
* (the uniformity guard needs exactly one comparison anchor per class).
@@ -373,11 +461,12 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
// keyless replay run for every comparable log, including authored
// scenarios that live record deliberately skips. The primary goes to
// session.jsonl, each child to session.<n>.jsonl in harvest order. A
// non-pinning scenario's fixtures are written header-scrubbed, so a
// re-record/refresh can never smuggle the full prompt/schema content
// back into every fixture.
// Every fixture is written with its system prompt scrubbed. A pinning
// scenario keeps the remaining header content (notably tool schemas);
// every other scenario scrubs that bulk too. Record/refresh therefore
// cannot smuggle prompt text back into JSONL or duplicate schemas.
const scrub = scenario.pinsHeader === true
? (log: string): string => log
? scrubSystemPrompts
: scrubRequestHeaders
const fixtureFiles = ['session.jsonl', ...Array.from({ length: childSessions }, (_, i) => `session.${i + 1}.jsonl`)]
const existingFixtures = REFRESHING
@@ -400,6 +489,21 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
REFRESHING ? stabilizeRefreshLog(child, existingFixtures[i] as string, replacements) : child,
))
}
if (scenario.pinsHeader === true) {
const prompts = result.sessionLogs.flatMap(log => normalizedSystemPrompts(log.content, ctx))
expect(prompts.length, `${mode} produced no system prompt to snapshot`).toBeGreaterThan(0)
const initialSnapshot = formatSystemPromptSnapshot(prompts[0] as string)
for (const prompt of prompts) {
expect(formatSystemPromptSnapshot(prompt), 'the pinning run produced divergent system prompts')
.toEqual(initialSnapshot)
}
const primary = result.sessionLogs[0] as HarvestedLog
const snapshot = formatSystemPromptSnapshot(
prompts[0] as string,
normalizedSystemPromptDeltas(primary.content, ctx),
)
await writeFile(join(dir, SYSTEM_PROMPT_SNAPSHOT), snapshot)
}
}
const stdout = normalizeStdout(result.rawStdout, ctx)
@@ -415,12 +519,10 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
// 1:1. Each side passes through normalizeSessionLog, scrubbed against ITS
// OWN volatile values — the live run's via `ctx`, the committed fixture's
// via its own header (a committed file cannot share the live run's ids).
// Unless this scenario pins the header, both sides ALSO pass through
// scrubRequestHeaders: the live log carries the real prompt/schemas, the
// fixture carries the `{{system}}`/`{{tools}}` tokens, and the scrub is
// idempotent — so the compare checks the header's presence, position,
// reason, and config, but not its bulk content (pinned once, in the
// `pinsHeader` scenario).
// Both sides pass through the scenario's idempotent scrub: every live
// prompt becomes the fixture's `{{system}}`; non-pinning scenarios
// additionally tokenize tools/prefix. The dedicated header guard below
// compares those omitted values against their class's pin artifacts.
expect(result.sessionLogs.length, 'this scenario must persist a session log').toBe(childSessions + 1)
for (let i = 0; i < fixtureFiles.length; i++) {
const harvested = scrub((result.sessionLogs[i] as HarvestedLog).content)
@@ -430,34 +532,42 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
}
// Header-uniformity guard: a class's single pin is sound only while
// every session in that class composes the SAME header and keeps it
// for the whole run. Assert both halves live. (1) Every
// request/header the run produced (parent, spawn child, fork child,
// initial or resume) must equal the CLASS's pinned fixture's header
// after each side is normalized against its own volatile values.
// (2) No request/header-delta may appear at all — a mid-run header
// change diverges from the pin by construction, and its content
// would be invisible under the scrub. If either fails, either the
// header changed (update the pin: re-record or hand-edit the pinning
// scenario's fixture) or composition became session-dependent by
// design (give the divergent shape its own pinning scenario and
// class).
if (scenario.pinsHeader !== true) {
/* v8 ignore next -- construction guarantees the pin exists; a miss would fail the one-header assertion loudly. */
const pinningScenario = pinningByClass.get(classOf(scenario)) ?? scenario
const pinnedFixture = await readFile(join(snapshotsDir, pinningScenario.name, 'session.jsonl'), 'utf8')
const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture))
expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`)
.toBe(1)
for (const log of result.sessionLogs) {
expect(headerDeltaCount(log.content), `session ${log.id}: a request/header-delta in a non-pinning scenario`)
.toBe(0)
const headers = normalizedHeaders(log.content, ctx)
for (const [k, header] of headers.entries()) {
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
.toEqual(pinned[0])
}
// Header-uniformity guard: every live header in a class must equal the
// class pin split across its JSONL header (system token + real tools)
// and readable Markdown prompt. A pinning scenario may carry its
// declared header deltas; their prompt edits live in the Markdown
// golden while JSONL retains the tokenized edit structure.
/* v8 ignore next -- construction guarantees the pin exists; a miss would fail the one-header assertion loudly. */
const pinningScenario = pinningByClass.get(classOf(scenario)) ?? scenario
const pinningDir = join(snapshotsDir, pinningScenario.name)
const pinnedFixture = await readFile(join(pinningDir, 'session.jsonl'), 'utf8')
const pinned = normalizedHeaders(pinnedFixture, fixtureContext(pinnedFixture))
const promptSnapshot = await readFile(join(pinningDir, SYSTEM_PROMPT_SNAPSHOT), 'utf8')
const initialPromptSnapshot = initialSystemPromptSnapshot(promptSnapshot)
expect(pinned.length, `the pinning fixture (${pinningScenario.name}) must carry exactly one request/header`)
.toBe(1)
for (const [logIndex, log] of result.sessionLogs.entries()) {
const expectedDeltas = scenario.pinsHeader === true && logIndex === 0
? scenario.expectedHeaderDeltas ?? 0
: 0
expect(headerDeltaCount(log.content), `session ${log.id}: request/header-delta count`)
.toBe(expectedDeltas)
const headers = normalizedHeaders(scrubSystemPrompts(log.content), ctx)
const prompts = normalizedSystemPrompts(log.content, ctx)
expect(prompts.length, `session ${log.id}: every request/header must carry a string system prompt`)
.toBe(headers.length)
for (const [k, header] of headers.entries()) {
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
.toEqual(pinned[0])
expect(formatSystemPromptSnapshot(prompts[k] as string), `session ${log.id}: initial system prompt #${k + 1} diverged from ${pinningScenario.name}/${SYSTEM_PROMPT_SNAPSHOT}`)
.toEqual(initialPromptSnapshot)
}
if (scenario.pinsHeader === true && logIndex === 0) {
expect(formatSystemPromptSnapshot(
prompts[0] as string,
normalizedSystemPromptDeltas(log.content, ctx),
), `session ${log.id}: system-prompt deltas diverged from ${pinningScenario.name}/${SYSTEM_PROMPT_SNAPSHOT}`)
.toEqual(promptSnapshot)
}
}
})
@@ -488,13 +598,15 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
// `overridden` flag: required when set, forbidden when not — the harness
// forwards the file purely on existence, so an unregistered stray sidecar
// would silently replace the derived script.
for (const { name, overridden, childSessions } of scenarios) {
for (const { name, overridden, childSessions, pinsHeader } of scenarios) {
const dir = join(snapshotsDir, name)
expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true)
expect(existsSync(join(dir, 'stdout.golden.jsonl')), `${name}/stdout.golden.jsonl`).toBe(true)
expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true)
expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json presence must match \`overridden\``)
.toBe(overridden === true)
expect(existsSync(join(dir, SYSTEM_PROMPT_SNAPSHOT)), `${name}/${SYSTEM_PROMPT_SNAPSHOT} presence must match \`pinsHeader\``)
.toBe(pinsHeader === true)
// A nested-agent scenario ships one child fixture per recorded subagent
// session (`session.1.jsonl` …), the replay source for that child session.
for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) {
@@ -520,7 +632,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
})
it('every pinning fixture carries exactly one request/header and its declared deltas', async () => {
it('every pinning fixture carries one request/header, one readable prompt, and its declared deltas', async () => {
// The live uniformity guard runs only in NON-pinning scenarios, so a
// class made of just its pinning scenario would otherwise accept a
// re-recorded pin with several headers or an undeclared mid-run
@@ -530,20 +642,19 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
for (const scenario of pinningByClass.values()) {
const fixture = await readFile(join(snapshotsDir, scenario.name, 'session.jsonl'), 'utf8')
const headers = normalizedHeaders(fixture, fixtureContext(fixture))
const promptSnapshot = await readFile(join(snapshotsDir, scenario.name, SYSTEM_PROMPT_SNAPSHOT), 'utf8')
expect(headers.length, `${scenario.name}: a pinning fixture must carry exactly one request/header`).toBe(1)
expect(promptSnapshot.length, `${scenario.name}/${SYSTEM_PROMPT_SNAPSHOT} must not be empty`).toBeGreaterThan(0)
expect(promptSnapshot.endsWith('\n'), `${scenario.name}/${SYSTEM_PROMPT_SNAPSHOT} must end in a newline`).toBe(true)
expect(headerDeltaCount(fixture), `${scenario.name}: a pinning fixture must carry exactly its declared request/header-deltas`)
.toBe(scenario.expectedHeaderDeltas ?? 0)
}
})
it('committed fixtures carry request-header content ONLY in the pinning scenario', async () => {
// The whole point of the pin: a system-prompt or tool-schema change must
// churn exactly one committed line. A non-pinning fixture that carries the
// full header (a hand-recorded file, or a header line hand-edited out of
// its canonical JSON form) silently reopens the suite-wide churn, so fail
// loud here: every non-pinning session*.jsonl must be a fixed point of
// scrubRequestHeaders (apply the scrub to fix a violation), and the
// pinning scenario's fixtures must NOT be (their content IS the pin).
it('every committed JSONL omits system prompts and only pinning fixtures keep other header bulk', async () => {
// System prompts always live in the readable Markdown artifact. Header
// pins keep tool schemas/prefixes in JSONL; every other fixture tokenizes
// all header bulk. Fixed-point checks make both storage rules fail loud.
for (const scenario of scenarios) {
const dir = join(snapshotsDir, scenario.name)
const files = [
@@ -552,8 +663,10 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
]
for (const file of files) {
const fixture = await readFile(join(dir, file), 'utf8')
expect(scrubSystemPrompts(fixture), `${scenario.name}/${file} carries an unscrubbed system prompt`)
.toEqual(fixture)
if (scenario.pinsHeader === true) {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must PIN the full header content`)
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must pin the non-system header content`)
.not.toEqual(fixture)
} else {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} carries unscrubbed header content`)

View File

@@ -1,2 +1,2 @@
{"type":"session","id":"ccdc749f-56f3-4267-9750-598b5c60b7b2","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy"}
{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}

View File

@@ -0,0 +1 @@
SYS PROMPT

View File

@@ -5,7 +5,8 @@
"lines": [
{ "type": "session", "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}" },
{ "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } },
{ "type": "turn/start", "seq": 1, "time": 100, "data": { "turn": 1 } }
{ "type": "request/header-delta", "seq": 1, "time": 100, "data": { "system": { "keepStart": 1, "keepEnd": 0, "insert": ["NEW PROMPT LINE"] } } },
{ "type": "turn/start", "seq": 2, "time": 100, "data": { "turn": 1 } }
]
}]
}

View File

@@ -1,3 +1,4 @@
{"type":"session","id":"12121212-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/pin-cwd"}
{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"SYS PROMPT","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"turn/start","seq":1,"time":7,"data":{"turn":1}}
{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"request/header-delta","seq":1,"time":7,"data":{"system":{"keepStart":1,"keepEnd":0,"insert":["{{system}}"]}}}
{"type":"turn/start","seq":2,"time":7,"data":{"turn":1}}

View File

@@ -0,0 +1,5 @@
SYS PROMPT
<!-- request/header-delta 1: keepStart=1, keepEnd=0 -->
NEW PROMPT LINE

View File

@@ -1,5 +1,11 @@
import { describe, expect, it } from 'vitest'
import { type NormalizeContext, normalizeSessionLog, normalizeStdout, scrubRequestHeaders } from '../src/normalize.ts'
import {
type NormalizeContext,
normalizeSessionLog,
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
} from '../src/normalize.ts'
/**
* Unit tests for the pure snapshot normalizers. Live as a *.spec.ts (runs in
@@ -260,3 +266,43 @@ describe('scrubRequestHeaders', () => {
expect(scrubRequestHeaders(once)).toBe(once)
})
})
describe('scrubSystemPrompts', () => {
it('scrubs only system prompt payloads while keeping tools and prefixes verbatim', () => {
const header = JSON.stringify({
type: 'request/header', seq: 1, time: 2,
data: {
header: {
system: 'full prompt',
tools: [{ name: 'read', description: 'full schema' }],
messagePrefix: [{ role: 'user', content: [{ type: 'text', text: 'full prefix' }] }],
},
reason: 'initial',
},
})
const delta = JSON.stringify({
type: 'request/header-delta', seq: 2, time: 3,
data: {
system: { keepStart: 1, keepEnd: 2, insert: ['new prompt line'] },
tools: { changed: [{ name: 'read', description: 'changed schema' }] },
messagePrefix: [{ role: 'user', content: [{ type: 'text', text: 'changed prefix' }] }],
},
})
const toolsOnly = JSON.stringify({
type: 'request/header', seq: 3, time: 4,
data: { header: { tools: [{ name: 'read', description: 'schema only' }] }, reason: 'resume' },
})
const out = scrubSystemPrompts(`${header}\n${delta}\n${toolsOnly}\n`)
expect(out).toContain('"system":"{{system}}"')
expect(out).toContain('"insert":["{{system}}"]')
expect(out).not.toContain('full prompt')
expect(out).not.toContain('new prompt line')
expect(out).toContain('full schema')
expect(out).toContain('full prefix')
expect(out).toContain('changed schema')
expect(out).toContain('changed prefix')
expect(out.split('\n')[2]).toBe(toolsOnly)
expect(scrubSystemPrompts(out)).toBe(out)
})
})

View File

@@ -8,8 +8,11 @@ import { defineAcpSnapshotSuite, type HarvestedLog, type Scenario } from '../src
import {
childFixturePaths,
fixtureContext,
formatSystemPromptSnapshot,
headerDeltaCount,
normalizedHeaders,
normalizedSystemPromptDeltas,
normalizedSystemPrompts,
refreshFixtureReplacements,
stabilizeRefreshLog,
} from '../src/suite.ts'
@@ -48,7 +51,7 @@ const RECORD_SRC = fileURLToPath(new URL('./fixtures/record-suite', import.meta.
// is what this suite can exercise; the real overlay boot is the acp-agent
// example's code-mode scenarios).
const REPLAY_SCENARIOS: Scenario[] = [
{ name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'main' },
{ name: 'pin-turn', hasModelTurn: true, recorded: true, pinsHeader: true, expectedHeaderDeltas: 1, headerClass: 'main' },
{ name: 'plain-turn', hasModelTurn: true, recorded: true, childSessions: 1, headerClass: 'main', configPath: AGENT.configPath },
{ name: 'no-model', hasModelTurn: false, recorded: false, headerClass: 'main' },
{ name: 'blocked-log', hasModelTurn: false, comparesLog: true, recorded: false, headerClass: 'main' },
@@ -78,6 +81,7 @@ afterAll(async () => {
function staleRefreshFixtures(dir: string): void {
writeFileSync(join(dir, 'plain-turn', 'stdout.golden.jsonl'), 'stale stdout\n')
writeFileSync(join(dir, 'pin-turn', 'system-prompt.golden.md'), 'STALE PROMPT\n')
const plainBehaviorFile = join(dir, 'plain-turn', 'behavior.json')
const plainBehavior = JSON.parse(readFileSync(plainBehaviorFile, 'utf8')) as Record<string, unknown>
@@ -124,6 +128,15 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => {
const authored = readFileSync(join(refreshDir, 'authored-error', 'session.jsonl'), 'utf8')
expect(authored).toContain('"error":"model exploded"')
expect(authored).not.toContain('"error":"stale"')
expect(readFileSync(join(refreshDir, 'pin-turn', 'system-prompt.golden.md'), 'utf8')).toBe([
'SYS PROMPT',
'',
'<!-- request/header-delta 1: keepStart=1, keepEnd=0 -->',
'',
'NEW PROMPT LINE',
'',
].join('\n'))
})
})
@@ -219,6 +232,55 @@ describe('normalizedHeaders', () => {
})
})
describe('normalizedSystemPrompts', () => {
it('extracts normalized string prompts and omits absent or non-string fields', () => {
const log = [
'{"type":"session","id":"a","createdAt":5,"cwd":"/w"}',
'{"type":"request/header","seq":0,"time":9,"data":{"header":{"system":"work in /w"}}}',
'{"type":"request/header","seq":1,"time":9,"data":{"header":{}}}',
'{"type":"request/header","seq":2,"time":9,"data":{"header":{"system":null}}}',
'{"type":"request/header","seq":3,"time":9,"data":{"header":null}}',
'{"type":"request/header","seq":4,"time":9,"data":{"header":"invalid"}}',
'',
].join('\n')
expect(normalizedSystemPrompts(log, { sessionIds: [], cwd: '/w' })).toEqual(['work in {{cwd}}'])
})
})
describe('normalizedSystemPromptDeltas', () => {
it('extracts and normalizes well-formed system edits', () => {
const log = [
'{"type":"request/header-delta","data":{"system":{"keepStart":1,"keepEnd":0,"insert":["work in /w"]}}}',
'{"type":"request/header-delta","data":{"tools":{"replace":[]}}}',
'{"type":"request/header-delta","data":{"system":{"keepStart":"1","keepEnd":0,"insert":[]}}}',
'{"type":"request/header-delta","data":{"system":{"keepStart":1,"keepEnd":0,"insert":[null]}}}',
'',
].join('\n')
expect(normalizedSystemPromptDeltas(log, { sessionIds: [], cwd: '/w' })).toEqual([
{ keepStart: 1, keepEnd: 0, insert: ['work in {{cwd}}'] },
])
})
})
describe('formatSystemPromptSnapshot', () => {
it('adds a missing terminal newline without changing an existing one', () => {
expect(formatSystemPromptSnapshot('prompt')).toBe('prompt\n')
expect(formatSystemPromptSnapshot('prompt\n')).toBe('prompt\n')
})
it('renders readable system-prompt delta sections', () => {
expect(formatSystemPromptSnapshot('prompt', [
{ keepStart: 1, keepEnd: 0, insert: ['new', 'lines'] },
])).toBe('prompt\n\n<!-- request/header-delta 1: keepStart=1, keepEnd=0 -->\n\nnew\nlines\n')
})
it('does not double the newline of a delta insert with a trailing blank line', () => {
expect(formatSystemPromptSnapshot('prompt\n', [
{ keepStart: 2, keepEnd: 1, insert: ['tail', ''] },
])).toBe('prompt\n\n<!-- request/header-delta 1: keepStart=2, keepEnd=1 -->\n\ntail\n')
})
})
describe('headerDeltaCount', () => {
it('counts request/header-delta events, ignoring blanks and other lines', () => {
const delta = JSON.stringify({ type: 'request/header-delta', seq: 2, time: 9, data: {} })

View File

@@ -23,7 +23,11 @@ What the seam guarantees regardless, because benign scripts hit these constantly
`start()` shape-validates the meta DATA host-side and parse-checks the body with the identical wrapper the worker compiles (`new vm.Script`, discarded), preserving the seam's synchronous `META_INVALID`/`SCRIPT_PARSE` throws; one redundant parse per run is the deliberate price. It then spawns the worker (`src/worker.ts` unbuilt via an explicit tsx `execArgv`; the sibling `lib/worker.js` bundle when built) with the meta, body, `args`, and worker-side limits as `workerData`.
Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**: `agent()` sends `child-start` and the host starts the child on `ctx.subagents` (parent attribution, the shared per-run abort signal, `outputSchema`/`model` pass-through), replying with the child id, its settlement (a JSON projection; an infrastructure REJECTION crosses as `child-failed` and stays the fatal `AGENT_RESULT`), and dispose acks. Observer narration (`phase`/`log`/`agent-start`/`agent-end`) crosses as messages and re-emits as the seam's `workflow/*` events. A **ready→go handshake** gates the body: a cancellation racing worker boot arrives before `go`, so a run cancelled before start never executes the body at all.
Inside the worker, `runWorkerSession` builds the execution core (hooks, combinators, concurrency semaphore, caps, fatal-error discipline) over a **child port**. `agent()` sends `child-start`, and the host starts the child on `ctx.subagents` with parent attribution, the shared per-run abort signal, and `outputSchema`/`model` pass-through.
The host observes `run.result` immediately but buffers its snapshotted wire projection until `run.started` fulfills. It then replies `child-started` with the child id before forwarding settlement, so `workflow/agent-start` always names a ready child and precedes its end. A readiness rejection replies `child-start-error`, emits no workflow agent pair, and makes the host dispose the attempt because the worker never received a handle; the worker classifies it as fatal `AGENT_START` unless cancellation already owns the run. If readiness fulfills, an infrastructure result rejection crosses as `child-failed`/`AGENT_RESULT` regardless of whether that rejection settled before or after readiness. Child disposal acknowledgements complete the RPC.
Observer narration (`phase`/`log`/`agent-start`/`agent-end`) crosses as messages and re-emits as the seam's `workflow/*` events. A **ready→go handshake** gates the body: a cancellation racing worker boot arrives before `go`, so a run cancelled before start never executes the body at all.
## The value boundary

View File

@@ -14,12 +14,16 @@
* (a script that never settles is force-settled `cancelled` and its worker
* terminated — the real kill an in-process engine could not perform).
*
* Children live in a host-side registry (callId → run): the worker drives
* their disposal by RPC on the graceful path, `dispose()` host-drives every
* registered child's disposal immediately (a wedged worker can relay no
* dispose RPC, and child teardown must overlap the grace, not start after
* it), and the registry is what lets the host abort and dispose every
* survivor when the worker dies or is terminated mid-flight. The three
* Children live in a host-side registry (callId → run) as soon as the provider
* accepts them, so cancellation reaches even a pre-publication attempt. The
* host observes `result` immediately but acknowledges the child to the worker
* only after `started` fulfills; readiness failure is a start error and the
* host disposes the attempt because the worker never received a handle. The
* worker drives disposal by RPC on the graceful path, `dispose()` host-drives
* every registered child's disposal immediately (a wedged worker can relay no
* dispose RPC, and child teardown must overlap the grace, not start after it),
* and the registry lets the host abort and dispose every survivor when the
* worker dies or is terminated mid-flight. The three
* paths share ONE disposal per child (memoized by callId; the seam's
* dispose() is idempotent anyway, the memo keeps the bookkeeping and the
* containment warn single). Lifecycle pairing is host-guaranteed the same
@@ -46,7 +50,7 @@ import { renderThrown } from './realm.ts'
import type { ExecutionObserver } from './runtime.ts'
import { HostToWorkerType, WorkerToHostType } from './protocol.ts'
import type { HostToWorkerPayloads, WorkerToHostMessage } from './protocol.ts'
import type { ChildStartRequest, WorkerInit } from './types.ts'
import type { ChildResult, ChildStartRequest, WorkerInit } from './types.ts'
/**
* Resolve the worker entry and spawn options for the current runtime shape.
@@ -307,19 +311,49 @@ export class WorkerRun implements WorkflowRun {
return
}
this.children.set(callId, run)
this.post(HostToWorkerType.ChildStarted, { callId, childId: run.id })
run.result.then(
const childId = run.id
// Observe settlement IMMEDIATELY, before readiness. A provider may reject
// result and started in the same turn; delaying this handler would make the
// result transiently unhandled. Buffer a forwarding closure so the worker
// still sees ChildStarted before ChildSettled/ChildFailed. Snapshot a
// resolved result now: a provider mutating its resolved object while
// publication is pending must not change what crosses the worker boundary.
const forwardResult = run.result.then<() => void, () => void>(
(result) => {
this.post(HostToWorkerType.ChildSettled, {
callId,
result: {
try {
const snapshot: ChildResult = structuredClone({
output: result.output,
...result.structured !== undefined ? { structured: result.structured } : {},
stopReason: result.stopReason,
},
})
})
return () => { this.post(HostToWorkerType.ChildSettled, { callId, result: snapshot }) }
} catch (error: unknown) {
const rendered = `workflow child result could not cross the worker boundary: ${renderThrown(error)}`
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
}
},
(error: unknown) => {
const rendered = renderThrown(error)
return () => { this.post(HostToWorkerType.ChildFailed, { callId, rendered }) }
},
)
// The provider owns the publication boundary. Only acknowledge the child
// after it is real, then flush any result that settled unusually early. A
// readiness rejection is a START failure, not AGENT_RESULT: the worker
// never receives a handle, so the host must also dispose the registered
// attempt. A concurrent host disposal may already have removed it; the
// identity guard preserves the one-disposal memo in that race.
void run.started.then(
() => {
this.post(HostToWorkerType.ChildStarted, { callId, childId })
void forwardResult.then((forward) => { forward() })
},
(error: unknown) => {
this.post(HostToWorkerType.ChildStartError, { callId, rendered: renderThrown(error) })
if (this.children.get(callId) === run) void this.disposeChild(callId, run)
},
(error: unknown) => { this.post(HostToWorkerType.ChildFailed, { callId, rendered: renderThrown(error) }) },
)
}
@@ -350,7 +384,10 @@ export class WorkerRun implements WorkflowRun {
private disposeChild(callId: number, run: SubagentRun): Promise<void> {
let disposal = this.childDisposals.get(callId)
if (disposal === undefined) {
disposal = run.dispose().then(
// The seam promises a Promise, but invoke inside an async boundary so a
// contract-violating synchronous throw is contained exactly like a
// rejected disposal and cannot break host quiescence.
disposal = (async () => { await run.dispose() })().then(
() => { this.finishChild(callId) },
(error: unknown) => {
this.ctx.logger.warn(`workflow-workerthread: child dispose failed: ${renderThrown(error)}`)

View File

@@ -69,9 +69,9 @@ export enum HostToWorkerType {
Go = 'go',
/** Cancel the run: hooks start throwing and the script dies at its next await. */
Cancel = 'cancel',
/** Child RPC reply: the start succeeded (exactly one of ChildStarted/ChildStartError per ChildStart). */
/** Child RPC reply: provider publication/readiness fulfilled (exactly one start reply per ChildStart). */
ChildStarted = 'child-started',
/** Child RPC reply: the start was refused or threw. */
/** Child RPC reply: synchronous start or asynchronous publication/readiness failed. */
ChildStartError = 'child-start-error',
/** Child RPC: a started child's result RESOLVED (its JSON projection). */
ChildSettled = 'child-settled',

View File

@@ -19,8 +19,9 @@
* (a benign-bug guard; the postMessage clone already isolated the caller).
*
* Failure discipline: fatal {@link WorkflowError}s (bad hook arguments,
* unsupported options/schemas, tripped caps, host start refusals and child
* result rejections, cancellation) ALWAYS propagate through
* unsupported options/schemas, tripped caps, synchronous start refusal,
* pre-publication readiness failure, ready-child result rejection, and
* cancellation) ALWAYS propagate through
* `parallel`/`pipeline` — recognized by `instanceof` against this realm's
* class, which a script inside the vm context cannot forge — and the per-item
* `null` is reserved for child-run failures and ordinary in-stage script

View File

@@ -89,24 +89,26 @@ class ChildRpcBridge implements ChildPort {
settled: Promise.withResolvers<ChildResult>(),
disposed: Promise.withResolvers<void>(),
}
// Containment: when the start is refused (or the run torn down) the
// settled promise may never gain a consumer — it must not surface as an
// unhandled rejection and kill the worker.
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after a refused start */ })
// Containment: when synchronous start or asynchronous readiness fails (or
// the run is torn down), the settled promise may never gain a consumer —
// it must not surface as an unhandled rejection and kill the worker.
entry.settled.promise.catch(() => { /* consumed: unconsumed child settlement after failed start/readiness */ })
this.pending.set(callId, entry)
this.post(WorkerToHostType.ChildStart, { callId, request })
const childId = await entry.started.promise
return new RpcChildHandle(this.post, callId, entry, childId)
}
/** The host started the child; releases the `startAgent` await. */
/** The host established a ready child; releases the `startAgent` await. */
onChildStarted(callId: number, childId: string): void {
this.pending.get(callId)?.started.resolve(childId)
}
/** The host refused the start; `startAgent` rejects with the rendered cause. */
/** Synchronous start or asynchronous readiness failed; reject and retire the pending RPC. */
onChildStartError(callId: number, rendered: string): void {
this.pending.get(callId)?.started.reject(new Error(rendered))
const entry = this.pending.get(callId)
this.pending.delete(callId)
entry?.started.reject(new Error(rendered))
}
/** The child's terminal result arrived. */

View File

@@ -91,7 +91,8 @@ export interface ChildPort {
/**
* Start one child agent on the host (the `agent()` hook's start half).
* @param request - the prompt and validated options.
* @returns the child handle; rejects when the host refuses the start.
* @returns the ready child handle; rejects when synchronous start or the
* provider's asynchronous publication/readiness boundary fails.
*/
startAgent(request: ChildStartRequest): Promise<ChildHandle>
}

View File

@@ -48,7 +48,12 @@ describe('dsh-workflow-workerthread over the real in-process stack', () => {
toolCallResponse('c1', STRUCTURED_OUTPUT_TOOL, { verdict: 'real', confidence: 0.9 }),
])
const childIds: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => { childIds.push(agent.childId) })
ctx.on('workflow/agent-start', (_info, agent) => {
// The workflow bridge must honor SubagentRun.started: a start observer
// sees the real spawn child already published, never a reserved id.
expect(ctx.agents.get(agent.childId)).toBeDefined()
childIds.push(agent.childId)
})
const run = ctx.workflows.start({
meta: { name: 'integration', description: 'plain + structured children' },
script: `phase('Read')

View File

@@ -8,7 +8,7 @@ import SubagentService from '@deepseek-ai/dsh-subagent'
import type { SubagentCapabilities, SubagentProvider, SubagentResult, SubagentRun, SubagentStartRequest } from '@deepseek-ai/dsh-subagent'
import type { WorkflowMeta, WorkflowResult, WorkflowResultInfo, WorkflowRunInfo } from '@deepseek-ai/dsh-workflow'
import * as workerEngineModule from '../src/index.ts'
import WorkerWorkflowEngine, { type Config } from '../src/index.ts'
import WorkerWorkflowEngine, { HostToWorkerType, type Config } from '../src/index.ts'
/** A minimal parent stand-in: the engine only threads it through to the provider. */
function fakeParent(): Agent {
@@ -47,7 +47,12 @@ const ESCAPE = "globalThis.constructor.constructor('return process')()"
/** One controllable child run: the test (or auto mode) settles it. */
interface ControlledRun {
request: SubagentStartRequest
/** Fulfill the provider publication/readiness boundary. */
publish(): void
/** Reject the provider publication/readiness boundary. */
rejectStart(error: unknown): void
settle(result: SubagentResult): void
rejectResult(error: unknown): void
cancelled: string | undefined
disposed: boolean
disposeCalls: number
@@ -68,26 +73,37 @@ class StubProvider implements SubagentProvider {
readonly name: string,
private readonly reply?: (request: SubagentStartRequest, index: number) => SubagentResult,
private readonly disposeDelayMs = 0,
private readonly deferStart = false,
) {}
start(request: SubagentStartRequest): SubagentRun {
let settle!: (result: SubagentResult) => void
const result = new Promise<SubagentResult>((resolve) => { settle = resolve })
const controlled: ControlledRun = { request, settle, cancelled: undefined, disposed: false, disposeCalls: 0 }
const readiness = Promise.withResolvers<undefined>()
const terminal = Promise.withResolvers<SubagentResult>()
const controlled: ControlledRun = {
request,
publish: () => { readiness.resolve(undefined) },
rejectStart: (error) => { readiness.reject(error) },
settle: (result) => { terminal.resolve(result) },
rejectResult: (error) => { terminal.reject(error) },
cancelled: undefined,
disposed: false,
disposeCalls: 0,
}
this.runs.push(controlled)
const index = this.runs.length - 1
request.signal?.addEventListener('abort', () => { settle({ output: [], stopReason: 'aborted' }) }, { once: true })
request.signal?.addEventListener('abort', () => { terminal.resolve({ output: [], stopReason: 'aborted' }) }, { once: true })
if (!this.deferStart) readiness.resolve(undefined)
if (this.reply) {
const reply = this.reply
queueMicrotask(() => { settle(reply(request, index)) })
queueMicrotask(() => { terminal.resolve(reply(request, index)) })
}
return {
id: AgentId(`stub-child-${index}`),
started: Promise.resolve(),
result,
started: readiness.promise,
result: terminal.promise,
cancel: (reason?: string) => {
controlled.cancelled = reason ?? 'cancelled'
settle({ output: [], stopReason: 'aborted' })
terminal.resolve({ output: [], stopReason: 'aborted' })
},
dispose: () => {
controlled.disposeCalls += 1
@@ -116,6 +132,7 @@ interface SetupOptions {
reply?: (request: SubagentStartRequest, index: number) => SubagentResult
manual?: boolean
disposeDelayMs?: number
deferStart?: boolean
}
async function setup(options?: SetupOptions) {
@@ -125,6 +142,7 @@ async function setup(options?: SetupOptions) {
'stub',
options?.manual ? undefined : options?.reply ?? (() => text('stub reply')),
options?.disposeDelayMs ?? 0,
options?.deferStart ?? false,
)
ctx.subagents.registerProvider(provider)
// A fixed concurrency ceiling: the auto-resolved default is machine-derived
@@ -214,6 +232,116 @@ describe('dsh-workflow-workerthread', () => {
expect(result.error).toContain('agent() could not start a child')
})
it('waits for child readiness before announcing it and snapshots a result that settled early', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const order: string[] = []
ctx.on('workflow/agent-start', (_info, agent) => { order.push(`start:${agent.seq}`) })
ctx.on('workflow/agent-end', (_info, agent) => { order.push(`end:${agent.outcome}`) })
ctx.on('workflow/end', () => { order.push('run-end') })
const handle = ctx.workflows.start({ ...scripted("return await agent('p')"), parent })
await waitFor(() => { expect(provider.runs.length).toBe(1) })
const early = text('accepted value')
provider.runs[0]!.settle(early)
// Let the host observe + snapshot result while readiness remains pending.
await new Promise(resolve => setTimeout(resolve, 0))
const earlyText = early.output[0] as { type: 'text'; text: string }
earlyText.text = 'mutated after settlement'
expect(order).toEqual([])
provider.runs[0]!.publish()
const result = await handle.result
expect(result.value).toBe('accepted value')
expect(order).toEqual(['start:1', 'end:completed', 'run-end'])
await handle.dispose()
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('observes an early result rejection but sends ChildStarted before ChildFailed after readiness', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', (_info, agent) => { lifecycle.push(`end:${agent.outcome}`) })
const handle = ctx.workflows.start({
...scripted("try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }"),
parent,
})
const worker = (handle as unknown as { worker: { postMessage(message: unknown): void } }).worker
const post = vi.spyOn(worker, 'postMessage')
const childMessageTypes = (): HostToWorkerType[] => post.mock.calls
.map(([message]) => (message as { type: HostToWorkerType }).type)
.filter(type => type === HostToWorkerType.ChildStarted || type === HostToWorkerType.ChildFailed)
await waitFor(() => { expect(provider.runs.length).toBe(1) })
provider.runs[0]!.rejectResult(new Error('backend failed before publication'))
await new Promise(resolve => setTimeout(resolve, 0))
expect(childMessageTypes()).toEqual([])
expect(lifecycle).toEqual([])
provider.runs[0]!.publish()
const result = await handle.result
expect(result.value).toMatchObject({ code: 'AGENT_RESULT' })
expect((result.value as { message: string }).message).toContain('backend failed before publication')
expect(childMessageTypes()).toEqual([HostToWorkerType.ChildStarted, HostToWorkerType.ChildFailed])
expect(lifecycle).toEqual(['start', 'end:failed'])
post.mockRestore()
await handle.dispose()
})
it('classifies readiness rejection as AGENT_START, drops an early result, and emits no false lifecycle pair', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', () => { lifecycle.push('end') })
const handle = ctx.workflows.start({
...scripted("try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }"),
parent,
})
await waitFor(() => { expect(provider.runs.length).toBe(1) })
// ACP-style failure can settle result(error) before its session/publication
// boundary rejects. Readiness must dominate that buffered child outcome.
provider.runs[0]!.settle({ output: [], stopReason: 'error' })
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('publication rolled back'))
const result = await handle.result
expect(result.value).toMatchObject({ code: 'AGENT_START' })
expect((result.value as { message: string }).message).toContain('publication rolled back')
expect(lifecycle).toEqual([])
await waitFor(() => {
expect(provider.runs[0]!.disposed).toBe(true)
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
await handle.dispose()
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('cancels and disposes a readiness-pending child once without publishing workflow lifecycle', async () => {
const { ctx, parent, provider } = await setup({ manual: true, deferStart: true, config: { disposeGraceMs: 500 } })
const lifecycle: string[] = []
ctx.on('workflow/agent-start', () => { lifecycle.push('start') })
ctx.on('workflow/agent-end', () => { lifecycle.push('end') })
const handle = ctx.workflows.start({ ...scripted("return await agent('pending')"), parent })
await waitFor(() => { expect(provider.runs.length).toBe(1) })
const disposal = handle.dispose()
await waitFor(() => {
expect(provider.runs[0]!.cancelled).toBe('workflow disposed')
expect(provider.runs[0]!.disposed).toBe(true)
})
// Ensure the host-driven disposal removed the registry entry before the
// late readiness rejection; its callback must not invoke dispose again.
await new Promise(resolve => setTimeout(resolve, 0))
provider.runs[0]!.rejectStart(new Error('cancelled before publication'))
const result = await handle.result
await disposal
expect(result.stopReason).toBe('cancelled')
expect(lifecycle).toEqual([])
expect(provider.runs[0]!.disposeCalls).toBe(1)
})
it('a child result REJECTION crosses back as a fatal AGENT_RESULT error (a broken provider is not a failed child)', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
@@ -238,7 +366,18 @@ describe('dsh-workflow-workerthread', () => {
expect((result.value as { message: string }).message).toContain('backend exploded')
})
it('a child whose dispose() rejects cannot wedge the script (the host acks anyway)', async () => {
it('maps an uncloneable ready-child result to fatal AGENT_RESULT instead of wedging the bridge', async () => {
const { ctx, parent } = await setup({
reply: () => ({ output: [], structured: () => { /* deliberately not cloneable */ }, stopReason: 'completed' }),
})
const result = await run(ctx, parent, scripted(`
try { await agent('p'); return 'unreachable' } catch (e) { return { code: e.code, message: e.message } }
`))
expect(result.value).toMatchObject({ code: 'AGENT_RESULT' })
expect((result.value as { message: string }).message).toContain('could not cross the worker boundary')
})
it('a child whose dispose() throws synchronously cannot wedge the script (the host acks anyway)', async () => {
const ctx = new Context()
await ctx.plugin(SubagentService)
const provider: SubagentProvider = {
@@ -250,7 +389,7 @@ describe('dsh-workflow-workerthread', () => {
started: Promise.resolve(),
result: Promise.resolve({ output: [{ type: 'text', text: 'fine' }], stopReason: 'completed' }),
cancel: () => { /* settled already */ },
dispose: () => Promise.reject(new Error('dispose exploded')),
dispose: () => { throw new Error('dispose exploded') },
}),
}
ctx.subagents.registerProvider(provider)

View File

@@ -22,7 +22,7 @@ All observe-only emits carrying DATA SNAPSHOTS (`WorkflowRunInfo` = id + meta)
- `workflow/start`(info) / `workflow/end`(info, resultInfo) — run lifecycle; `resultInfo` deliberately omits the value.
- `workflow/phase`(info, title) / `workflow/log`(info, message) — script narration.
- `workflow/agent-start`(info, agent) / `workflow/agent-end`(info, agent + outcome) — one pair per `agent()` call that STARTED a child run (a call rejected at validation or caps, refused at start, or cancelled while queued for a slot emits no pair), correlated by `seq`.
- `workflow/agent-start`(info, agent) / `workflow/agent-end`(info, agent + outcome) — ready-child lifecycle correlated by `seq`; the [generated event contract](../../../docs/cordis-catalog/events.md#workflowagent-start--emit) defines publication and pairing.
## Non-goals (this cut)

View File

@@ -76,8 +76,10 @@ declare module 'cordis' {
*/
'workflow/log'(info: WorkflowRunInfo, message: string): void
/**
* One `agent()` call started a child run. Paired with
* {@link Events['workflow/agent-end']} by `agent.seq`.
* One `agent()` call established a ready child run. Paired with
* {@link Events['workflow/agent-end']} by `agent.seq`. A call that never
* crosses the provider's publication/readiness boundary emits neither
* event in this pair.
* @param info - the run's identity snapshot.
* @param agent - the call's sequence number, label, phase, and child id.
* @mode emit
@@ -129,10 +131,12 @@ export type WorkflowEventName =
* - `UNSUPPORTED_SCHEMA` — an `agent()` schema outside the structured-output
* subset (see dsh-tools).
* - `AGENT_CAP` / `ITEM_CAP` — the run/agent caps tripped.
* - `AGENT_START` — the subagent seam refused to start a child.
* - `AGENT_RESULT` — a child's `result` REJECTED: an infrastructure fault at
* the subagent seam, distinct from a child that failed and resolved (which
* is the per-item `null`, never an error).
* - `AGENT_START` — synchronous subagent start or the provider's asynchronous
* publication/readiness boundary failed before cancellation took precedence.
* - `AGENT_RESULT` — a run whose readiness FULFILLED had its `result` REJECT: an
* infrastructure fault at the subagent seam, even if the rejection settled
* before readiness. This is distinct from a child that failed and resolved
* (which is the per-item `null`, never an error).
* - `RESULT_UNSERIALIZABLE` — a value crossing the script/host value boundary
* is not plain JSON data.
* - `CANCELLED` — the run was cancelled; pending and future hooks reject