feat(subagent): make final reports the continuable return contract

A continuable child could finish without giving its parent a usable
result: the report tool was described as optional and its default
delivery stayed quiet. Install the child-scoped report obligation (tool
description plus the tool:report prompt section), default reportDelivery
to wakeup, bind shipped subagent_fork delegations to one-shot so fork
keeps its prefix reuse, and extend the ACP snapshot harness to pin child
system prompts alongside child tool schemas.
This commit is contained in:
Hypatia May
2026-08-11 11:50:39 +08:00
parent 6d3cabf295
commit 76cf6cbd0b
79 changed files with 916 additions and 326 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md
README.md: 8a9669b1f555fceb7af6d017d273b62d7dcad1e1
README.zh.md: efa31df86d3b494ab14fc3742fc84adce570a593
README.md: 06f1cb67cfcd954254db480ea696d10d81b37438
README.zh.md: c4a5643f8c5d7e5b62a160cd32e5969187d34046

View File

@@ -55,6 +55,8 @@ A scenario booting a differently-composed tree sets its own `configPath` (an ove
A pin owns its generated `system-prompt.expected.md` or `tool-schemas.expected.json` by default; `systemPromptSource` and `toolSchemasSource` name another pin when the complete corresponding sequence is identical, so each distinct version is committed once. The pin's `session.jsonl` stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A pin with legitimate mid-run header changes declares `expectedHeaderChanges`; a shared source must declare the same count, and record/refresh rejects claimants that generate different bytes.
A child session whose own scope composes a different request declares it per fixture index: `pinsChildToolSchemas` moves that child's tool sequence into `tool-schemas.<n>.expected.json`, and `pinsChildSystemPrompts` moves its prompt into `system-prompt.<n>.expected.md`. Each names the `session.<n>.jsonl` fixture it describes, leaves every other request-header field to the class pin, and requires its sidecar to exist exactly when declared. A child prompt sidecar must also differ from its class pin, so a redundant copy fails instead of drifting. A continuable child carrying the scope-local `report` tool and its guidance section is the shipped case for both.
Every scenario compares `stdout.expected.jsonl` with cwd-rooted separators canonicalized to `/`. On Windows, `pinsNativeWindowsStdout` additionally compares the complete `stdout.expected.windows.jsonl` after the shared expected output and requires that sidecar exactly when enabled. A scenario requiring a non-Windows host declares `posixOnly`, which skips its run test on Windows while the fixture guards keep covering its committed files everywhere; examples include POSIX process semantics (e.g. cancelling a live bash call kills a detached process group) and generated paths Windows cannot represent. A scenario whose composition needs a usable `pwsh` declares `pwshOnly`; the caller-supplied `hasPwsh` probe (the shipped acp-agent suite follows the executor's own resolution, so Program Files installs count) skips the run test when no usable `pwsh` resolves while the fixture guards keep covering its committed files everywhere.
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md).

View File

@@ -55,6 +55,8 @@ defineAcpSnapshotSuite({
每个 pin 默认拥有其生成的 `system-prompt.expected.md` 或 `tool-schemas.expected.json`;当完整的对应序列相同时,`systemPromptSource` 和 `toolSchemasSource` 指定另一个 pin 作为来源,因此每个不同版本只提交一次。该 pin 的 `session.jsonl` 存储 `"system":"{{system}}","tools":"{{tools}}"`,同时保留配置、原因和任何模型可见前缀。具有合法运行中 header 变更的 pin 声明 `expectedHeaderChanges`;共享来源必须声明相同的 header 变更数量,录制/刷新会拒绝生成不同字节的共享引用方。
自身作用域组合出不同请求的 child 会话按 fixture 索引单独声明:`pinsChildToolSchemas` 把该 child 的工具序列移入 `tool-schemas.<n>.expected.json`,`pinsChildSystemPrompts` 把其提示词移入 `system-prompt.<n>.expected.md`。两者都指名自己描述的 `session.<n>.jsonl` fixture,其余请求 header 字段仍归类别 pin 所有,并要求 sidecar 恰好在声明时存在。child 提示词 sidecar 还必须与其类别 pin 不同,因此冗余副本会直接失败,而不会悄悄漂移。携带作用域局部 `report` 工具及其指引 section 的可继续 child 是两者的随附用例。
每个场景都比较 `stdout.expected.jsonl`,其中以 cwd 为根的分隔符规范化为 `/`。在 Windows 上,`pinsNativeWindowsStdout` 还会在共享预期输出之后比较完整 `stdout.expected.windows.jsonl`,并在启用时精确要求该 sidecar。需要非 Windows 主机的场景声明 `posixOnly`,在 Windows 上跳过运行测试,但 fixture 保护仍在所有平台覆盖其已提交文件;示例包括 POSIX 进程语义(例如取消实时 bash 调用会终止脱离进程组)和 Windows 无法表示的生成路径。组合需要可用 `pwsh` 的场景声明 `pwshOnly`;调用方提供的 `hasPwsh` 探测(随附的 acp-agent 套件遵循执行器自身的解析,因此 Program Files 安装也计入)在解析不到可用 `pwsh` 时跳过运行测试,而 fixture 保护仍处处覆盖其已提交文件。
示例还发布 `cordis.snapshot.yml` 回放 overlay,位于 `cordis.yml` 旁边(bin 在 `DSH_SNAPSHOT=replay` 下交换它们,见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md));回放 fixture 由 [`dsh-llm-replay`](../llm-replay/README.md) 提供,该包通过对子级设置的 `DSH_SNAPSHOT_*` env var 指向它。`pnpm run test:snapshot:record` 调用实时 LLM,并重写已记录场景的模型 fixture;`pnpm run test:snapshot:refresh` 保持无密钥,运行回放 overlay,并从已提交模型脚本重写 stdout、可比较会话日志预期输出,以及各 pin 自有的提示词与工具 schema sidecar。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md) 中。

View File

@@ -46,6 +46,11 @@ function childToolSchemasSnapshot(index: number): string {
return `tool-schemas.${index}.expected.json`
}
/** Return the dedicated system-prompt sidecar for one child fixture index. */
function childSystemPromptSnapshot(index: number): string {
return `system-prompt.${index}.expected.md`
}
/** The optional full Windows-native stdout transcript. */
const WINDOWS_STDOUT_SNAPSHOT = 'stdout.expected.windows.jsonl'
@@ -116,6 +121,13 @@ export interface Scenario {
* request-header field.
*/
pinsChildToolSchemas?: readonly number[]
/**
* Child fixture indices whose own system prompt is pinned separately, where
* `1` names `session.1.jsonl` and `system-prompt.1.expected.md`. A child
* scope that installs its own prompt section (the continuable `report`
* guidance) composes a prompt the class pin cannot describe.
*/
pinsChildSystemPrompts?: readonly number[]
/**
* How many changed `request/header` snapshots this PINNING scenario's primary
* fixture legitimately carries (default 0). Their full prompt text is kept in
@@ -491,6 +503,18 @@ export function formatSystemPromptSnapshot(
return snapshot
}
/**
* Reject a child prompt sidecar that cannot own distinct, canonical prompt text.
* @param sidecar - committed child prompt snapshot.
* @param classPin - initial prompt snapshot owned by the scenario's header class.
* @param label - repository-relative fixture label for diagnostics.
*/
export function assertChildSystemPromptSnapshot(sidecar: string, classPin: string, label: string): void {
if (sidecar.trim().length === 0) throw new Error(`${label} must pin a non-empty prompt`)
if (!sidecar.endsWith('\n')) throw new Error(`${label} must end in a newline`)
if (sidecar === classPin) throw new Error(`${label} must differ from its class pin`)
}
/** Return the initial-prompt portion of a possibly multi-header snapshot. */
function initialSystemPromptSnapshot(snapshot: string): string {
const marker = snapshot.indexOf('\n<!-- request/header change ')
@@ -1192,6 +1216,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
const childSchemaPins = new Set(scenario.pinsChildToolSchemas ?? [])
const childPromptPins = new Set(scenario.pinsChildSystemPrompts ?? [])
// Record writes live model fixtures; keyless refresh writes every comparable replayed
// fixture. Pinning JSONL keeps prefixes but moves prompts and schemas into sidecars.
@@ -1281,6 +1306,18 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
schemaSets.slice(1),
))
}
for (const index of childPromptPins) {
const log = result.sessionLogs[index]
expect(log, `${mode}: no child session log at index ${index} to snapshot a prompt from`)
.toBeDefined()
const prompts = normalizedSystemPrompts((log as HarvestedLog).content, ctx)
expect(prompts.length, `${mode}: child ${index} produced no system prompt to snapshot`)
.toBeGreaterThan(0)
await writeFile(
join(dir, childSystemPromptSnapshot(index)),
formatSystemPromptSnapshot(prompts[0] as string),
)
}
}
for (const expected of stdoutExpectedVariants(scenario)) {
@@ -1340,6 +1377,13 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
const parsed = parseToolSchemasSnapshot(sidecar)
childPinnedSchemas.set(index, [parsed.initial, ...parsed.changes])
}
const childPinnedPrompts = new Map<number, string>()
for (const index of childPromptPins) {
childPinnedPrompts.set(
index,
await readFile(join(dir, childSystemPromptSnapshot(index)), 'utf8'),
)
}
for (const [logIndex, log] of result.sessionLogs.entries()) {
const childSchemas = childPinnedSchemas.get(logIndex)
const expectedChanges = scenario.pinsHeader === true && logIndex === 0
@@ -1366,8 +1410,14 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
.toEqual(expected)
if (expectedChanges === 0) {
expect(formatSystemPromptSnapshot(prompts[k] as string), `session ${log.id}: initial system prompt #${k + 1} diverged from ${promptSource.name}/${SYSTEM_PROMPT_SNAPSHOT}`)
.toEqual(initialPromptSnapshot)
// A pinned child owns its whole prompt: its scope-local sections
// are exactly what the class pin cannot describe.
const childPrompt = childPinnedPrompts.get(logIndex)
const promptOrigin = childPrompt === undefined
? `${promptSource.name}/${SYSTEM_PROMPT_SNAPSHOT}`
: childSystemPromptSnapshot(logIndex)
expect(formatSystemPromptSnapshot(prompts[k] as string), `session ${log.id}: initial system prompt #${k + 1} diverged from ${promptOrigin}`)
.toEqual(childPrompt ?? initialPromptSnapshot)
}
}
if (scenario.pinsHeader === true && logIndex === 0) {
@@ -1400,16 +1450,19 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
it('every registered scenario has its required fixture files', async () => {
// Every scenario needs input, stdout, a primary session fixture, and matching optional sidecars.
for (const { name, overridden, pinsNativeWindowsStdout, pinsChildToolSchemas } of scenarios) {
for (const { name, overridden, pinsNativeWindowsStdout, pinsChildToolSchemas, pinsChildSystemPrompts } of scenarios) {
const dir = join(snapshotsDir, name)
const declaredChildPins = new Set(pinsChildToolSchemas ?? [])
const childSidecars = (await readdir(dir, { withFileTypes: true }))
const files = (await readdir(dir, { withFileTypes: true }))
.filter(entry => entry.isFile())
.map(entry => /^tool-schemas\.([1-9]\d*)\.expected\.json$/.exec(entry.name))
.map(entry => entry.name)
const childIndices = (pattern: RegExp): Set<number> => new Set(files
.map(file => pattern.exec(file))
.filter((match): match is RegExpExecArray => match !== null)
.map(match => Number(match[1]))
expect(new Set(childSidecars), `${name}: child tool-schema sidecars must match \`pinsChildToolSchemas\``)
.toEqual(declaredChildPins)
.map(match => Number(match[1])))
expect(childIndices(/^tool-schemas\.([1-9]\d*)\.expected\.json$/), `${name}: child tool-schema sidecars must match \`pinsChildToolSchemas\``)
.toEqual(new Set(pinsChildToolSchemas ?? []))
expect(childIndices(/^system-prompt\.([1-9]\d*)\.expected\.md$/), `${name}: child system-prompt sidecars must match \`pinsChildSystemPrompts\``)
.toEqual(new Set(pinsChildSystemPrompts ?? []))
expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true)
expect(existsSync(join(dir, 'stdout.expected.jsonl')), `${name}/stdout.expected.jsonl`).toBe(true)
expect(
@@ -1492,13 +1545,11 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
assertUniqueSnapshotContents('tool-schema', schemas)
})
it('every declared child tool-schema sidecar is canonical and names a real child', async () => {
it('every declared child sidecar is canonical and names a real child', async () => {
for (const scenario of scenarios) {
const pins = scenario.pinsChildToolSchemas ?? []
if (pins.length === 0) continue
const dir = join(snapshotsDir, scenario.name)
const files = await sessionFixtures(dir)
for (const index of pins) {
for (const index of scenario.pinsChildToolSchemas ?? []) {
expect(files[index], `${scenario.name}: child schema pin ${index} must name an existing session.<n>.jsonl fixture`)
.toBeDefined()
const file = childToolSchemasSnapshot(index)
@@ -1509,6 +1560,16 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
expect(parsed.initial.length, `${scenario.name}/${file} must pin at least one schema`)
.toBeGreaterThan(0)
}
for (const index of scenario.pinsChildSystemPrompts ?? []) {
expect(files[index], `${scenario.name}: child prompt pin ${index} must name an existing session.<n>.jsonl fixture`)
.toBeDefined()
const file = childSystemPromptSnapshot(index)
const sidecar = await readFile(join(dir, file), 'utf8')
/* v8 ignore next -- registration guarantees every scenario class has resolved sources. */
const promptSource = promptSourceByClass.get(classOf(scenario)) ?? scenario
const classPin = await readFile(join(snapshotsDir, promptSource.name, SYSTEM_PROMPT_SNAPSHOT), 'utf8')
assertChildSystemPromptSnapshot(sidecar, initialSystemPromptSnapshot(classPin), `${scenario.name}/${file}`)
}
}
})

View File

@@ -9,7 +9,7 @@
]},
{ "file": "b/child/session.jsonl", "lines": [
{ "type": "session", "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 },
{ "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } }
{ "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nCHILD GUIDANCE", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } }
]}
]
}

View File

@@ -0,0 +1,3 @@
SYS PROMPT
CHILD GUIDANCE

View File

@@ -12,6 +12,7 @@ import {
type Scenario,
} from '../src/index.ts'
import {
assertChildSystemPromptSnapshot,
assertUniqueSnapshotContents,
claimSharedSnapshot,
fixtureContext,
@@ -85,6 +86,7 @@ const REPLAY_SCENARIOS: Scenario[] = [
configPath: AGENT.configPath,
workspaceParent: tmpdir(),
pinsChildToolSchemas: [1],
pinsChildSystemPrompts: [1],
prepareWorkspace: (cwd) => {
writeFileSync(join(cwd, 'seed.txt'), 'prepared at runtime')
},
@@ -126,6 +128,7 @@ function staleRefreshFixtures(dir: string): void {
writeFileSync(join(dir, 'pin-turn', 'system-prompt.expected.md'), 'STALE PROMPT\n')
writeFileSync(join(dir, 'pin-turn', 'tool-schemas.expected.json'), '{"initial":[{"name":"stale"}],"changes":[]}\n')
writeFileSync(join(dir, 'plain-turn', 'tool-schemas.1.expected.json'), '{"initial":[{"name":"stale-child"}],"changes":[]}\n')
writeFileSync(join(dir, 'plain-turn', 'system-prompt.1.expected.md'), 'STALE CHILD PROMPT\n')
const plainBehaviorFile = join(dir, 'plain-turn', 'behavior.json')
const plainBehavior = JSON.parse(readFileSync(plainBehaviorFile, 'utf8')) as Record<string, unknown>
@@ -191,6 +194,8 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => {
const childSchemas = readFileSync(join(refreshDir, 'plain-turn', 'tool-schemas.1.expected.json'), 'utf8')
expect(childSchemas).toContain('"name": "child-only"')
expect(childSchemas).not.toContain('stale-child')
const childPrompt = readFileSync(join(refreshDir, 'plain-turn', 'system-prompt.1.expected.md'), 'utf8')
expect(childPrompt).toBe('SYS PROMPT\n\nCHILD GUIDANCE\n')
const pinSession = readFileSync(join(refreshDir, 'pin-turn', 'session.jsonl'), 'utf8')
expect(pinSession).toContain('"cwd":"{{cwd}}"')
@@ -591,6 +596,34 @@ describe('formatSystemPromptSnapshot', () => {
})
})
describe('assertChildSystemPromptSnapshot', () => {
const label = 'plain-turn/system-prompt.1.expected.md'
it('accepts a distinct non-empty canonical child prompt', () => {
expect(() => {
assertChildSystemPromptSnapshot('SYS PROMPT\n\nCHILD GUIDANCE\n', 'SYS PROMPT\n', label)
}).not.toThrow()
})
it('rejects an empty or non-canonical child prompt', () => {
expect(() => { assertChildSystemPromptSnapshot('\n', 'SYS PROMPT\n', label) }).toThrow(/non-empty prompt/)
expect(() => {
assertChildSystemPromptSnapshot('CHILD GUIDANCE', 'SYS PROMPT\n', label)
}).toThrow(/end in a newline/)
})
it('rejects a child prompt that duplicates its class pin', () => {
const classSnapshot = readFileSync(join(REPLAY_DIR, 'pin-turn', 'system-prompt.expected.md'), 'utf8')
const marker = classSnapshot.indexOf('\n<!-- request/header change ')
expect(marker).toBeGreaterThan(0)
const initialClassPin = classSnapshot.slice(0, marker)
expect(() => {
assertChildSystemPromptSnapshot(initialClassPin, initialClassPin, label)
}).toThrow(/must differ from its class pin/)
})
})
describe('headerChangeCount', () => {
it('counts changed request headers, ignoring anchors, blanks, and other lines', () => {
const change = JSON.stringify({ type: 'request/header', seq: 2, time: 9, data: { reason: 'change' } })