feat(subagent): make final reports the continuable return contract
A continuable child could finish without giving its parent a usable result: the report tool was described as optional and its default delivery stayed quiet. Install the child-scoped report obligation (tool description plus the tool:report prompt section), default reportDelivery to wakeup, bind shipped subagent_fork delegations to one-shot so fork keeps its prefix reuse, and extend the ACP snapshot harness to pin child system prompts alongside child tool schemas.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/support/acp-snapshot/README.md
|
||||
README.md: 8a9669b1f555fceb7af6d017d273b62d7dcad1e1
|
||||
README.zh.md: efa31df86d3b494ab14fc3742fc84adce570a593
|
||||
README.md: 06f1cb67cfcd954254db480ea696d10d81b37438
|
||||
README.zh.md: c4a5643f8c5d7e5b62a160cd32e5969187d34046
|
||||
|
||||
@@ -55,6 +55,8 @@ A scenario booting a differently-composed tree sets its own `configPath` (an ove
|
||||
|
||||
A pin owns its generated `system-prompt.expected.md` or `tool-schemas.expected.json` by default; `systemPromptSource` and `toolSchemasSource` name another pin when the complete corresponding sequence is identical, so each distinct version is committed once. The pin's `session.jsonl` stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A pin with legitimate mid-run header changes declares `expectedHeaderChanges`; a shared source must declare the same count, and record/refresh rejects claimants that generate different bytes.
|
||||
|
||||
A child session whose own scope composes a different request declares it per fixture index: `pinsChildToolSchemas` moves that child's tool sequence into `tool-schemas.<n>.expected.json`, and `pinsChildSystemPrompts` moves its prompt into `system-prompt.<n>.expected.md`. Each names the `session.<n>.jsonl` fixture it describes, leaves every other request-header field to the class pin, and requires its sidecar to exist exactly when declared. A child prompt sidecar must also differ from its class pin, so a redundant copy fails instead of drifting. A continuable child carrying the scope-local `report` tool and its guidance section is the shipped case for both.
|
||||
|
||||
Every scenario compares `stdout.expected.jsonl` with cwd-rooted separators canonicalized to `/`. On Windows, `pinsNativeWindowsStdout` additionally compares the complete `stdout.expected.windows.jsonl` after the shared expected output and requires that sidecar exactly when enabled. A scenario requiring a non-Windows host declares `posixOnly`, which skips its run test on Windows while the fixture guards keep covering its committed files everywhere; examples include POSIX process semantics (e.g. cancelling a live bash call kills a detached process group) and generated paths Windows cannot represent. A scenario whose composition needs a usable `pwsh` declares `pwshOnly`; the caller-supplied `hasPwsh` probe (the shipped acp-agent suite follows the executor's own resolution, so Program Files installs count) skips the run test when no usable `pwsh` resolves while the fixture guards keep covering its committed files everywhere.
|
||||
|
||||
The example also ships a `cordis.snapshot.yml` replay overlay next to its `cordis.yml` (the bin swaps them under `DSH_SNAPSHOT=replay` — [single-source replay config Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md)); replay fixtures are served by [`dsh-llm-replay`](../llm-replay/README.md), which this package points at via the `DSH_SNAPSHOT_*` env vars it sets on the child. `pnpm run test:snapshot:record` calls the live LLM and rewrites the recorded scenarios' model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from the committed model scripts. Fixture roles, record/replay/refresh semantics, and scenario-table fields are documented on `Scenario` and in the [snapshot Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md).
|
||||
|
||||
@@ -55,6 +55,8 @@ defineAcpSnapshotSuite({
|
||||
|
||||
每个 pin 默认拥有其生成的 `system-prompt.expected.md` 或 `tool-schemas.expected.json`;当完整的对应序列相同时,`systemPromptSource` 和 `toolSchemasSource` 指定另一个 pin 作为来源,因此每个不同版本只提交一次。该 pin 的 `session.jsonl` 存储 `"system":"{{system}}","tools":"{{tools}}"`,同时保留配置、原因和任何模型可见前缀。具有合法运行中 header 变更的 pin 声明 `expectedHeaderChanges`;共享来源必须声明相同的 header 变更数量,录制/刷新会拒绝生成不同字节的共享引用方。
|
||||
|
||||
自身作用域组合出不同请求的 child 会话按 fixture 索引单独声明:`pinsChildToolSchemas` 把该 child 的工具序列移入 `tool-schemas.<n>.expected.json`,`pinsChildSystemPrompts` 把其提示词移入 `system-prompt.<n>.expected.md`。两者都指名自己描述的 `session.<n>.jsonl` fixture,其余请求 header 字段仍归类别 pin 所有,并要求 sidecar 恰好在声明时存在。child 提示词 sidecar 还必须与其类别 pin 不同,因此冗余副本会直接失败,而不会悄悄漂移。携带作用域局部 `report` 工具及其指引 section 的可继续 child 是两者的随附用例。
|
||||
|
||||
每个场景都比较 `stdout.expected.jsonl`,其中以 cwd 为根的分隔符规范化为 `/`。在 Windows 上,`pinsNativeWindowsStdout` 还会在共享预期输出之后比较完整 `stdout.expected.windows.jsonl`,并在启用时精确要求该 sidecar。需要非 Windows 主机的场景声明 `posixOnly`,在 Windows 上跳过运行测试,但 fixture 保护仍在所有平台覆盖其已提交文件;示例包括 POSIX 进程语义(例如取消实时 bash 调用会终止脱离进程组)和 Windows 无法表示的生成路径。组合需要可用 `pwsh` 的场景声明 `pwshOnly`;调用方提供的 `hasPwsh` 探测(随附的 acp-agent 套件遵循执行器自身的解析,因此 Program Files 安装也计入)在解析不到可用 `pwsh` 时跳过运行测试,而 fixture 保护仍处处覆盖其已提交文件。
|
||||
|
||||
示例还发布 `cordis.snapshot.yml` 回放 overlay,位于 `cordis.yml` 旁边(bin 在 `DSH_SNAPSHOT=replay` 下交换它们,见[单源回放配置 Agent Note](../../../.agents/notes/archived/testing/2026-07-04-single-source-acp-replay-config.md));回放 fixture 由 [`dsh-llm-replay`](../llm-replay/README.md) 提供,该包通过对子级设置的 `DSH_SNAPSHOT_*` env var 指向它。`pnpm run test:snapshot:record` 调用实时 LLM,并重写已记录场景的模型 fixture;`pnpm run test:snapshot:refresh` 保持无密钥,运行回放 overlay,并从已提交模型脚本重写 stdout、可比较会话日志预期输出,以及各 pin 自有的提示词与工具 schema sidecar。Fixture 角色、录制/回放/刷新语义和场景表字段记录在 `Scenario` 以及[快照 Agent Note](../../../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md) 中。
|
||||
|
||||
@@ -46,6 +46,11 @@ function childToolSchemasSnapshot(index: number): string {
|
||||
return `tool-schemas.${index}.expected.json`
|
||||
}
|
||||
|
||||
/** Return the dedicated system-prompt sidecar for one child fixture index. */
|
||||
function childSystemPromptSnapshot(index: number): string {
|
||||
return `system-prompt.${index}.expected.md`
|
||||
}
|
||||
|
||||
/** The optional full Windows-native stdout transcript. */
|
||||
const WINDOWS_STDOUT_SNAPSHOT = 'stdout.expected.windows.jsonl'
|
||||
|
||||
@@ -116,6 +121,13 @@ export interface Scenario {
|
||||
* request-header field.
|
||||
*/
|
||||
pinsChildToolSchemas?: readonly number[]
|
||||
/**
|
||||
* Child fixture indices whose own system prompt is pinned separately, where
|
||||
* `1` names `session.1.jsonl` and `system-prompt.1.expected.md`. A child
|
||||
* scope that installs its own prompt section (the continuable `report`
|
||||
* guidance) composes a prompt the class pin cannot describe.
|
||||
*/
|
||||
pinsChildSystemPrompts?: readonly number[]
|
||||
/**
|
||||
* How many changed `request/header` snapshots this PINNING scenario's primary
|
||||
* fixture legitimately carries (default 0). Their full prompt text is kept in
|
||||
@@ -491,6 +503,18 @@ export function formatSystemPromptSnapshot(
|
||||
return snapshot
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject a child prompt sidecar that cannot own distinct, canonical prompt text.
|
||||
* @param sidecar - committed child prompt snapshot.
|
||||
* @param classPin - initial prompt snapshot owned by the scenario's header class.
|
||||
* @param label - repository-relative fixture label for diagnostics.
|
||||
*/
|
||||
export function assertChildSystemPromptSnapshot(sidecar: string, classPin: string, label: string): void {
|
||||
if (sidecar.trim().length === 0) throw new Error(`${label} must pin a non-empty prompt`)
|
||||
if (!sidecar.endsWith('\n')) throw new Error(`${label} must end in a newline`)
|
||||
if (sidecar === classPin) throw new Error(`${label} must differ from its class pin`)
|
||||
}
|
||||
|
||||
/** Return the initial-prompt portion of a possibly multi-header snapshot. */
|
||||
function initialSystemPromptSnapshot(snapshot: string): string {
|
||||
const marker = snapshot.indexOf('\n<!-- request/header change ')
|
||||
@@ -1192,6 +1216,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
}
|
||||
|
||||
const childSchemaPins = new Set(scenario.pinsChildToolSchemas ?? [])
|
||||
const childPromptPins = new Set(scenario.pinsChildSystemPrompts ?? [])
|
||||
|
||||
// Record writes live model fixtures; keyless refresh writes every comparable replayed
|
||||
// fixture. Pinning JSONL keeps prefixes but moves prompts and schemas into sidecars.
|
||||
@@ -1281,6 +1306,18 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
schemaSets.slice(1),
|
||||
))
|
||||
}
|
||||
for (const index of childPromptPins) {
|
||||
const log = result.sessionLogs[index]
|
||||
expect(log, `${mode}: no child session log at index ${index} to snapshot a prompt from`)
|
||||
.toBeDefined()
|
||||
const prompts = normalizedSystemPrompts((log as HarvestedLog).content, ctx)
|
||||
expect(prompts.length, `${mode}: child ${index} produced no system prompt to snapshot`)
|
||||
.toBeGreaterThan(0)
|
||||
await writeFile(
|
||||
join(dir, childSystemPromptSnapshot(index)),
|
||||
formatSystemPromptSnapshot(prompts[0] as string),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
for (const expected of stdoutExpectedVariants(scenario)) {
|
||||
@@ -1340,6 +1377,13 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
const parsed = parseToolSchemasSnapshot(sidecar)
|
||||
childPinnedSchemas.set(index, [parsed.initial, ...parsed.changes])
|
||||
}
|
||||
const childPinnedPrompts = new Map<number, string>()
|
||||
for (const index of childPromptPins) {
|
||||
childPinnedPrompts.set(
|
||||
index,
|
||||
await readFile(join(dir, childSystemPromptSnapshot(index)), 'utf8'),
|
||||
)
|
||||
}
|
||||
for (const [logIndex, log] of result.sessionLogs.entries()) {
|
||||
const childSchemas = childPinnedSchemas.get(logIndex)
|
||||
const expectedChanges = scenario.pinsHeader === true && logIndex === 0
|
||||
@@ -1366,8 +1410,14 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
|
||||
.toEqual(expected)
|
||||
if (expectedChanges === 0) {
|
||||
expect(formatSystemPromptSnapshot(prompts[k] as string), `session ${log.id}: initial system prompt #${k + 1} diverged from ${promptSource.name}/${SYSTEM_PROMPT_SNAPSHOT}`)
|
||||
.toEqual(initialPromptSnapshot)
|
||||
// A pinned child owns its whole prompt: its scope-local sections
|
||||
// are exactly what the class pin cannot describe.
|
||||
const childPrompt = childPinnedPrompts.get(logIndex)
|
||||
const promptOrigin = childPrompt === undefined
|
||||
? `${promptSource.name}/${SYSTEM_PROMPT_SNAPSHOT}`
|
||||
: childSystemPromptSnapshot(logIndex)
|
||||
expect(formatSystemPromptSnapshot(prompts[k] as string), `session ${log.id}: initial system prompt #${k + 1} diverged from ${promptOrigin}`)
|
||||
.toEqual(childPrompt ?? initialPromptSnapshot)
|
||||
}
|
||||
}
|
||||
if (scenario.pinsHeader === true && logIndex === 0) {
|
||||
@@ -1400,16 +1450,19 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
|
||||
it('every registered scenario has its required fixture files', async () => {
|
||||
// Every scenario needs input, stdout, a primary session fixture, and matching optional sidecars.
|
||||
for (const { name, overridden, pinsNativeWindowsStdout, pinsChildToolSchemas } of scenarios) {
|
||||
for (const { name, overridden, pinsNativeWindowsStdout, pinsChildToolSchemas, pinsChildSystemPrompts } of scenarios) {
|
||||
const dir = join(snapshotsDir, name)
|
||||
const declaredChildPins = new Set(pinsChildToolSchemas ?? [])
|
||||
const childSidecars = (await readdir(dir, { withFileTypes: true }))
|
||||
const files = (await readdir(dir, { withFileTypes: true }))
|
||||
.filter(entry => entry.isFile())
|
||||
.map(entry => /^tool-schemas\.([1-9]\d*)\.expected\.json$/.exec(entry.name))
|
||||
.map(entry => entry.name)
|
||||
const childIndices = (pattern: RegExp): Set<number> => new Set(files
|
||||
.map(file => pattern.exec(file))
|
||||
.filter((match): match is RegExpExecArray => match !== null)
|
||||
.map(match => Number(match[1]))
|
||||
expect(new Set(childSidecars), `${name}: child tool-schema sidecars must match \`pinsChildToolSchemas\``)
|
||||
.toEqual(declaredChildPins)
|
||||
.map(match => Number(match[1])))
|
||||
expect(childIndices(/^tool-schemas\.([1-9]\d*)\.expected\.json$/), `${name}: child tool-schema sidecars must match \`pinsChildToolSchemas\``)
|
||||
.toEqual(new Set(pinsChildToolSchemas ?? []))
|
||||
expect(childIndices(/^system-prompt\.([1-9]\d*)\.expected\.md$/), `${name}: child system-prompt sidecars must match \`pinsChildSystemPrompts\``)
|
||||
.toEqual(new Set(pinsChildSystemPrompts ?? []))
|
||||
expect(existsSync(join(dir, 'input.json')), `${name}/input.json`).toBe(true)
|
||||
expect(existsSync(join(dir, 'stdout.expected.jsonl')), `${name}/stdout.expected.jsonl`).toBe(true)
|
||||
expect(
|
||||
@@ -1492,13 +1545,11 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
assertUniqueSnapshotContents('tool-schema', schemas)
|
||||
})
|
||||
|
||||
it('every declared child tool-schema sidecar is canonical and names a real child', async () => {
|
||||
it('every declared child sidecar is canonical and names a real child', async () => {
|
||||
for (const scenario of scenarios) {
|
||||
const pins = scenario.pinsChildToolSchemas ?? []
|
||||
if (pins.length === 0) continue
|
||||
const dir = join(snapshotsDir, scenario.name)
|
||||
const files = await sessionFixtures(dir)
|
||||
for (const index of pins) {
|
||||
for (const index of scenario.pinsChildToolSchemas ?? []) {
|
||||
expect(files[index], `${scenario.name}: child schema pin ${index} must name an existing session.<n>.jsonl fixture`)
|
||||
.toBeDefined()
|
||||
const file = childToolSchemasSnapshot(index)
|
||||
@@ -1509,6 +1560,16 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
|
||||
expect(parsed.initial.length, `${scenario.name}/${file} must pin at least one schema`)
|
||||
.toBeGreaterThan(0)
|
||||
}
|
||||
for (const index of scenario.pinsChildSystemPrompts ?? []) {
|
||||
expect(files[index], `${scenario.name}: child prompt pin ${index} must name an existing session.<n>.jsonl fixture`)
|
||||
.toBeDefined()
|
||||
const file = childSystemPromptSnapshot(index)
|
||||
const sidecar = await readFile(join(dir, file), 'utf8')
|
||||
/* v8 ignore next -- registration guarantees every scenario class has resolved sources. */
|
||||
const promptSource = promptSourceByClass.get(classOf(scenario)) ?? scenario
|
||||
const classPin = await readFile(join(snapshotsDir, promptSource.name, SYSTEM_PROMPT_SNAPSHOT), 'utf8')
|
||||
assertChildSystemPromptSnapshot(sidecar, initialSystemPromptSnapshot(classPin), `${scenario.name}/${file}`)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
]},
|
||||
{ "file": "b/child/session.jsonl", "lines": [
|
||||
{ "type": "session", "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 },
|
||||
{ "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } }
|
||||
{ "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nCHILD GUIDANCE", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } }
|
||||
]}
|
||||
]
|
||||
}
|
||||
|
||||
3
packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/system-prompt.1.expected.md
vendored
Normal file
3
packages/support/acp-snapshot/tests/fixtures/suite/plain-turn/system-prompt.1.expected.md
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
SYS PROMPT
|
||||
|
||||
CHILD GUIDANCE
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
type Scenario,
|
||||
} from '../src/index.ts'
|
||||
import {
|
||||
assertChildSystemPromptSnapshot,
|
||||
assertUniqueSnapshotContents,
|
||||
claimSharedSnapshot,
|
||||
fixtureContext,
|
||||
@@ -85,6 +86,7 @@ const REPLAY_SCENARIOS: Scenario[] = [
|
||||
configPath: AGENT.configPath,
|
||||
workspaceParent: tmpdir(),
|
||||
pinsChildToolSchemas: [1],
|
||||
pinsChildSystemPrompts: [1],
|
||||
prepareWorkspace: (cwd) => {
|
||||
writeFileSync(join(cwd, 'seed.txt'), 'prepared at runtime')
|
||||
},
|
||||
@@ -126,6 +128,7 @@ function staleRefreshFixtures(dir: string): void {
|
||||
writeFileSync(join(dir, 'pin-turn', 'system-prompt.expected.md'), 'STALE PROMPT\n')
|
||||
writeFileSync(join(dir, 'pin-turn', 'tool-schemas.expected.json'), '{"initial":[{"name":"stale"}],"changes":[]}\n')
|
||||
writeFileSync(join(dir, 'plain-turn', 'tool-schemas.1.expected.json'), '{"initial":[{"name":"stale-child"}],"changes":[]}\n')
|
||||
writeFileSync(join(dir, 'plain-turn', 'system-prompt.1.expected.md'), 'STALE CHILD PROMPT\n')
|
||||
|
||||
const plainBehaviorFile = join(dir, 'plain-turn', 'behavior.json')
|
||||
const plainBehavior = JSON.parse(readFileSync(plainBehaviorFile, 'utf8')) as Record<string, unknown>
|
||||
@@ -191,6 +194,8 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => {
|
||||
const childSchemas = readFileSync(join(refreshDir, 'plain-turn', 'tool-schemas.1.expected.json'), 'utf8')
|
||||
expect(childSchemas).toContain('"name": "child-only"')
|
||||
expect(childSchemas).not.toContain('stale-child')
|
||||
const childPrompt = readFileSync(join(refreshDir, 'plain-turn', 'system-prompt.1.expected.md'), 'utf8')
|
||||
expect(childPrompt).toBe('SYS PROMPT\n\nCHILD GUIDANCE\n')
|
||||
|
||||
const pinSession = readFileSync(join(refreshDir, 'pin-turn', 'session.jsonl'), 'utf8')
|
||||
expect(pinSession).toContain('"cwd":"{{cwd}}"')
|
||||
@@ -591,6 +596,34 @@ describe('formatSystemPromptSnapshot', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('assertChildSystemPromptSnapshot', () => {
|
||||
const label = 'plain-turn/system-prompt.1.expected.md'
|
||||
|
||||
it('accepts a distinct non-empty canonical child prompt', () => {
|
||||
expect(() => {
|
||||
assertChildSystemPromptSnapshot('SYS PROMPT\n\nCHILD GUIDANCE\n', 'SYS PROMPT\n', label)
|
||||
}).not.toThrow()
|
||||
})
|
||||
|
||||
it('rejects an empty or non-canonical child prompt', () => {
|
||||
expect(() => { assertChildSystemPromptSnapshot('\n', 'SYS PROMPT\n', label) }).toThrow(/non-empty prompt/)
|
||||
expect(() => {
|
||||
assertChildSystemPromptSnapshot('CHILD GUIDANCE', 'SYS PROMPT\n', label)
|
||||
}).toThrow(/end in a newline/)
|
||||
})
|
||||
|
||||
it('rejects a child prompt that duplicates its class pin', () => {
|
||||
const classSnapshot = readFileSync(join(REPLAY_DIR, 'pin-turn', 'system-prompt.expected.md'), 'utf8')
|
||||
const marker = classSnapshot.indexOf('\n<!-- request/header change ')
|
||||
expect(marker).toBeGreaterThan(0)
|
||||
const initialClassPin = classSnapshot.slice(0, marker)
|
||||
|
||||
expect(() => {
|
||||
assertChildSystemPromptSnapshot(initialClassPin, initialClassPin, label)
|
||||
}).toThrow(/must differ from its class pin/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('headerChangeCount', () => {
|
||||
it('counts changed request headers, ignoring anchors, blanks, and other lines', () => {
|
||||
const change = JSON.stringify({ type: 'request/header', seq: 2, time: 9, data: { reason: 'change' } })
|
||||
|
||||
Reference in New Issue
Block a user