Merge remote-tracking branch 'origin/master' into codex/simp-session-log-representation

# Conflicts:
#	docs/cordis-catalog/services.md
#	docs/rfc/INDEX.md
#	docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md
#	docs/rfc/proposed/simplification/2026-07-12-simplify-session-log-representation.md
#	packages/support/acp-snapshot/README.md
#	packages/support/acp-snapshot/src/normalize.ts
#	packages/support/acp-snapshot/src/suite.ts
#	packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl
#	packages/support/acp-snapshot/tests/suite.spec.ts
This commit is contained in:
Tianyi Cui
2026-07-15 15:53:56 +08:00
290 changed files with 5428 additions and 2039 deletions

View File

@@ -6,7 +6,8 @@ Packages that exist to serve development, testing, and the examples rather than
|---|---|---|
| `acp-snapshot/` | ACP snapshot suite kit: subprocess scenario harness + golden normalizers + the `defineAcpSnapshotSuite` factory | (library — imported by example `*.snapshot.ts` suites) |
| `invariants/` | Runtime event-contract assertions for development diagnostics | (listens on `session/*`, `agent/*`) |
| `loader-smoke/` | Shared real-Loader subprocess harness for keyless example smokes | (library — imported by example e2e suites) |
| `llm-replay/` | Record/replay adapter: short-circuits `llm/stream` from a recorded session JSONL (keyless snapshot tests) | (listens on `llm/stream`) |
| `subagent-mock/` | Scripted `SubagentProvider` for deterministic seam/tool tests | (registers on `ctx.subagents`) |
`invariants` is development support but has no environment guard: it runs wherever registered, and the default `dsh-agent-core` bundle mounts it unconditionally. `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `acp-snapshot` carries the snapshot tier's harness/normalizer/suite machinery so every example's suite is a scenario table over one shared, gate-covered implementation. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers.
`invariants` is development support but has no environment guard: it runs wherever registered, and the default `dsh-agent-core` bundle mounts it unconditionally. `llm-replay` backs the demos and the snapshot test tier under the per-file coverage gate. `acp-snapshot` carries the snapshot tier's harness/normalizer/suite machinery, while `loader-smoke` owns the parallel stdio/Loader process boundary used by keyless example e2e suites. `subagent-mock` exercises the real `ctx.subagents` load path without a model or child agent. A package graduates OUT of `support/` into a product group only when it gains documented product consumers.

View File

@@ -5,8 +5,8 @@ The ACP snapshot suite kit: the shared machinery behind the keyless snapshot tie
Three layers, importable separately:
- **`runScenario` (harness)** — boots the real agent bin as a subprocess via tsx (unbuilt, Loader path), drives it over ACP JSON-RPC stdio from a deterministic `input.json` script, tees raw stdout for the golden + purity check, and harvests every persisted session JSONL (parent + subagent children, primary-first) after a graceful stdin-EOF shutdown. Parameterized by `AgentUnderTest` (`binScript`, `configPath`, `tsconfigPath` — absolute paths; the subprocess cwd is a temp dir outside the repo).
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), `scrubSystemPrompts` (prompt text → `{{system}}` in every JSONL), and `scrubRequestHeaders` (the remaining header bulk → `{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record/refresh fixture write-back, the per-header-class pin (`system-prompt.golden.md` plus the JSONL's full tool schemas) with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL prompt-scrubbed, non-pinning fixtures fully header-scrubbed). Must be called at vitest collection time.
- **Normalizers** — pure functions turning the two captured surfaces into stable text: `normalizeStdout` (JSON-RPC ids → first-seen sequence; UUIDs/cwd → tokens; doubles as the stdout-purity check), `normalizeSessionLog` (times zeroed, `seq` kept), `scrubSystemPrompts` (prompt text → `{{system}}`), `scrubToolSchemas` (schema bulk → `{{tools}}`), and `scrubRequestHeaders` (all header bulk → `{{system}}`/`{{tools}}`/`{{messagePrefix}}` outside each pin, structure kept — [pinned-header RFC](../../../docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
- **`defineAcpSnapshotSuite` (factory)** — registers the whole describe/it tree for a scenario table: per-scenario golden + re-persisted-log compares, record/refresh fixture write-back, rejection of structured `UNKNOWN_TOOL` results, the per-header-class pin (`system-prompt.golden.md` plus `tool-schemas.golden.json`) with its live uniformity guard, and the fixture guard block (no orphan scenario dirs, required files present, exactly one pin per class, every JSONL prompt/schema-scrubbed, non-pinning fixtures fully header-scrubbed). Must be called at vitest collection time.
A consuming `*.snapshot.ts` is the scenario table plus one factory call:
@@ -35,9 +35,9 @@ defineAcpSnapshotSuite({
})
```
A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode scenarios are the template. Each pinning directory's generated `system-prompt.golden.md` is the reviewable snapshot of the normalized composed prompt; `session.jsonl` stores `"system":"{{system}}"` while retaining the complete tool list. A pin whose scenario legitimately changes its header mid-run declares `expectedHeaderChanges`; the Markdown snapshot then records each later full prompt under a `request/header change` marker.
A scenario booting a differently-composed tree sets its own `configPath` (an overlay whose basename still ends in `cordis.yml`, so the bin's replay swap finds the sibling `*cordis.snapshot.yml`) and, when that composition changes the request header, its own `headerClass` with its own pinning scenario — the acp-agent example's Code Mode and filesystem scenarios are templates. Each pinning directory stores the normalized full prompt sequence in generated `system-prompt.golden.md` and the corresponding full tool-schema sequence in generated `tool-schemas.golden.json`; `session.jsonl` stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A pin with legitimate mid-run header changes declares `expectedHeaderChanges`, which fixes the length of both sidecar sequences.
Examples use a `cordis.snapshot.yml` overlay with [`dsh-llm-replay`](../llm-replay/README.md). Recording calls the live model and updates model fixtures; keyless refresh replays those fixtures and updates derived stdout, session-log, and prompt snapshots. See the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
Examples use a `cordis.snapshot.yml` overlay with [`dsh-llm-replay`](../llm-replay/README.md). Recording calls the live model and updates model fixtures; keyless refresh replays those fixtures and updates derived stdout, session-log, prompt, and tool-schema snapshots. See the [snapshot RFC](../../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md).
`suite.ts` imports Vitest, so use this package only inside a Vitest run. The ACP-specific script queues permission answers by stable option kind and maps them to current option ids; a missing answer cancels, while an unavailable kind fails the scenario after cancelling the agent request. It can also set session config options or assert that unknown ids and values are rejected in the transcript.

View File

@@ -27,9 +27,9 @@
"vitest": "^4.1.8"
},
"peerDependencies": {
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -20,6 +20,7 @@ export {
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
scrubToolSchemas,
type NormalizeContext,
} from './normalize.ts'
export {

View File

@@ -1,8 +1,8 @@
/**
* Pure ACP transcript and session-log normalizers. They scrub session ids, temp cwd, RPC ids,
* timestamps, and hook duration while preserving deterministic event sequence numbers.
* Request-header scrubbers stay separate so one scenario per header class can pin tools and a
* readable prompt while other fixtures omit duplicated header bulk.
* Request-header scrubbers stay composable so one scenario per header class can pin prompt and
* tool-schema sidecars while retaining any model-visible prefix in the session log.
* @module @deepseek-ai/dsh-acp-snapshot/normalize
*/
@@ -123,7 +123,21 @@ export function normalizeSessionLog(rawLog: string, ctx: NormalizeContext): stri
* @returns The JSONL with system-prompt content tokenized.
*/
export function scrubSystemPrompts(rawLog: string): string {
return scrubHeaderContent(rawLog, false)
return scrubHeaderContent(rawLog, { system: true })
}
/**
* Replace tool schemas in full request-header snapshots with `{{tools}}`
* tokens while retaining field presence. System prompts and session-prefix
* messages stay verbatim so pinning fixtures can move only schema bulk into
* their dedicated JSON sidecar. Lines without a tool payload pass through
* byte-for-byte; the transform is idempotent.
*
* @param rawLog The raw session `.jsonl` content.
* @returns The JSONL with tool-schema content tokenized.
*/
export function scrubToolSchemas(rawLog: string): string {
return scrubHeaderContent(rawLog, { tools: true })
}
/**
@@ -138,11 +152,18 @@ export function scrubSystemPrompts(rawLog: string): string {
* @returns The JSONL with all header bulk tokenized, other lines byte-identical.
*/
export function scrubRequestHeaders(rawLog: string): string {
return scrubHeaderContent(rawLog, true)
return scrubHeaderContent(rawLog, { system: true, tools: true, prefix: true })
}
/** Transform header content, optionally including tool schemas and the session prefix. */
function scrubHeaderContent(rawLog: string, scrubToolsAndPrefix: boolean): string {
/** Which independent request-header payloads a scrubber replaces. */
interface HeaderScrubOptions {
system?: boolean
tools?: boolean
prefix?: boolean
}
/** Transform the selected request-header payloads. */
function scrubHeaderContent(rawLog: string, options: HeaderScrubOptions): string {
const lines = rawLog.split('\n')
const out = lines.map((line) => {
if (line.trim().length === 0) return line
@@ -153,9 +174,9 @@ function scrubHeaderContent(rawLog: string, scrubToolsAndPrefix: boolean): strin
const header = data.header as Record<string, unknown> | null | undefined
if (header === null || typeof header !== 'object') return line
let touched = false
if ('system' in header) { header.system = SYSTEM; touched = true }
if (scrubToolsAndPrefix && 'tools' in header) { header.tools = TOOLS; touched = true }
if (scrubToolsAndPrefix && Array.isArray(header.messagePrefix)) {
if (options.system === true && 'system' in header) { header.system = SYSTEM; touched = true }
if (options.tools === true && 'tools' in header) { header.tools = TOOLS; touched = true }
if (options.prefix === true && Array.isArray(header.messagePrefix)) {
header.messagePrefix = header.messagePrefix.map(() => MESSAGE_PREFIX)
touched = true
}

View File

@@ -5,11 +5,10 @@
* model scenarios from the live API, while refresh mode replays committed
* scripts and rewrites derived artifacts without a key.
*
* Exactly one scenario per header-composition class pins tool schemas in JSONL
* and the system prompt in Markdown. Every live header is checked against that
* pin, so session-dependent composition must declare a separate class instead
* of escaping coverage.
*
* Exactly one scenario per header-composition class pins the full prompt and
* tool-schema sequences in dedicated sidecars. Every live header is checked
* against that pin, so session-dependent composition must declare a separate
* class instead of escaping coverage.
* @module @deepseek-ai/dsh-acp-snapshot/suite
*/
@@ -24,11 +23,18 @@ import {
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
scrubToolSchemas,
} from './normalize.ts'
/** The readable system-prompt snapshot beside each header-pinning fixture. */
const SYSTEM_PROMPT_SNAPSHOT = 'system-prompt.golden.md'
/** The structured tool-schema snapshot beside each header-pinning fixture. */
const TOOL_SCHEMAS_SNAPSHOT = 'tool-schemas.golden.json'
/** Stable session-log token standing in for the sidecar's initial schemas. */
const TOOLS_TOKEN = '{{tools}}'
/** A snapshot scenario and how its fixtures are produced. */
export interface Scenario {
name: string
@@ -71,8 +77,8 @@ export interface Scenario {
*/
childSessions?: number
/**
* Whether this scenario is its header class's sole request-header pin. Its Markdown file owns
* the prompt, its JSONL keeps tool schemas, and every classmate is checked for equality.
* Whether this scenario is its header class's sole request-header pin. Dedicated sidecars own
* the prompt and tool schemas, while every classmate is checked for equality.
*/
pinsHeader?: boolean
/**
@@ -184,6 +190,77 @@ export function normalizedSystemPrompts(rawLog: string, ctx: NormalizeContext):
})
}
/**
* The normalized tool-schema arrays carried by request headers in a session
* JSONL, in log order. Headers without an array-valued tools field are omitted
* so callers can assert one schema set per header explicitly.
*
* @param rawLog The session `.jsonl` content to inspect.
* @param ctx The volatile values of the run that produced it.
* @returns The normalized initial tool-schema arrays, in header order.
*/
export function normalizedToolSchemas(rawLog: string, ctx: NormalizeContext): unknown[][] {
return normalizedHeaders(rawLog, ctx).flatMap((header) => {
if (header === null || typeof header !== 'object') return []
const tools = (header as { tools?: unknown }).tools
return Array.isArray(tools) ? [tools] : []
})
}
/** The structured contents of a tool-schema sidecar. */
export interface ToolSchemasSnapshot {
/** The complete tool schemas from the pinned request header. */
initial: unknown[]
/** Complete tool schemas from subsequent changed-header snapshots. */
changes: unknown[][]
}
/**
* Render the full tool-schema sequence as canonical, readable JSON.
*
* @param initial The pinned request header's complete tool schemas.
* @param changes Complete tool schemas from later changed headers.
* @returns A pretty-printed JSON snapshot ending in one newline.
*/
export function formatToolSchemasSnapshot(initial: readonly unknown[], changes: readonly unknown[][] = []): string {
return `${JSON.stringify({ initial, changes }, null, 2)}\n`
}
/**
* Parse and validate the stable top-level shape of a tool-schema sidecar.
*
* @param snapshot The JSON sidecar text.
* @returns Its initial and changed-header schema sets.
*/
export function parseToolSchemasSnapshot(snapshot: string): ToolSchemasSnapshot {
const parsed = JSON.parse(snapshot) as unknown
if (parsed === null || typeof parsed !== 'object' || Array.isArray(parsed)) {
throw new Error('acp-snapshot: tool-schema snapshot must be an object')
}
const { initial, changes } = parsed as { initial?: unknown; changes?: unknown }
if (!Array.isArray(initial) || !Array.isArray(changes) || !changes.every(Array.isArray)) {
throw new Error('acp-snapshot: tool-schema snapshot must carry array-valued initial and changes fields')
}
return { initial, changes }
}
/**
* Restore one sidecar schema set into a tokenized pinned header.
*
* @param header The parsed request header carrying `tools: "{{tools}}"`.
* @param schemas The complete schemas for this full header snapshot.
* @returns A copy of the header with its complete schemas restored.
*/
export function restorePinnedToolSchemas(header: unknown, schemas: readonly unknown[]): unknown {
if (header === null || typeof header !== 'object' || Array.isArray(header)) {
throw new Error('acp-snapshot: pinned request header must be an object')
}
if ((header as { tools?: unknown }).tools !== TOOLS_TOKEN) {
throw new Error(`acp-snapshot: pinned request header tools must equal ${TOOLS_TOKEN}`)
}
return { ...header, tools: schemas }
}
/**
* Render a normalized prompt as a repository-friendly Markdown snapshot.
* Prompt text is unchanged except that a missing terminal newline is added so
@@ -241,6 +318,27 @@ function parseJsonlRecords(text: string): Record<string, unknown>[] {
.map(line => JSON.parse(line) as Record<string, unknown>)
}
/**
* Find tool calls whose structured result reports `UNKNOWN_TOOL`.
*
* Snapshot refresh must not turn a missing registration into accepted behavior;
* intentional unknown-tool behavior belongs in a focused unit or e2e test.
*
* @param rawLog The session JSONL to inspect.
* @returns The failing call ids in log order, using a diagnostic placeholder when absent.
*/
export function unknownToolCallIds(rawLog: string): string[] {
return parseJsonlRecords(rawLog).flatMap((record) => {
if (record.type !== 'tool/result') return []
const data = record.data
if (data === null || typeof data !== 'object') return []
const { callId, error } = data as { callId?: unknown; error?: unknown }
if (error === null || typeof error !== 'object') return []
if ((error as { code?: unknown }).code !== 'UNKNOWN_TOOL') return []
return [typeof callId === 'string' ? callId : '<missing callId>']
})
}
/**
* Build the cross-log id/cwd replacements used by refresh write-back.
*
@@ -368,6 +466,11 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
...scenario.configPath !== undefined ? { configPath: scenario.configPath } : {},
})
for (const log of result.sessionLogs) {
expect(unknownToolCallIds(log.content), `session ${log.id}: snapshot scenarios must not accept UNKNOWN_TOOL`)
.toEqual([])
}
// Scrub every volatile id the run produced: the ACP server-issued session id plus every
// harvested log's recorded id (a subagent child id never surfaces over ACP, but it
// appears in the child's own log header).
@@ -380,9 +483,9 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
// Record writes live model fixtures; keyless refresh writes every comparable replayed
// fixture. Pins keep tools but all JSONL files scrub prompt text.
// fixture. Pinning JSONL keeps prefixes but moves prompts and schemas into sidecars.
const scrub = scenario.pinsHeader === true
? scrubSystemPrompts
? (log: string): string => scrubToolSchemas(scrubSystemPrompts(log))
: scrubRequestHeaders
const fixtureFiles = ['session.jsonl', ...Array.from({ length: childSessions }, (_, i) => `session.${i + 1}.jsonl`)]
const existingFixtures = REFRESHING
@@ -411,6 +514,15 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
expect(prompts.length, `${mode} produced no system prompt to snapshot`).toBeGreaterThan(0)
const snapshot = formatSystemPromptSnapshot(prompts[0] as string, prompts.slice(1))
await writeFile(join(dir, SYSTEM_PROMPT_SNAPSHOT), snapshot)
const schemaSets = normalizedToolSchemas(primary.content, ctx)
expect(schemaSets.length, `${mode} produced no tool schemas to snapshot`).toBeGreaterThan(0)
expect(schemaSets.length, `${mode} produced a tool-schema sequence that differs from its prompt sequence`)
.toBe(prompts.length)
await writeFile(join(dir, TOOL_SCHEMAS_SNAPSHOT), formatToolSchemasSnapshot(
schemaSets[0] as unknown[],
schemaSets.slice(1),
))
}
}
@@ -433,10 +545,8 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
}
// Header-uniformity guard: every live header in a class must equal the
// class pin split across its JSONL header (system token + real tools)
// and readable Markdown prompt. A pinning scenario may carry declared
// changed-header snapshots; each full prompt lives in that Markdown.
// Every live full header must equal its class pin reconstructed from
// tokenized JSONL plus readable prompt and structured schema sidecars.
/* v8 ignore next -- construction guarantees the pin exists; a miss would fail the one-header assertion loudly. */
const pinningScenario = pinningByClass.get(classOf(scenario)) ?? scenario
const pinningDir = join(snapshotsDir, pinningScenario.name)
@@ -446,6 +556,15 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
const initialPromptSnapshot = initialSystemPromptSnapshot(promptSnapshot)
expect(pinned.length, `the pinning fixture (${pinningScenario.name}) has an unexpected request/header count`)
.toBe(1 + (pinningScenario.expectedHeaderChanges ?? 0))
const toolSchemasSnapshot = await readFile(join(pinningDir, TOOL_SCHEMAS_SNAPSHOT), 'utf8')
const toolSchemas = parseToolSchemasSnapshot(toolSchemasSnapshot)
const pinnedSchemaSets = [toolSchemas.initial, ...toolSchemas.changes]
expect(pinnedSchemaSets.length, `the pinning fixture (${pinningScenario.name}) has an unexpected tool-schema count`)
.toBe(pinned.length)
const pinnedHeaders = pinned.map((header, index) => restorePinnedToolSchemas(
header,
pinnedSchemaSets[index] as unknown[],
))
for (const [logIndex, log] of result.sessionLogs.entries()) {
const expectedChanges = scenario.pinsHeader === true && logIndex === 0
? scenario.expectedHeaderChanges ?? 0
@@ -454,10 +573,13 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
.toBe(expectedChanges)
const headers = normalizedHeaders(scrubSystemPrompts(log.content), ctx)
const prompts = normalizedSystemPrompts(log.content, ctx)
const schemaSets = normalizedToolSchemas(log.content, ctx)
expect(prompts.length, `session ${log.id}: every request/header must carry a string system prompt`)
.toBe(headers.length)
expect(schemaSets.length, `session ${log.id}: every request/header must carry an array-valued tools field`)
.toBe(headers.length)
for (const [k, header] of headers.entries()) {
const expected = expectedChanges > 0 ? pinned[k] : pinned[0]
const expected = expectedChanges > 0 ? pinnedHeaders[k] : pinnedHeaders[0]
expect(header, `session ${log.id}: request/header #${k + 1} diverged from the pinned (${pinningScenario.name}) header`)
.toEqual(expected)
if (expectedChanges === 0) {
@@ -471,6 +593,11 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
prompts.slice(1),
), `session ${log.id}: changed system prompts diverged from ${pinningScenario.name}/${SYSTEM_PROMPT_SNAPSHOT}`)
.toEqual(promptSnapshot)
expect(formatToolSchemasSnapshot(
schemaSets[0] as unknown[],
schemaSets.slice(1),
), `session ${log.id}: changed tool schemas diverged from ${pinningScenario.name}/${TOOL_SCHEMAS_SNAPSHOT}`)
.toEqual(toolSchemasSnapshot)
}
}
})
@@ -499,6 +626,8 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
.toBe(overridden === true)
expect(existsSync(join(dir, SYSTEM_PROMPT_SNAPSHOT)), `${name}/${SYSTEM_PROMPT_SNAPSHOT} presence must match \`pinsHeader\``)
.toBe(pinsHeader === true)
expect(existsSync(join(dir, TOOL_SCHEMAS_SNAPSHOT)), `${name}/${TOOL_SCHEMAS_SNAPSHOT} presence must match \`pinsHeader\``)
.toBe(pinsHeader === true)
// A nested-agent scenario ships one child fixture per recorded subagent
// session (`session.1.jsonl` …), the replay source for that child session.
for (const childFixture of childFixturePaths(dir, childSessions ?? 0)) {
@@ -522,29 +651,37 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
}
})
it('every pinning fixture carries one anchor, one readable prompt artifact, and its declared changes', async () => {
// The live uniformity guard runs only in NON-pinning scenarios, so a
// class made of just its pinning scenario would otherwise accept a
// re-recorded pin with undeclared mid-run header changes. Assert the
// committed pins directly; a scenario whose arc legitimately rewrites
// a prompt section declares the exact count via expectedHeaderChanges.
it('every pinning fixture carries one tokenized header sequence and two sidecars', async () => {
// Assert the committed pin directly because a class containing only its
// pinning scenario has no non-pinning live run to catch undeclared changes.
for (const scenario of pinningByClass.values()) {
const fixture = await readFile(join(snapshotsDir, scenario.name, 'session.jsonl'), 'utf8')
const headers = normalizedHeaders(fixture, fixtureContext(fixture))
const promptSnapshot = await readFile(join(snapshotsDir, scenario.name, SYSTEM_PROMPT_SNAPSHOT), 'utf8')
expect(headers.length, `${scenario.name}: unexpected request/header count`)
.toBe(1 + (scenario.expectedHeaderChanges ?? 0))
const toolSchemasSnapshot = await readFile(join(snapshotsDir, scenario.name, TOOL_SCHEMAS_SNAPSHOT), 'utf8')
const toolSchemas = parseToolSchemasSnapshot(toolSchemasSnapshot)
const schemaSets = [toolSchemas.initial, ...toolSchemas.changes]
expect(schemaSets.length, `${scenario.name}: tool-schema sequence must match the header sequence`)
.toBe(headers.length)
for (const [index, header] of headers.entries()) {
expect(() => restorePinnedToolSchemas(header, schemaSets[index] as unknown[]), `${scenario.name}: tools must use the sidecar token`)
.not.toThrow()
}
expect(promptSnapshot.length, `${scenario.name}/${SYSTEM_PROMPT_SNAPSHOT} must not be empty`).toBeGreaterThan(0)
expect(promptSnapshot.endsWith('\n'), `${scenario.name}/${SYSTEM_PROMPT_SNAPSHOT} must end in a newline`).toBe(true)
expect(toolSchemasSnapshot, `${scenario.name}/${TOOL_SCHEMAS_SNAPSHOT} must use canonical JSON formatting`)
.toBe(formatToolSchemasSnapshot(toolSchemas.initial, toolSchemas.changes))
expect(headerChangeCount(fixture), `${scenario.name}: a pinning fixture must carry exactly its declared changed headers`)
.toBe(scenario.expectedHeaderChanges ?? 0)
}
})
it('every committed JSONL omits system prompts and only pinning fixtures keep other header bulk', async () => {
// System prompts always live in the readable Markdown artifact. Header
// pins keep tool schemas/prefixes in JSONL; every other fixture tokenizes
// all header bulk. Fixed-point checks make both storage rules fail loud.
it('every committed JSONL has valid tool results and canonical header storage', async () => {
// Prompts and schemas always leave JSONL. Header pins retain prefixes;
// every other fixture tokenizes those too. Fixed-point checks make both
// storage rules fail loud.
for (const scenario of scenarios) {
const dir = join(snapshotsDir, scenario.name)
const files = [
@@ -553,12 +690,13 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
]
for (const file of files) {
const fixture = await readFile(join(dir, file), 'utf8')
expect(unknownToolCallIds(fixture), `${scenario.name}/${file} contains UNKNOWN_TOOL`)
.toEqual([])
expect(scrubSystemPrompts(fixture), `${scenario.name}/${file} carries an unscrubbed system prompt`)
.toEqual(fixture)
if (scenario.pinsHeader === true) {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} must pin the non-system header content`)
.not.toEqual(fixture)
} else {
expect(scrubToolSchemas(fixture), `${scenario.name}/${file} carries unscrubbed tool schemas`)
.toEqual(fixture)
if (scenario.pinsHeader !== true) {
expect(scrubRequestHeaders(fixture), `${scenario.name}/${file} carries unscrubbed header content`)
.toEqual(fixture)
}

View File

@@ -1,2 +1,2 @@
{"type":"session","id":"ccdc749f-56f3-4267-9750-598b5c60b7b2","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy"}
{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"request/header","seq":0,"time":4,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}

View File

@@ -0,0 +1,12 @@
{
"initial": [
{
"name": "t1",
"description": "D1",
"parameters": {
"type": "object"
}
}
],
"changes": []
}

View File

@@ -1,4 +1,4 @@
{"type":"session","id":"12121212-3434-4545-8686-787878787878","createdAt":7,"cwd":"/rec/pin-cwd"}
{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"initial"}}
{"type":"request/header","seq":1,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":[{"name":"t1","description":"D1","parameters":{"type":"object"}}]},"reason":"change"}}
{"type":"request/header","seq":0,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
{"type":"request/header","seq":1,"time":7,"data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}}
{"type":"turn/start","seq":2,"time":7,"data":{"turn":1}}

View File

@@ -0,0 +1,22 @@
{
"initial": [
{
"name": "t1",
"description": "D1",
"parameters": {
"type": "object"
}
}
],
"changes": [
[
{
"name": "t1",
"description": "D1",
"parameters": {
"type": "object"
}
}
]
]
}

View File

@@ -5,6 +5,7 @@ import {
normalizeStdout,
scrubRequestHeaders,
scrubSystemPrompts,
scrubToolSchemas,
} from '../src/normalize.ts'
/**
@@ -238,3 +239,45 @@ describe('scrubSystemPrompts', () => {
expect(scrubSystemPrompts(out)).toBe(out)
})
})
describe('scrubToolSchemas', () => {
it('scrubs only tool-schema payloads while keeping prompts and prefixes verbatim', () => {
const header = JSON.stringify({
type: 'request/header', seq: 1, time: 2,
data: {
header: {
system: 'full prompt',
tools: [{ name: 'read', description: 'full schema', parameters: { type: 'object' } }],
messagePrefix: [{ role: 'user', content: [{ type: 'text', text: 'full prefix' }] }],
},
reason: 'initial',
},
})
const changed = JSON.stringify({
type: 'request/header', seq: 2, time: 3,
data: {
header: {
system: 'new prompt',
tools: [{ name: 'grep', description: 'new schema' }],
messagePrefix: [{ role: 'user', content: [{ type: 'text', text: 'changed prefix' }] }],
},
reason: 'change',
},
})
const systemOnly = JSON.stringify({
type: 'request/header', seq: 3, time: 4,
data: { header: { system: 'prompt only' }, reason: 'resume' },
})
const out = scrubToolSchemas(`${header}\n${changed}\n${systemOnly}\n`)
expect(out.match(/"tools":"{{tools}}"/g)).toHaveLength(2)
expect(out).not.toContain('full schema')
expect(out).not.toContain('new schema')
expect(out).toContain('full prompt')
expect(out).toContain('new prompt')
expect(out).toContain('full prefix')
expect(out).toContain('changed prefix')
expect(out.split('\n')[2]).toBe(systemOnly)
expect(scrubToolSchemas(out)).toBe(out)
})
})

View File

@@ -10,10 +10,15 @@ import {
fixtureContext,
formatSystemPromptSnapshot,
headerChangeCount,
formatToolSchemasSnapshot,
normalizedHeaders,
normalizedSystemPrompts,
normalizedToolSchemas,
parseToolSchemasSnapshot,
refreshFixtureReplacements,
restorePinnedToolSchemas,
stabilizeRefreshLog,
unknownToolCallIds,
} from '../src/suite.ts'
/**
@@ -69,6 +74,7 @@ afterAll(async () => {
function staleRefreshFixtures(dir: string): void {
writeFileSync(join(dir, 'plain-turn', 'stdout.golden.jsonl'), 'stale stdout\n')
writeFileSync(join(dir, 'pin-turn', 'system-prompt.golden.md'), 'STALE PROMPT\n')
writeFileSync(join(dir, 'pin-turn', 'tool-schemas.golden.json'), '{"initial":[{"name":"stale"}],"changes":[]}\n')
const plainBehaviorFile = join(dir, 'plain-turn', 'behavior.json')
const plainBehavior = JSON.parse(readFileSync(plainBehaviorFile, 'utf8')) as Record<string, unknown>
@@ -126,6 +132,9 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => {
'NEW PROMPT LINE',
'',
].join('\n'))
const schemas = readFileSync(join(refreshDir, 'pin-turn', 'tool-schemas.golden.json'), 'utf8')
expect(schemas).toContain('"description": "D1"')
expect(schemas).not.toContain('"name":"stale"')
})
})
@@ -236,6 +245,23 @@ describe('normalizedSystemPrompts', () => {
})
})
describe('normalizedToolSchemas', () => {
it('extracts normalized schema arrays and omits absent or non-array fields', () => {
const log = [
'{"type":"session","id":"a","createdAt":5,"cwd":"/w"}',
'{"type":"request/header","seq":0,"time":9,"data":{"header":{"tools":[{"name":"read","description":"work in /w"}]}}}',
'{"type":"request/header","seq":1,"time":9,"data":{"header":{}}}',
'{"type":"request/header","seq":2,"time":9,"data":{"header":{"tools":null}}}',
'{"type":"request/header","seq":3,"time":9,"data":{"header":null}}',
'{"type":"request/header","seq":4,"time":9,"data":{"header":"invalid"}}',
'',
].join('\n')
expect(normalizedToolSchemas(log, { sessionIds: [], cwd: '/w' })).toEqual([
[{ name: 'read', description: 'work in {{cwd}}' }],
])
})
})
describe('formatSystemPromptSnapshot', () => {
it('adds a missing terminal newline without changing an existing one', () => {
expect(formatSystemPromptSnapshot('prompt')).toBe('prompt\n')
@@ -263,6 +289,61 @@ describe('headerChangeCount', () => {
})
})
describe('tool-schema snapshots', () => {
const snapshot = {
initial: [{ name: 'read', description: 'Read a file.' }],
changes: [[{ name: 'grep', description: 'Search files.' }]],
}
it('formats and parses canonical structured JSON', () => {
const formatted = formatToolSchemasSnapshot(snapshot.initial, snapshot.changes)
expect(formatted).toBe(`${JSON.stringify(snapshot, null, 2)}\n`)
expect(parseToolSchemasSnapshot(formatted)).toEqual(snapshot)
})
it('rejects invalid top-level and field shapes', () => {
expect(() => parseToolSchemasSnapshot('null')).toThrow(/must be an object/)
expect(() => parseToolSchemasSnapshot('"invalid"')).toThrow(/must be an object/)
expect(() => parseToolSchemasSnapshot('[]')).toThrow(/must be an object/)
expect(() => parseToolSchemasSnapshot('{"initial":{},"changes":[]}')).toThrow(/array-valued/)
expect(() => parseToolSchemasSnapshot('{"initial":[],"changes":{}}')).toThrow(/array-valued/)
expect(() => parseToolSchemasSnapshot('{"initial":[],"changes":[{}]}')).toThrow(/array-valued/)
})
it('restores initial schemas into the pinned header token', () => {
expect(restorePinnedToolSchemas({ system: '{{system}}', tools: '{{tools}}' }, snapshot.initial))
.toEqual({ system: '{{system}}', tools: snapshot.initial })
})
it('rejects invalid headers and a missing tool token', () => {
expect(() => restorePinnedToolSchemas(null, snapshot.initial)).toThrow(/must be an object/)
expect(() => restorePinnedToolSchemas('invalid', snapshot.initial)).toThrow(/must be an object/)
expect(() => restorePinnedToolSchemas([], snapshot.initial)).toThrow(/must be an object/)
expect(() => restorePinnedToolSchemas({ tools: [] }, snapshot.initial)).toThrow(/must equal/)
})
})
describe('unknownToolCallIds', () => {
it('returns structured UNKNOWN_TOOL call ids and ignores other results', () => {
const log = [
'{"type":"tool/result","data":{"callId":"missing","error":{"code":"UNKNOWN_TOOL"}}}',
'{"type":"tool/result","data":{"callId":"failed","error":{"code":"EXECUTION_FAILED"}}}',
'{"type":"tool/result","data":null}',
'{"type":"tool/result","data":"invalid"}',
'{"type":"tool/result","data":{"error":null}}',
'{"type":"tool/result","data":{"error":"invalid"}}',
'{"type":"assistant/message","data":{"error":{"code":"UNKNOWN_TOOL"}}}',
'{"type":"tool/result","data":{"error":{"code":"UNKNOWN_TOOL"}}}',
'',
].join('\n')
expect(unknownToolCallIds(log)).toEqual(['missing', '<missing callId>'])
})
it('returns no failures for ordinary tool results', () => {
expect(unknownToolCallIds('{"type":"tool/result","data":{"callId":"ok"}}\n')).toEqual([])
})
})
describe('refreshFixtureReplacements', () => {
it('maps fresh ids and cwd values to the existing fixture values, skipping non-replacements', () => {
const log = (content: string): HarvestedLog => ({ id: 'diagnostic', createdAt: 1, content })

View File

@@ -26,13 +26,17 @@
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-scope": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-scope": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"cordis": "^4.0.0-rc.6"
"@deepseek-ai/dsh-subagent": "workspace:^",
"@deepseek-ai/dsh-system-prompt": "workspace:^",
"@deepseek-ai/dsh-tools": "workspace:^",
"@deepseek-ai/dsh-user-approval": "workspace:^",
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -14,6 +14,7 @@ import type { CallId, GenerateOptions } from '@deepseek-ai/dsh-llm'
import type { Agent, AgentStatus } from '@deepseek-ai/dsh-agent'
import { Session, SessionId, foldRequestHeader } from '@deepseek-ai/dsh-session'
import type { SessionEvent, SurfaceEventType } from '@deepseek-ai/dsh-session'
import { scopedSubjectResolverFor } from './scoped-events.generated.ts'
export const name = 'invariants'
export const inject = ['sessions']
@@ -75,17 +76,6 @@ interface SessionTraceTransition {
seq: number
}
/** Event payload prefix for scoped seams whose first argument names its agent. */
interface AgentSubject {
agent: Agent
}
/** Structural subject fields used without coupling this dev plugin to owning services. */
interface ScopedSubjectFields {
agent?: Agent
scope?: object
}
/** Assert that a step-scoped event names the currently open turn and step. */
function requireOpenStep(trace: SessionTrace, kind: string, turn: number, step: number): void {
if (trace.openTurn !== turn || trace.openStep !== step) {
@@ -410,40 +400,12 @@ export function apply(ctx: Context): void {
// (agent-scoped listeners over-hear foreign agents), and a mis-keyed one
// delivers to the wrong agent's listeners. `internal/dispatch` fires
// synchronously before listener delivery, so a violation throws at the
// dispatching call site. The table maps each family to how its subject is
// read from the event arguments; `null` = the subject is not recoverable
// from the arguments (session events key by the OWNING agent; subagent
// lifecycle events key by the delegating parent), so only carrier
// PRESENCE is asserted there.
const scopedSubject: Record<string, ((args: unknown[]) => unknown) | null> = {
'agent/created': args => args[0],
'agent/disposed': args => args[0],
'agent/status': args => args[0],
'agent/queued': args => args[0],
'agent/session-start': args => args[0],
'agent/pre-step': args => args[0],
'agent/prompt-submit': args => args[0],
'agent/request': args => args[0],
'agent/session-prefix': args => args[0],
'agent/step-result': args => args[0],
'agent/turn-continuation': args => args[0],
'agent/turn-stop': args => args[0],
'agent/error': args => args[0],
'approval/request': args => (args[0] as AgentSubject).agent,
'tools/pre-execute': args => (args[0] as ScopedSubjectFields).agent,
'tools/execute': args => (args[0] as ScopedSubjectFields).agent,
'tools/post-execute': args => (args[0] as ScopedSubjectFields).agent,
'tools/result': args => (args[0] as ScopedSubjectFields).agent,
'system-prompt/assemble': args => (args[1] as ScopedSubjectFields).scope,
'session/created': null,
'session/disposed': null,
'session/event': null,
'session/flush': null,
'subagent/start': null,
'subagent/end': null,
}
// dispatching call site. The generated table maps each family to the unique
// payload path whose Program type matches the real scopeTarget routing key;
// `null` means the key is external to the payload, so only carrier presence
// can be asserted.
ctx.on('internal/dispatch', (_mode, name, args, thisArg) => {
const subjectOf = scopedSubject[name]
const subjectOf = scopedSubjectResolverFor(name)
if (subjectOf === undefined) return
if (!isScopeCarrier(thisArg)) {
throw new InvariantError(

View File

@@ -0,0 +1,69 @@
/**
* Generated scoped-event routing-subject resolvers for dsh-invariants.
* Do not edit by hand; run `pnpm run gen-scoped-events`.
*
* @module @deepseek-ai/dsh-invariants/scoped-events.generated
*/
import type { Events } from 'cordis'
import type { Scoped } from '@deepseek-ai/dsh-scope'
import type {} from '@deepseek-ai/dsh-agent'
import type {} from '@deepseek-ai/dsh-session'
import type {} from '@deepseek-ai/dsh-subagent'
import type {} from '@deepseek-ai/dsh-system-prompt'
import type {} from '@deepseek-ai/dsh-tools'
import type {} from '@deepseek-ai/dsh-user-approval'
type ScopedEventName = {
[K in keyof Events]: ThisParameterType<Events[K]> extends Scoped<object> ? K : never
}[keyof Events]
type ScopedSubjectResolver = (args: readonly unknown[]) => unknown
function adapt<K extends ScopedEventName>(
resolver: (args: Parameters<Events[K]>) => unknown,
): ScopedSubjectResolver {
return args => resolver(args as Parameters<Events[K]>)
}
const scopedSubjectResolvers = Object.freeze({
'agent/created': adapt<'agent/created'>(args => args[0]),
'agent/disposed': adapt<'agent/disposed'>(args => args[0]),
'agent/error': adapt<'agent/error'>(args => args[0]),
'agent/pre-step': adapt<'agent/pre-step'>(args => args[0]),
'agent/prompt-submit': adapt<'agent/prompt-submit'>(args => args[0]),
'agent/queued': adapt<'agent/queued'>(args => args[0]),
'agent/request': adapt<'agent/request'>(args => args[0]),
'agent/session-prefix': adapt<'agent/session-prefix'>(args => args[0]),
'agent/session-start': adapt<'agent/session-start'>(args => args[0]),
'agent/status': adapt<'agent/status'>(args => args[0]),
'agent/step-result': adapt<'agent/step-result'>(args => args[0]),
'agent/turn-continuation': adapt<'agent/turn-continuation'>(args => args[0]),
'agent/turn-stop': adapt<'agent/turn-stop'>(args => args[0]),
'approval/request': adapt<'approval/request'>(args => args[0].agent),
'session/created': null,
'session/disposed': null,
'session/event': null,
'session/flush': null,
'subagent/end': null,
'subagent/start': null,
'system-prompt/assemble': adapt<'system-prompt/assemble'>(args => args[1].scope),
'tools/execute': adapt<'tools/execute'>(args => args[0].agent),
'tools/post-execute': adapt<'tools/post-execute'>(args => args[0].agent),
'tools/pre-execute': adapt<'tools/pre-execute'>(args => args[0].agent),
'tools/result': adapt<'tools/result'>(args => args[0].agent),
} as const satisfies Readonly<Record<ScopedEventName, ScopedSubjectResolver | null>>)
const scopedSubjectResolverIndex: Readonly<Record<string, ScopedSubjectResolver | null>> = scopedSubjectResolvers
/**
* Resolve the routing key named by one scoped event payload. A null
* resolver means the payload cannot expose its external routing key, so the
* invariant checks carrier presence only.
* @param event - runtime Cordis event name.
* @returns the generated subject resolver, null for presence-only,
* or undefined when the event is not scope-filtered.
*/
export function scopedSubjectResolverFor(event: string): ScopedSubjectResolver | null | undefined {
return scopedSubjectResolverIndex[event]
}

View File

@@ -834,7 +834,7 @@ describe('scoped-dispatch invariants', () => {
it('accepts a matching carrier and rejects a mismatched one for EVERY agent-subject event', async () => {
const ctx = await scopedCtx()
// Real Session objects: the session-start tracker WeakSet-keys them.
// Real Session objects keep the synthetic Agent handles structurally valid.
const agent = { id: 'a1', session: new Session(SessionId('a1-s')) } as unknown as Agent
const other = { id: 'a2', session: new Session(SessionId('a2-s')) } as unknown as Agent
// One dispatch per table row keeps every subject extractor covered: the
@@ -855,9 +855,9 @@ describe('scoped-dispatch invariants', () => {
['agent/error', [agent, 1, 0, new Error('x')]],
['approval/request', [{ agent, toolName: 'echo' }, () => Promise.resolve('unavailable')]],
['tools/pre-execute', [{ callId: 'c', name: 't', arguments: {}, agent }, () => Promise.resolve({ kind: 'allow' })]],
['tools/execute', [{ callId: 'c', name: 't', arguments: {}, agent }, () => Promise.resolve({ callId: 'c', content: [], isError: false })]],
['tools/post-execute', [{ callId: 'c', name: 't', arguments: {}, agent }, { callId: 'c', content: [], isError: false }, () => Promise.resolve({ kind: 'accept' })]],
['tools/result', [{ callId: 'c', name: 't', arguments: {}, agent }, { callId: 'c', content: [], isError: false }]],
['tools/execute', [{ callId: 'c', name: 't', arguments: {}, agent }, () => Promise.resolve({ content: [], isError: false })]],
['tools/post-execute', [{ callId: 'c', name: 't', arguments: {}, agent }, { content: [], isError: false }, () => Promise.resolve({ kind: 'accept' })]],
['tools/result', [{ callId: 'c', name: 't', arguments: {}, agent }, { content: [], isError: false }]],
]
for (const [event, args] of rows) {
const subject = agent

View File

@@ -25,6 +25,18 @@
},
{
"path": "../../core/scope"
},
{
"path": "../../core/system-prompt"
},
{
"path": "../../ui/user-approval"
},
{
"path": "../../core/tools"
},
{
"path": "../../subagent/subagent"
}
]
}

View File

@@ -24,11 +24,11 @@
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"devDependencies": {
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
}
}

View File

@@ -0,0 +1,17 @@
# `@deepseek-ai/dsh-loader-smoke`
Shared subprocess harness for keyless example smokes that boot the real stdio-agent bin and a real `cordis.yml` through the Cordis Loader. A test supplies absolute bin/config/tsconfig paths, optional environment overrides, and stdin lines; `runLoaderSmoke` owns the isolated cwd, DSH homes, tsx path resolution, 30-second process deadline, captured diagnostics, forced kill, EOF, and cleanup.
Successful runs return stdout and stderr only after a zero exit. Non-zero exits and deadlines reject with both captured streams. `LOADER_SMOKE_TEST_TIMEOUT_MS` leaves Vitest enough room for the process-owned diagnostic timeout to fire first.
This is support-tier test infrastructure, not product API. The consumers are the Loader-path smokes under `examples/{echo-agent,coding-agent,cordis-agent}`.
## Model Experience
None, as this test-only harness boots example processes and inspects their streams without changing an assembled model request.
## Known Limitations and Deferred Work
- **Only the unbuilt tsx/Loader path is exercised** — built-bin artifacts remain the responsibility of their separate e2e smokes.
- **Captured stdout and stderr are unbounded** — a runaway child can consume memory until the deadline kills it.
- **Timeout kills only the direct child** — a process tree spawned by a faulty fixture can outlive the smoke and needs external cleanup.

View File

@@ -0,0 +1,33 @@
{
"name": "@deepseek-ai/dsh-loader-smoke",
"description": "Shared subprocess harness for keyless real-Loader example smoke tests",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"dependencies": {
"tsx": "^4.22.4"
},
"peerDependencies": {
"cordis": "^4.0.0-rc.6"
},
"devDependencies": {
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,117 @@
/**
* Shared subprocess harness for keyless example smokes that boot a real
* `cordis.yml` through the stdio-agent bin and Cordis Loader.
*
* @module @deepseek-ai/dsh-loader-smoke
*/
import { spawn } from 'node:child_process'
import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
const DEFAULT_PROCESS_TIMEOUT_MS = 30_000
const TSX_LOADER = fileURLToPath(import.meta.resolve('tsx'))
/** Vitest deadline that leaves room for the subprocess-owned 30-second diagnostic timeout. */
export const LOADER_SMOKE_TEST_TIMEOUT_MS = DEFAULT_PROCESS_TIMEOUT_MS + 15_000
/** Inputs that vary between real-Loader example smokes. */
export interface LoaderSmokeOptions {
/** Human-readable example name used in failure diagnostics. */
readonly label: string
/** Prefix for the isolated temporary process cwd. */
readonly tempDirPrefix: string
/** Absolute stdio-agent bin path. */
readonly binScript: string
/** Absolute real Loader config path. */
readonly configPath: string
/** Absolute repo tsconfig path used for unbuilt workspace-package resolution. */
readonly tsconfigPath: string
/** Environment overrides layered over the parent and isolated DSH homes. */
readonly env?: Readonly<NodeJS.ProcessEnv>
/** Lines written to stdin before EOF; omitted means immediate EOF. */
readonly stdinLines?: readonly string[]
/** Process deadline override for harness tests. */
readonly processTimeoutMs?: number
}
/** Captured output from a Loader smoke that exited successfully. */
export interface LoaderSmokeResult {
/** Complete stdout after clean exit. */
readonly stdout: string
/** Complete stderr after clean exit. */
readonly stderr: string
}
/**
* Boot one real Loader tree from an isolated cwd, write the requested stdin
* script, close stdin, and await a clean exit. The helper owns process kill and
* temp-directory cleanup on every outcome.
* @param options - example paths, environment, stdin, and diagnostic identity.
* @returns captured stdout and stderr after a zero exit.
*/
export async function runLoaderSmoke(options: LoaderSmokeOptions): Promise<LoaderSmokeResult> {
const cwd = await mkdtemp(join(tmpdir(), options.tempDirPrefix))
const processTimeoutMs = options.processTimeoutMs ?? DEFAULT_PROCESS_TIMEOUT_MS
try {
return await new Promise((resolve, reject) => {
const child = spawn(
process.execPath,
['--expose-internals', '--import', TSX_LOADER, options.binScript, options.configPath],
{
cwd,
env: {
...process.env,
DSH_HOME: join(cwd, '.dsh'),
DSH_AGENTS_HOME: join(cwd, '.agents'),
...options.env,
TSX_TSCONFIG_PATH: options.tsconfigPath,
},
stdio: ['pipe', 'pipe', 'pipe'],
},
)
let stdout = ''
let stderr = ''
let deferredFailure: Error | undefined
child.stdout.setEncoding('utf8')
child.stdout.on('data', (chunk: string) => { stdout += chunk })
child.stderr.setEncoding('utf8')
child.stderr.on('data', (chunk: string) => { stderr += chunk })
const timer = setTimeout(() => {
deferredFailure = new Error(`${options.label} did not exit within ${processTimeoutMs / 1_000}s. stdout:\n${stdout}\nstderr:\n${stderr}`)
child.kill('SIGKILL')
}, processTimeoutMs)
child.once('exit', (code) => {
clearTimeout(timer)
if (deferredFailure !== undefined) {
reject(deferredFailure)
} else if (code === 0) {
resolve({ stdout, stderr })
} else {
reject(new Error(`${options.label} exited ${String(code)}. stdout:\n${stdout}\nstderr:\n${stderr}`))
}
})
// process.execPath and a just-created pipe make these OS-error paths
// impractical to induce without replacing the boundary under test.
/* v8 ignore start */
child.once('error', (error) => {
clearTimeout(timer)
reject(new Error(`${options.label} failed to start: ${error.message}`))
})
child.stdin.once('error', (error) => {
deferredFailure ??= new Error(`${options.label} stdin failed: ${error.message}`)
child.kill('SIGKILL')
})
/* v8 ignore stop */
child.stdin.end((options.stdinLines ?? []).map(line => `${line}\n`).join(''))
})
} finally {
await rm(cwd, { recursive: true, force: true })
}
}

View File

@@ -0,0 +1,4 @@
/** Non-zero subprocess fixture for the Loader-smoke harness. */
console.error('fixture failed')
process.exitCode = 7

View File

@@ -0,0 +1,4 @@
/** Deadline subprocess fixture for the Loader-smoke harness. */
console.log('fixture hanging')
setInterval(() => {}, 1_000)

View File

@@ -0,0 +1,16 @@
/** Successful subprocess fixture for the Loader-smoke harness. */
let input = ''
process.stdin.setEncoding('utf8')
process.stdin.on('data', (chunk: string) => { input += chunk })
process.stdin.on('end', () => {
console.log(JSON.stringify({
configPath: process.argv[2],
cwd: process.cwd(),
dshHome: process.env.DSH_HOME,
agentsHome: process.env.DSH_AGENTS_HOME,
marker: process.env.LOADER_SMOKE_MARKER,
input,
}))
console.error('fixture stderr')
})

View File

@@ -0,0 +1,61 @@
import { existsSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import { describe, expect, it } from 'vitest'
import { LOADER_SMOKE_TEST_TIMEOUT_MS, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke'
const configPath = '/tmp/fixture.cordis.yml'
const tsconfigPath = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url))
const fixture = (name: string): string => fileURLToPath(new URL(`./fixtures/${name}.ts`, import.meta.url))
const canonicalTempPath = (path: string): string => path.replace(/^\/private(?=\/var\/)/, '')
describe('runLoaderSmoke', () => {
it('isolates the process, writes stdin, captures output, and removes the cwd', async () => {
const result = await runLoaderSmoke({
label: 'success fixture',
tempDirPrefix: 'loader-smoke-success-',
binScript: fixture('success'),
configPath,
tsconfigPath,
env: { LOADER_SMOKE_MARKER: 'present' },
stdinLines: ['one', 'two'],
})
const output = JSON.parse(result.stdout) as {
configPath: string
cwd: string
dshHome: string
agentsHome: string
marker: string
input: string
}
expect(output).toMatchObject({
configPath,
marker: 'present',
input: 'one\ntwo\n',
})
expect(canonicalTempPath(output.dshHome)).toBe(`${canonicalTempPath(output.cwd)}/.dsh`)
expect(canonicalTempPath(output.agentsHome)).toBe(`${canonicalTempPath(output.cwd)}/.agents`)
expect(result.stderr).toContain('fixture stderr')
expect(existsSync(output.cwd)).toBe(false)
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
it('rejects a non-zero exit with captured diagnostics', async () => {
await expect(runLoaderSmoke({
label: 'failure fixture',
tempDirPrefix: 'loader-smoke-fail-',
binScript: fixture('fail'),
configPath,
tsconfigPath,
})).rejects.toThrow('failure fixture exited 7. stdout:\n\nstderr:\nfixture failed')
})
it('kills a process at its deadline and reports captured output', async () => {
await expect(runLoaderSmoke({
label: 'hanging fixture',
tempDirPrefix: 'loader-smoke-hang-',
binScript: fixture('hang'),
configPath,
tsconfigPath,
processTimeoutMs: 100,
})).rejects.toThrow('hanging fixture did not exit within 0.1s.')
})
})

View File

@@ -0,0 +1,11 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": [
"src"
],
"references": []
}

View File

@@ -25,7 +25,7 @@
"@deepseek-ai/dsh-agent": "^0.0.1",
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-subagent": "^0.0.1",
"cordis": "^4.0.0-rc.6"
"cordis": "^4.0.0-rc.7"
},
"dependencies": {
"schemastery": "^3.18.0"
@@ -34,7 +34,7 @@
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-subagent": "workspace:^",
"@cordisjs/plugin-loader": "^1.0.0-rc.4",
"cordis": "^4.0.0-rc.6"
"@cordisjs/plugin-loader": "^1.0.0-rc.5",
"cordis": "^4.0.0-rc.7"
}
}