feat(spill): bound the durable copy of Code Mode sub-dispatch results

New tools/code-dispatch-log waterfall (run via registry.shapeDispatchLog,
contained — a throwing listener falls back to the unshaped content) lets
listeners reshape the tool/code-dispatch event's content before the
bridge appends it. dsh-spill-policy registers a second arm sharing the
model-facing arm's exact replacement pipeline (same maxInlineBytes cap,
preview + locator, within-cap invariant, best-effort fallbacks), with
artifacts labeled dispatch under the sub-call id. The program's value is
untouched; read sub-calls ARE bounded (a log copy is not model context,
and read produces the biggest logs). Resolves the tools README's
uncapped-dispatch-log Known Limitation.
This commit is contained in:
Tianyi Cui
2026-07-26 09:01:03 +08:00
parent 366419862f
commit 4987261d55
16 changed files with 415 additions and 75 deletions

View File

@@ -13,7 +13,7 @@ This plugin registers **no service** and owns no storage or preview mechanics: p
## Behavior
1. Let the tool run (delegates via `next()`, so it bounds whatever a downstream hook accepted).
2. Skip nested executions (`exec.parent` is present), accepted value replacements (the registry must revalidate and rerender them), `read` (avoids a `read → spill → read again` loop), and any non-`accept` decision (a `block`'s corrective feedback passes through).
2. Skip nested executions (`exec.parent` is present — their DURABLE copy is bounded by the dispatch-log arm below), accepted value replacements (the registry must revalidate and rerender them), `read` (avoids a `read → spill → read again` loop), and any non-`accept` decision (a `block`'s corrective feedback passes through).
3. Flatten the accepted content only when it is **plain text** (all `text` blocks); a result with any non-text block is left untouched.
4. If its UTF-8 size is `≤ maxInlineBytes`, leave it unchanged.
5. Otherwise save the full text and replace the result with a preview + this notice, sized so the whole replacement (preview + blank line + notice) stays within `maxInlineBytes` — the notice's byte cost is reserved out of the budget, so the preview shrinks to fit and the model-facing result never exceeds the cap:
@@ -28,6 +28,8 @@ This plugin registers **no service** and owns no storage or preview mechanics: p
**Best-effort:** no session owner, no `ctx.spillStore` backend, or a `saveText` rejection ⇒ the policy logs a warning and returns the original result. A spill failure never turns a successful call into an `isError` or hides the inline result. A successful replacement changes only `content`; the canonical programmatic value is preserved.
**The dispatch-log arm:** a second listener on `tools/code-dispatch-log` applies the same cap, replacement pipeline, and best-effort fallbacks to the DURABLE copy of each `run_code` sub-call result (artifact label `dispatch`, keyed by the sub-call id). The program's value is untouched — it already crossed the worker boundary whole — and `read` sub-calls are bounded too: a log copy is not model context, so the read-again loop cannot occur, and `read` is precisely the tool that produces huge logs ([rationale](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md)).
## Scope
The policy sees only the FINAL formatted surface result—not a tool's internal resource or canonical value. If a provider already truncated (e.g. `web-fetch-local.maxBodyChars`), the spill artifact holds the full formatted result the tool returned, not the full original source. Provider/resource caps stay mandatory and separate. `glob`/`grep` own item-level surface spill because their complete acquired values still exist before rendering; bash streams own acquisition-time spill. The generic policy prepends its waterfall listener, then delegates, so ordinary tool-owned asynchronous projections complete before generic byte bounding regardless of plugin load order. See the [tool output spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md).

View File

@@ -10,18 +10,26 @@
* `@deepseek-ai/dsh-retention` (`TextRetainer`), storage is `ctx.spillStore`.
* The policy only decides WHEN to spill and composes the notice.
*
* A second arm applies the SAME cap to the durable log: the
* `tools/code-dispatch-log` waterfall bounds the `tool/code-dispatch` event's
* copy of an oversized `run_code` sub-call result (the program's value is
* untouched; UIs and replay read the full text through the spill artifact).
*
* ## Deliberately narrow
*
* - Omitted `maxInlineBytes` ⇒ the plugin registers nothing (a true no-op).
* - Plain-text results only: a result carrying any non-text block is left
* untouched (the policy knows only the final formatted text, not tool
* internals).
* - Nested composite calls are skipped; only their outer surface result may
* become model-facing and spillable.
* - Nested composite calls skip the MODEL-facing arm; their durable log copy
* is bounded by the dispatch-log arm instead.
* - Accepted value replacements pass through for registry revalidation and
* rendering; this presentation policy cannot also replace content in the
* same mutually exclusive decision.
* - `read` is skipped to avoid a `read → spill → read again` loop.
* - `read` is skipped by the model-facing arm to avoid a
* `read → spill → read again` loop; the dispatch-log arm bounds `read`
* sub-calls too (a log copy is not model context, and `read` is precisely
* the tool that produces huge logs).
* - Best-effort: no session owner, no `ctx.spillStore` backend, or a save
* failure ⇒ log and return the original result. A spill failure must NEVER
* turn a successful tool call into an `isError` or hide the inline result.
@@ -42,6 +50,7 @@ import { TextRetainer, describeOmitted } from '@deepseek-ai/dsh-retention'
import type { Omitted } from '@deepseek-ai/dsh-retention'
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
import type { SessionId } from '@deepseek-ai/dsh-session'
import type { CallId } from '@deepseek-ai/dsh-llm'
import type { PostToolDecision, ToolExecution } from '@deepseek-ai/dsh-tools'
import type { SpillPolicyExec } from './types.ts'
@@ -108,6 +117,75 @@ export function apply(ctx: Context, config: Config): void {
if (!Number.isInteger(maxInlineBytes) || maxInlineBytes < 0) {
throw new Error(`spill-policy: maxInlineBytes must be a non-negative integer (got ${maxInlineBytes})`)
}
// Narrowed once for the nested arms (closure narrowing does not survive awaits).
const cap: number = maxInlineBytes
/**
* Spill `text` and build the bounded replacement (preview + notice), or
* return `undefined` when the policy must keep the original (no session
* owner, no backend, storage failure, or no within-cap replacement).
* Shared verbatim by the model-facing post-execute arm and the durable
* dispatch-log arm so both produce byte-identical projections.
*/
async function spillReplacement(
text: string,
totalBytes: number,
sessionId: SessionId | undefined,
toolName: string,
callId: CallId,
label: 'result' | 'dispatch',
): Promise<string | undefined> {
if (sessionId === undefined) {
ctx.logger.warn(`spill-policy: no session owner for ${toolName} ${label}; keeping the inline content`)
return undefined
}
const spillStore = ctx.get('spillStore')
if (!spillStore) {
ctx.logger.warn('spill-policy: no ctx.spillStore backend loaded; keeping the inline content')
return undefined
}
const save: SaveTextSpill = {
owner: { sessionId },
source: { toolName, callId, label },
suggestedName: `${toolName}.txt`,
content: text,
}
let ref: SpillRef
try {
ref = await spillStore.saveText(save)
} catch (error: unknown) {
// Best-effort: a storage failure (permissions, ENOSPC, backend down) must
// never fail the call or hide the content — keep the original inline.
ctx.logger.warn(`spill-policy: saveText failed for ${toolName}: ${String(error)}; keeping the inline content`)
return undefined
}
// Reserve the notice's byte cost INSIDE maxInlineBytes so the replacement
// (preview + blank line + notice) never exceeds the documented cap — a naive
// preview that spent the whole budget then appended the notice could be
// larger than the cap, and for a marginally-over result even larger than the
// original. The reservation uses a notice priced at the worst-case omission
// count (the full byte total): its digit count bounds the real count's, so
// the reserved size is a safe upper bound and the final notice is never
// longer than what we reserved. `\n\n` is the 2-byte join.
const reserve = Buffer.byteLength(spillNotice({ kind: 'exact', count: totalBytes }, ref), 'utf8') + 2
const previewBudget = Math.max(0, cap - reserve)
const { text: previewText, omitted } = preview(text, previewBudget)
const notice = spillNotice(omitted, ref)
const replacedText = previewText.length > 0 ? `${previewText}\n\n${notice}` : notice
// Invariant: the policy NEVER emits a replacement larger than the cap. When
// the notice alone exceeds maxInlineBytes (a tiny cap or a long spill root),
// there is no within-cap replacement, so keep the inline content — spilling
// would break the advertised cap. (A within-cap replacement is always
// smaller than the original, which is > cap by the entry condition, so this
// one check subsumes "not smaller than the original" too. The spill file
// already written is a harmless orphan; cleanup is deferred.)
if (Buffer.byteLength(replacedText, 'utf8') > cap) {
ctx.logger.warn(`spill-policy: spill notice for ${toolName} exceeds maxInlineBytes; keeping the inline content`)
return undefined
}
return replacedText
}
ctx.on('tools/post-execute', async (exec, result, next): Promise<PostToolDecision> => {
// Delegate first so a downstream listener (e.g. a hook) settles the result;
@@ -124,58 +202,31 @@ export function apply(ctx: Context, config: Config): void {
const totalBytes = Buffer.byteLength(text, 'utf8')
if (totalBytes <= maxInlineBytes) return decision
const sessionId = ownerSessionId(exec)
if (sessionId === undefined) {
ctx.logger.warn(`spill-policy: no session owner for ${exec.name} result; keeping the inline result`)
return decision
}
const spillStore = ctx.get('spillStore')
if (!spillStore) {
ctx.logger.warn('spill-policy: no ctx.spillStore backend loaded; keeping the inline result')
return decision
}
const save: SaveTextSpill = {
owner: { sessionId },
source: { toolName: exec.name, callId: exec.callId, label: 'result' },
suggestedName: `${exec.name}.txt`,
content: text,
}
let ref: SpillRef
try {
ref = await spillStore.saveText(save)
} catch (error: unknown) {
// Best-effort: a storage failure (permissions, ENOSPC, backend down) must
// never fail the call or hide the result — keep the original inline.
ctx.logger.warn(`spill-policy: saveText failed for ${exec.name}: ${String(error)}; keeping the inline result`)
return decision
}
// Reserve the notice's byte cost INSIDE maxInlineBytes so the replacement
// (preview + blank line + notice) never exceeds the documented cap — a naive
// preview that spent the whole budget then appended the notice could be
// larger than the cap, and for a marginally-over result even larger than the
// original. The reservation uses a notice priced at the worst-case omission
// count (the full byte total): its digit count bounds the real count's, so
// the reserved size is a safe upper bound and the final notice is never
// longer than what we reserved. `\n\n` is the 2-byte join.
const reserve = Buffer.byteLength(spillNotice({ kind: 'exact', count: totalBytes }, ref), 'utf8') + 2
const previewBudget = Math.max(0, maxInlineBytes - reserve)
const { text: previewText, omitted } = preview(text, previewBudget)
const notice = spillNotice(omitted, ref)
const replacedText = previewText.length > 0 ? `${previewText}\n\n${notice}` : notice
// Invariant: the policy NEVER emits a replacement larger than the cap. When
// the notice alone exceeds maxInlineBytes (a tiny cap or a long spill root),
// there is no within-cap replacement, so keep the inline result — spilling
// would break the advertised context cap. (A within-cap replacement is
// always smaller than the original, which is > cap by the entry condition,
// so this one check subsumes "not smaller than the original" too. The spill
// file already written is a harmless orphan; cleanup is deferred.)
if (Buffer.byteLength(replacedText, 'utf8') > maxInlineBytes) {
ctx.logger.warn(`spill-policy: spill notice for ${exec.name} exceeds maxInlineBytes; keeping the inline result`)
return decision
}
const replacedText = await spillReplacement(text, totalBytes, ownerSessionId(exec), exec.name, exec.callId, 'result')
if (replacedText === undefined) return decision
const replaced: ContentBlock[] = [{ type: 'text', text: replacedText }]
return { kind: 'accept', content: replaced, ...decision.additionalContexts ? { additionalContexts: decision.additionalContexts } : {} }
}, { prepend: true })
// The durable-log arm: bound the `tool/code-dispatch` event's copy of an
// oversized sub-call result the same way the model-facing arm bounds an
// outer result. The program's returned value is untouched (it already
// crossed the worker boundary whole); only the session log's copy shrinks
// to preview + locator, so replay and UIs read the full text through the
// spill artifact exactly as they do for spilled native results.
ctx.on('tools/code-dispatch-log', async (dispatch, next): Promise<ContentBlock[]> => {
const content = await next()
// `read` sub-calls spill too: the log copy is not model context, so the
// read → spill → read-again loop the post-execute arm avoids cannot
// happen here, and read is precisely the tool that produces huge logs.
const text = flattenPlainText(content)
if (text === undefined) return content
const totalBytes = Buffer.byteLength(text, 'utf8')
if (totalBytes <= maxInlineBytes) return content
const replacedText = await spillReplacement(
text, totalBytes, ownerSessionId(dispatch.exec), dispatch.name, dispatch.subCallId, 'dispatch')
if (replacedText === undefined) return content
return [{ type: 'text', text: replacedText }]
}, { prepend: true })
}

View File

@@ -230,6 +230,97 @@ describe('read skip', () => {
})
})
describe('the durable dispatch-log arm', () => {
/** Boot code mode + the policy + the worker runtime; run one program via the real bridge. */
async function runCodeWith(program: string, maxInlineBytes: number) {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry, { mode: 'code' })
await ctx.plugin(StubStore)
await ctx.plugin(SpillPolicy, { maxInlineBytes })
await ctx.plugin(WorkerCodeRuntime, {})
const events: { type: string; data: unknown }[] = []
const agent = {
session: {
header: { id: SessionId('dispatch-spill'), cwd: '/workspace' },
append: (type: string, data: unknown) => { events.push({ type, data }) },
},
}
ctx.tools.register(textTool('huge_read', 'H'.repeat(2_000)))
ctx.tools.register(textTool('small_read', 'tiny'))
const result = await ctx.tools.execute({
signal: testToolSignal,
callId: CallId('parent-1'),
name: 'run_code',
arguments: { code: program, description: 'Drive dispatch-log spilling' },
agent: agent as never,
})
return { ctx, result, events, spill: ctx.spillStore as StubStore }
}
it('bounds the tool/code-dispatch copy of an oversized sub-result while the program value stays whole', async () => {
const { result, events, spill } = await runCodeWith(
'const blocks = await tools.huge_read({});\nreturn blocks[0].text.length', 200)
expect(result.isError).toBe(false)
if (result.isError) throw new Error('expected success')
// The program received the COMPLETE text (length 2000), untouched by spill.
expect(result.value).toMatchObject({ result: 2_000 })
// The durable settle event carries the bounded projection + locator.
const settle = events.find(event => event.type === 'tool/code-dispatch')
expect(settle).toBeDefined()
const logged = (settle!.data as { content: { type: string; text: string }[] }).content
expect(logged).toHaveLength(1)
const loggedText = logged[0]!.text
expect(Buffer.byteLength(loggedText, 'utf8')).toBeLessThanOrEqual(200)
expect(loggedText).toContain('Full formatted result stored at: /spill/huge_read.txt')
// The artifact holds the full text under the dispatch label and sub-call id.
const save = spill.saves.find(entry => entry.source.label === 'dispatch')
expect(save).toMatchObject({
source: { toolName: 'huge_read', callId: 'parent-1:code:1', label: 'dispatch' },
})
expect(save?.content).toBe('H'.repeat(2_000))
})
it('leaves a within-cap sub-result log untouched and saves nothing for it', async () => {
const { events, spill } = await runCodeWith(
'return await tools.small_read({})', 200)
const settle = events.find(event => event.type === 'tool/code-dispatch')
expect((settle!.data as { content: { type: string; text: string }[] }).content)
.toEqual([{ type: 'text', text: 'tiny' }])
expect(spill.saves.filter(entry => entry.source.label === 'dispatch')).toHaveLength(0)
})
it('a saveText failure keeps the complete content in the durable log (best-effort)', async () => {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry, { mode: 'code' })
await ctx.plugin(StubStore)
await ctx.plugin(SpillPolicy, { maxInlineBytes: 100 })
await ctx.plugin(WorkerCodeRuntime, {})
;(ctx.spillStore as StubStore).fail = true
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
const events: { type: string; data: unknown }[] = []
const agent = {
session: {
header: { id: SessionId('dispatch-spill-fail'), cwd: '/workspace' },
append: (type: string, data: unknown) => { events.push({ type, data }) },
},
}
ctx.tools.register(textTool('huge_read', 'H'.repeat(2_000)))
const result = await ctx.tools.execute({
signal: testToolSignal,
callId: CallId('parent-2'),
name: 'run_code',
arguments: { code: 'return (await tools.huge_read({}))[0].text.length', description: 'Fail the spill backend' },
agent: agent as never,
})
expect(result.isError).toBe(false)
const settle = events.find(event => event.type === 'tool/code-dispatch')
expect((settle!.data as { content: { text: string }[] }).content[0]!.text).toBe('H'.repeat(2_000))
expect(warn).toHaveBeenCalled()
})
})
describe('nested-call skip', () => {
it('leaves nested composite results complete and spillable only through their outer call', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 10 })