feat(spill): bound the durable copy of Code Mode sub-dispatch results
New tools/code-dispatch-log waterfall (run via registry.shapeDispatchLog, contained — a throwing listener falls back to the unshaped content) lets listeners reshape the tool/code-dispatch event's content before the bridge appends it. dsh-spill-policy registers a second arm sharing the model-facing arm's exact replacement pipeline (same maxInlineBytes cap, preview + locator, within-cap invariant, best-effort fallbacks), with artifacts labeled dispatch under the sub-call id. The program's value is untouched; read sub-calls ARE bounded (a log copy is not model context, and read produces the biggest logs). Resolves the tools README's uncapped-dispatch-log Known Limitation.
This commit is contained in:
@@ -13,7 +13,7 @@ This plugin registers **no service** and owns no storage or preview mechanics: p
|
||||
## Behavior
|
||||
|
||||
1. Let the tool run (delegates via `next()`, so it bounds whatever a downstream hook accepted).
|
||||
2. Skip nested executions (`exec.parent` is present), accepted value replacements (the registry must revalidate and rerender them), `read` (avoids a `read → spill → read again` loop), and any non-`accept` decision (a `block`'s corrective feedback passes through).
|
||||
2. Skip nested executions (`exec.parent` is present — their DURABLE copy is bounded by the dispatch-log arm below), accepted value replacements (the registry must revalidate and rerender them), `read` (avoids a `read → spill → read again` loop), and any non-`accept` decision (a `block`'s corrective feedback passes through).
|
||||
3. Flatten the accepted content only when it is **plain text** (all `text` blocks); a result with any non-text block is left untouched.
|
||||
4. If its UTF-8 size is `≤ maxInlineBytes`, leave it unchanged.
|
||||
5. Otherwise save the full text and replace the result with a preview + this notice, sized so the whole replacement (preview + blank line + notice) stays within `maxInlineBytes` — the notice's byte cost is reserved out of the budget, so the preview shrinks to fit and the model-facing result never exceeds the cap:
|
||||
@@ -28,6 +28,8 @@ This plugin registers **no service** and owns no storage or preview mechanics: p
|
||||
|
||||
**Best-effort:** no session owner, no `ctx.spillStore` backend, or a `saveText` rejection ⇒ the policy logs a warning and returns the original result. A spill failure never turns a successful call into an `isError` or hides the inline result. A successful replacement changes only `content`; the canonical programmatic value is preserved.
|
||||
|
||||
**The dispatch-log arm:** a second listener on `tools/code-dispatch-log` applies the same cap, replacement pipeline, and best-effort fallbacks to the DURABLE copy of each `run_code` sub-call result (artifact label `dispatch`, keyed by the sub-call id). The program's value is untouched — it already crossed the worker boundary whole — and `read` sub-calls are bounded too: a log copy is not model context, so the read-again loop cannot occur, and `read` is precisely the tool that produces huge logs ([rationale](../../../.agents/notes/implemented/feature/2026-07-26-code-dispatch-log-spill.md)).
|
||||
|
||||
## Scope
|
||||
|
||||
The policy sees only the FINAL formatted surface result—not a tool's internal resource or canonical value. If a provider already truncated (e.g. `web-fetch-local.maxBodyChars`), the spill artifact holds the full formatted result the tool returned, not the full original source. Provider/resource caps stay mandatory and separate. `glob`/`grep` own item-level surface spill because their complete acquired values still exist before rendering; bash streams own acquisition-time spill. The generic policy prepends its waterfall listener, then delegates, so ordinary tool-owned asynchronous projections complete before generic byte bounding regardless of plugin load order. See the [tool output spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md).
|
||||
|
||||
@@ -10,18 +10,26 @@
|
||||
* `@deepseek-ai/dsh-retention` (`TextRetainer`), storage is `ctx.spillStore`.
|
||||
* The policy only decides WHEN to spill and composes the notice.
|
||||
*
|
||||
* A second arm applies the SAME cap to the durable log: the
|
||||
* `tools/code-dispatch-log` waterfall bounds the `tool/code-dispatch` event's
|
||||
* copy of an oversized `run_code` sub-call result (the program's value is
|
||||
* untouched; UIs and replay read the full text through the spill artifact).
|
||||
*
|
||||
* ## Deliberately narrow
|
||||
*
|
||||
* - Omitted `maxInlineBytes` ⇒ the plugin registers nothing (a true no-op).
|
||||
* - Plain-text results only: a result carrying any non-text block is left
|
||||
* untouched (the policy knows only the final formatted text, not tool
|
||||
* internals).
|
||||
* - Nested composite calls are skipped; only their outer surface result may
|
||||
* become model-facing and spillable.
|
||||
* - Nested composite calls skip the MODEL-facing arm; their durable log copy
|
||||
* is bounded by the dispatch-log arm instead.
|
||||
* - Accepted value replacements pass through for registry revalidation and
|
||||
* rendering; this presentation policy cannot also replace content in the
|
||||
* same mutually exclusive decision.
|
||||
* - `read` is skipped to avoid a `read → spill → read again` loop.
|
||||
* - `read` is skipped by the model-facing arm to avoid a
|
||||
* `read → spill → read again` loop; the dispatch-log arm bounds `read`
|
||||
* sub-calls too (a log copy is not model context, and `read` is precisely
|
||||
* the tool that produces huge logs).
|
||||
* - Best-effort: no session owner, no `ctx.spillStore` backend, or a save
|
||||
* failure ⇒ log and return the original result. A spill failure must NEVER
|
||||
* turn a successful tool call into an `isError` or hide the inline result.
|
||||
@@ -42,6 +50,7 @@ import { TextRetainer, describeOmitted } from '@deepseek-ai/dsh-retention'
|
||||
import type { Omitted } from '@deepseek-ai/dsh-retention'
|
||||
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
|
||||
import type { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { CallId } from '@deepseek-ai/dsh-llm'
|
||||
import type { PostToolDecision, ToolExecution } from '@deepseek-ai/dsh-tools'
|
||||
import type { SpillPolicyExec } from './types.ts'
|
||||
|
||||
@@ -108,6 +117,75 @@ export function apply(ctx: Context, config: Config): void {
|
||||
if (!Number.isInteger(maxInlineBytes) || maxInlineBytes < 0) {
|
||||
throw new Error(`spill-policy: maxInlineBytes must be a non-negative integer (got ${maxInlineBytes})`)
|
||||
}
|
||||
// Narrowed once for the nested arms (closure narrowing does not survive awaits).
|
||||
const cap: number = maxInlineBytes
|
||||
|
||||
/**
|
||||
* Spill `text` and build the bounded replacement (preview + notice), or
|
||||
* return `undefined` when the policy must keep the original (no session
|
||||
* owner, no backend, storage failure, or no within-cap replacement).
|
||||
* Shared verbatim by the model-facing post-execute arm and the durable
|
||||
* dispatch-log arm so both produce byte-identical projections.
|
||||
*/
|
||||
async function spillReplacement(
|
||||
text: string,
|
||||
totalBytes: number,
|
||||
sessionId: SessionId | undefined,
|
||||
toolName: string,
|
||||
callId: CallId,
|
||||
label: 'result' | 'dispatch',
|
||||
): Promise<string | undefined> {
|
||||
if (sessionId === undefined) {
|
||||
ctx.logger.warn(`spill-policy: no session owner for ${toolName} ${label}; keeping the inline content`)
|
||||
return undefined
|
||||
}
|
||||
const spillStore = ctx.get('spillStore')
|
||||
if (!spillStore) {
|
||||
ctx.logger.warn('spill-policy: no ctx.spillStore backend loaded; keeping the inline content')
|
||||
return undefined
|
||||
}
|
||||
const save: SaveTextSpill = {
|
||||
owner: { sessionId },
|
||||
source: { toolName, callId, label },
|
||||
suggestedName: `${toolName}.txt`,
|
||||
content: text,
|
||||
}
|
||||
let ref: SpillRef
|
||||
try {
|
||||
ref = await spillStore.saveText(save)
|
||||
} catch (error: unknown) {
|
||||
// Best-effort: a storage failure (permissions, ENOSPC, backend down) must
|
||||
// never fail the call or hide the content — keep the original inline.
|
||||
ctx.logger.warn(`spill-policy: saveText failed for ${toolName}: ${String(error)}; keeping the inline content`)
|
||||
return undefined
|
||||
}
|
||||
|
||||
// Reserve the notice's byte cost INSIDE maxInlineBytes so the replacement
|
||||
// (preview + blank line + notice) never exceeds the documented cap — a naive
|
||||
// preview that spent the whole budget then appended the notice could be
|
||||
// larger than the cap, and for a marginally-over result even larger than the
|
||||
// original. The reservation uses a notice priced at the worst-case omission
|
||||
// count (the full byte total): its digit count bounds the real count's, so
|
||||
// the reserved size is a safe upper bound and the final notice is never
|
||||
// longer than what we reserved. `\n\n` is the 2-byte join.
|
||||
const reserve = Buffer.byteLength(spillNotice({ kind: 'exact', count: totalBytes }, ref), 'utf8') + 2
|
||||
const previewBudget = Math.max(0, cap - reserve)
|
||||
const { text: previewText, omitted } = preview(text, previewBudget)
|
||||
const notice = spillNotice(omitted, ref)
|
||||
const replacedText = previewText.length > 0 ? `${previewText}\n\n${notice}` : notice
|
||||
// Invariant: the policy NEVER emits a replacement larger than the cap. When
|
||||
// the notice alone exceeds maxInlineBytes (a tiny cap or a long spill root),
|
||||
// there is no within-cap replacement, so keep the inline content — spilling
|
||||
// would break the advertised cap. (A within-cap replacement is always
|
||||
// smaller than the original, which is > cap by the entry condition, so this
|
||||
// one check subsumes "not smaller than the original" too. The spill file
|
||||
// already written is a harmless orphan; cleanup is deferred.)
|
||||
if (Buffer.byteLength(replacedText, 'utf8') > cap) {
|
||||
ctx.logger.warn(`spill-policy: spill notice for ${toolName} exceeds maxInlineBytes; keeping the inline content`)
|
||||
return undefined
|
||||
}
|
||||
return replacedText
|
||||
}
|
||||
|
||||
ctx.on('tools/post-execute', async (exec, result, next): Promise<PostToolDecision> => {
|
||||
// Delegate first so a downstream listener (e.g. a hook) settles the result;
|
||||
@@ -124,58 +202,31 @@ export function apply(ctx: Context, config: Config): void {
|
||||
const totalBytes = Buffer.byteLength(text, 'utf8')
|
||||
if (totalBytes <= maxInlineBytes) return decision
|
||||
|
||||
const sessionId = ownerSessionId(exec)
|
||||
if (sessionId === undefined) {
|
||||
ctx.logger.warn(`spill-policy: no session owner for ${exec.name} result; keeping the inline result`)
|
||||
return decision
|
||||
}
|
||||
const spillStore = ctx.get('spillStore')
|
||||
if (!spillStore) {
|
||||
ctx.logger.warn('spill-policy: no ctx.spillStore backend loaded; keeping the inline result')
|
||||
return decision
|
||||
}
|
||||
|
||||
const save: SaveTextSpill = {
|
||||
owner: { sessionId },
|
||||
source: { toolName: exec.name, callId: exec.callId, label: 'result' },
|
||||
suggestedName: `${exec.name}.txt`,
|
||||
content: text,
|
||||
}
|
||||
let ref: SpillRef
|
||||
try {
|
||||
ref = await spillStore.saveText(save)
|
||||
} catch (error: unknown) {
|
||||
// Best-effort: a storage failure (permissions, ENOSPC, backend down) must
|
||||
// never fail the call or hide the result — keep the original inline.
|
||||
ctx.logger.warn(`spill-policy: saveText failed for ${exec.name}: ${String(error)}; keeping the inline result`)
|
||||
return decision
|
||||
}
|
||||
|
||||
// Reserve the notice's byte cost INSIDE maxInlineBytes so the replacement
|
||||
// (preview + blank line + notice) never exceeds the documented cap — a naive
|
||||
// preview that spent the whole budget then appended the notice could be
|
||||
// larger than the cap, and for a marginally-over result even larger than the
|
||||
// original. The reservation uses a notice priced at the worst-case omission
|
||||
// count (the full byte total): its digit count bounds the real count's, so
|
||||
// the reserved size is a safe upper bound and the final notice is never
|
||||
// longer than what we reserved. `\n\n` is the 2-byte join.
|
||||
const reserve = Buffer.byteLength(spillNotice({ kind: 'exact', count: totalBytes }, ref), 'utf8') + 2
|
||||
const previewBudget = Math.max(0, maxInlineBytes - reserve)
|
||||
const { text: previewText, omitted } = preview(text, previewBudget)
|
||||
const notice = spillNotice(omitted, ref)
|
||||
const replacedText = previewText.length > 0 ? `${previewText}\n\n${notice}` : notice
|
||||
// Invariant: the policy NEVER emits a replacement larger than the cap. When
|
||||
// the notice alone exceeds maxInlineBytes (a tiny cap or a long spill root),
|
||||
// there is no within-cap replacement, so keep the inline result — spilling
|
||||
// would break the advertised context cap. (A within-cap replacement is
|
||||
// always smaller than the original, which is > cap by the entry condition,
|
||||
// so this one check subsumes "not smaller than the original" too. The spill
|
||||
// file already written is a harmless orphan; cleanup is deferred.)
|
||||
if (Buffer.byteLength(replacedText, 'utf8') > maxInlineBytes) {
|
||||
ctx.logger.warn(`spill-policy: spill notice for ${exec.name} exceeds maxInlineBytes; keeping the inline result`)
|
||||
return decision
|
||||
}
|
||||
const replacedText = await spillReplacement(text, totalBytes, ownerSessionId(exec), exec.name, exec.callId, 'result')
|
||||
if (replacedText === undefined) return decision
|
||||
const replaced: ContentBlock[] = [{ type: 'text', text: replacedText }]
|
||||
return { kind: 'accept', content: replaced, ...decision.additionalContexts ? { additionalContexts: decision.additionalContexts } : {} }
|
||||
}, { prepend: true })
|
||||
|
||||
// The durable-log arm: bound the `tool/code-dispatch` event's copy of an
|
||||
// oversized sub-call result the same way the model-facing arm bounds an
|
||||
// outer result. The program's returned value is untouched (it already
|
||||
// crossed the worker boundary whole); only the session log's copy shrinks
|
||||
// to preview + locator, so replay and UIs read the full text through the
|
||||
// spill artifact exactly as they do for spilled native results.
|
||||
ctx.on('tools/code-dispatch-log', async (dispatch, next): Promise<ContentBlock[]> => {
|
||||
const content = await next()
|
||||
// `read` sub-calls spill too: the log copy is not model context, so the
|
||||
// read → spill → read-again loop the post-execute arm avoids cannot
|
||||
// happen here, and read is precisely the tool that produces huge logs.
|
||||
const text = flattenPlainText(content)
|
||||
if (text === undefined) return content
|
||||
const totalBytes = Buffer.byteLength(text, 'utf8')
|
||||
if (totalBytes <= maxInlineBytes) return content
|
||||
|
||||
const replacedText = await spillReplacement(
|
||||
text, totalBytes, ownerSessionId(dispatch.exec), dispatch.name, dispatch.subCallId, 'dispatch')
|
||||
if (replacedText === undefined) return content
|
||||
return [{ type: 'text', text: replacedText }]
|
||||
}, { prepend: true })
|
||||
}
|
||||
|
||||
@@ -230,6 +230,97 @@ describe('read skip', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('the durable dispatch-log arm', () => {
|
||||
/** Boot code mode + the policy + the worker runtime; run one program via the real bridge. */
|
||||
async function runCodeWith(program: string, maxInlineBytes: number) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry, { mode: 'code' })
|
||||
await ctx.plugin(StubStore)
|
||||
await ctx.plugin(SpillPolicy, { maxInlineBytes })
|
||||
await ctx.plugin(WorkerCodeRuntime, {})
|
||||
const events: { type: string; data: unknown }[] = []
|
||||
const agent = {
|
||||
session: {
|
||||
header: { id: SessionId('dispatch-spill'), cwd: '/workspace' },
|
||||
append: (type: string, data: unknown) => { events.push({ type, data }) },
|
||||
},
|
||||
}
|
||||
ctx.tools.register(textTool('huge_read', 'H'.repeat(2_000)))
|
||||
ctx.tools.register(textTool('small_read', 'tiny'))
|
||||
const result = await ctx.tools.execute({
|
||||
signal: testToolSignal,
|
||||
callId: CallId('parent-1'),
|
||||
name: 'run_code',
|
||||
arguments: { code: program, description: 'Drive dispatch-log spilling' },
|
||||
agent: agent as never,
|
||||
})
|
||||
return { ctx, result, events, spill: ctx.spillStore as StubStore }
|
||||
}
|
||||
|
||||
it('bounds the tool/code-dispatch copy of an oversized sub-result while the program value stays whole', async () => {
|
||||
const { result, events, spill } = await runCodeWith(
|
||||
'const blocks = await tools.huge_read({});\nreturn blocks[0].text.length', 200)
|
||||
expect(result.isError).toBe(false)
|
||||
if (result.isError) throw new Error('expected success')
|
||||
// The program received the COMPLETE text (length 2000), untouched by spill.
|
||||
expect(result.value).toMatchObject({ result: 2_000 })
|
||||
// The durable settle event carries the bounded projection + locator.
|
||||
const settle = events.find(event => event.type === 'tool/code-dispatch')
|
||||
expect(settle).toBeDefined()
|
||||
const logged = (settle!.data as { content: { type: string; text: string }[] }).content
|
||||
expect(logged).toHaveLength(1)
|
||||
const loggedText = logged[0]!.text
|
||||
expect(Buffer.byteLength(loggedText, 'utf8')).toBeLessThanOrEqual(200)
|
||||
expect(loggedText).toContain('Full formatted result stored at: /spill/huge_read.txt')
|
||||
// The artifact holds the full text under the dispatch label and sub-call id.
|
||||
const save = spill.saves.find(entry => entry.source.label === 'dispatch')
|
||||
expect(save).toMatchObject({
|
||||
source: { toolName: 'huge_read', callId: 'parent-1:code:1', label: 'dispatch' },
|
||||
})
|
||||
expect(save?.content).toBe('H'.repeat(2_000))
|
||||
})
|
||||
|
||||
it('leaves a within-cap sub-result log untouched and saves nothing for it', async () => {
|
||||
const { events, spill } = await runCodeWith(
|
||||
'return await tools.small_read({})', 200)
|
||||
const settle = events.find(event => event.type === 'tool/code-dispatch')
|
||||
expect((settle!.data as { content: { type: string; text: string }[] }).content)
|
||||
.toEqual([{ type: 'text', text: 'tiny' }])
|
||||
expect(spill.saves.filter(entry => entry.source.label === 'dispatch')).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('a saveText failure keeps the complete content in the durable log (best-effort)', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SystemPrompt)
|
||||
await ctx.plugin(ToolRegistry, { mode: 'code' })
|
||||
await ctx.plugin(StubStore)
|
||||
await ctx.plugin(SpillPolicy, { maxInlineBytes: 100 })
|
||||
await ctx.plugin(WorkerCodeRuntime, {})
|
||||
;(ctx.spillStore as StubStore).fail = true
|
||||
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
|
||||
const events: { type: string; data: unknown }[] = []
|
||||
const agent = {
|
||||
session: {
|
||||
header: { id: SessionId('dispatch-spill-fail'), cwd: '/workspace' },
|
||||
append: (type: string, data: unknown) => { events.push({ type, data }) },
|
||||
},
|
||||
}
|
||||
ctx.tools.register(textTool('huge_read', 'H'.repeat(2_000)))
|
||||
const result = await ctx.tools.execute({
|
||||
signal: testToolSignal,
|
||||
callId: CallId('parent-2'),
|
||||
name: 'run_code',
|
||||
arguments: { code: 'return (await tools.huge_read({}))[0].text.length', description: 'Fail the spill backend' },
|
||||
agent: agent as never,
|
||||
})
|
||||
expect(result.isError).toBe(false)
|
||||
const settle = events.find(event => event.type === 'tool/code-dispatch')
|
||||
expect((settle!.data as { content: { text: string }[] }).content[0]!.text).toBe('H'.repeat(2_000))
|
||||
expect(warn).toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('nested-call skip', () => {
|
||||
it('leaves nested composite results complete and spillable only through their outer call', async () => {
|
||||
const { ctx, spill } = await setup({ maxInlineBytes: 10 })
|
||||
|
||||
Reference in New Issue
Block a user