feat(spill): add tool-output spill seam, local backend, and policy

Oversized plain-text tool results now spill to a session-scoped file and
return a bounded preview plus the spill path, so a verbose result stays
readable via `read` without consuming the next model request in full.

- dsh-spill: minimal SpillFiles seam (saveText → session-scoped SpillPath)
- dsh-spill-local: private 0700 session dirs, traversal-safe names, exclusive
  owner-only writes
- dsh-spill-policy: tools/post-execute transformer; no-op unless maxInlineBytes
  is set; skips read; best-effort on save failure (never turns a success into
  an isError)

web_fetch is the showcase — no tool-specific spill code. The coding-agent
example loads the stack so its keyless Loader smoke guards the namespace-plugin
export shape. Snapshot gap for a transcript-visible web_fetch spill is recorded
in the RFC's Consequences (ACP replay is keyless and cannot hit the web).
This commit is contained in:
Dudu-0223
2026-07-08 19:20:50 +08:00
parent 4f2f34c6fd
commit 463b72ce96
36 changed files with 1549 additions and 1 deletions

13
packages/spill/README.md Normal file
View File

@@ -0,0 +1,13 @@
# spill/ - spill storage capability family
The tool-output spill capability seam: an abstract storage interface, a local filesystem implementation, and the tool-result policy that uses it. All **product** packages.
| Package | Role | ctx key |
|---|---|---|
| `spill/` | Abstract spill storage seam (`saveText` — persist oversized tool text to a session-scoped path) | `ctx.spillFiles` |
| `spill-local/` | Local-filesystem backend: private, session-scoped files with traversal-safe names | (registers on `ctx.spillFiles`) |
| `spill-policy/` | `tools/post-execute` policy: replaces oversized plain-text results with a preview + spill path | (no service surface) |
The interface lives at `spill/spill/`. The split mirrors bash/fs: the seam owns storage only, `spill-local` owns the filesystem mechanics, and `spill-policy` owns WHEN to spill and the model-facing notice. Preview mechanics stay in [`util/retention`](../util/README.md) — the policy composes the two without either owning the other's job.
See the [tool output spill RFC](../../docs/rfc/implemented/architecture/2026-07-08-tool-output-spill-files.md) for the design rationale, including why final-result spill is separate from tool-owned early spill (bash streams, subagent rollouts) and why creation belongs to the runtime spill seam rather than the model-facing `write` tool.

View File

@@ -0,0 +1,19 @@
# @deepseek-ai/dsh-spill-local
The **local-filesystem** implementation of the [`@deepseek-ai/dsh-spill`](../spill) storage seam. Registers as `ctx.spillFiles` and persists a tool's oversized text to a private, session-scoped file the model's `read` tool can open.
## Storage layout
Files land at `<root>/session-<hash>/<random>-<safeName>`:
- **`root`** — the config `root` (resolved to absolute), or a lazily-created private (0700) per-process directory under the OS temp dir when omitted. A predictable, world-readable root would let other local users read spilled tool output or plant symlinks.
- **`session-<hash>`** — a short `sha256(sessionId)` prefix, so a session's spill files group together and a future cleanup can drop them per session.
- **`<random>-<safeName>`** — an unpredictable hex prefix (defeats symlink planting in a shared root) plus the caller's `suggestedName` sanitized to one safe path segment (traversal-proof; mirrors the JSONL persistence backend's `encodeSegment`). The write is exclusive + owner-only (`open(path, 'wx', 0o600)`): it fails on any pre-existing path, symlink or not, so a planted target cannot redirect it.
## Config
| Key | Default | Meaning |
|---|---|---|
| `root` | private 0700 temp dir | Root directory for spill files. Set to keep them under a known location. |
`saveText` rejects on a real storage failure (permissions, ENOSPC); the spill policy treats a rejection as best-effort and keeps the inline result. See the seam README for the vocabulary and the [tool output spill RFC](../../../docs/rfc/implemented/architecture/2026-07-08-tool-output-spill-files.md) for the design.

View File

@@ -0,0 +1,38 @@
{
"name": "@deepseek-ai/dsh-spill-local",
"description": "Local-filesystem implementation of the DeepSeek Harness spill storage seam (private session-scoped files)",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-spill": "^0.0.1",
"cordis": "^4.0.0-rc.6"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-brand": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-spill": "workspace:^",
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,61 @@
/**
* `LocalSpillFiles`: the host-filesystem implementation of the
* `@deepseek-ai/dsh-spill` storage seam. Persists a tool's oversized text to a
* private, session-scoped file (see `./store.ts` for the traversal-safe naming
* and exclusive owner-only write) and returns a path the local `read` tool can
* open.
*
* @module @deepseek-ai/dsh-spill-local
*/
import { Context } from 'cordis'
import { resolve } from 'node:path'
import z from 'schemastery'
import { SpillFiles, SpillPath } from '@deepseek-ai/dsh-spill'
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
import { privateRoot, saveTextFile } from './store.ts'
export { encodeSegment, privateRoot, saveTextFile, sessionDir } from './store.ts'
export type { SavedText, SaveTextOptions } from './store.ts'
/** Plugin config (all optional — `static Config` supplies the defaults). */
export interface Config {
/**
* Root directory for spill files. Omitted uses a lazily-created private
* (0700) per-process directory under the OS temp dir — the safe default for
* a local deployment. Set it to keep spill files under a known location.
*/
root?: string
}
/**
* Local-filesystem spill backend. Files land under `<root>/session-<hash>/…`
* with unpredictable names, an exclusive owner-only (0600) write, and a private
* (0700) root — a spilled tool result must not be readable by other local users
* or redirectable via a planted symlink.
*/
export class LocalSpillFiles extends SpillFiles {
static Config: z<Config> = z.object({
root: z.string(),
})
/** Resolved absolute spill root (config `root`, else the private default), fixed at construction. */
readonly root: string
constructor(ctx: Context, config: Config) {
super(ctx)
this.root = config.root !== undefined ? resolve(config.root) : privateRoot()
}
async saveText(input: SaveTextSpill): Promise<SpillRef> {
const saved = await saveTextFile({
root: this.root,
sessionId: input.owner.sessionId,
suggestedName: input.suggestedName,
content: input.content,
})
return { path: SpillPath(saved.path), bytes: saved.bytes }
}
}
export default LocalSpillFiles

View File

@@ -0,0 +1,102 @@
/**
* Cordis-free storage mechanics for the local spill backend: private
* session-scoped directory selection, safe-name derivation, path-traversal
* protection, and the exclusive owner-only write. Kept out of the service class
* (like `dsh-bash-local`'s `run.ts`) so the filesystem behavior is unit-testable
* without a `ctx` and without the OS temp dir.
*
* @module @deepseek-ai/dsh-spill-local/store
*/
import { createHash, randomBytes } from 'node:crypto'
import { mkdtempSync } from 'node:fs'
import { mkdir, open } from 'node:fs/promises'
import { join } from 'node:path'
import { tmpdir } from 'node:os'
let defaultRoot: string | undefined
/**
* The default spill root: a private (0700) per-process directory under the OS
* tmpdir, created lazily. Predictable world-readable paths would let other
* local users read spilled tool output or pre-create symlinks; `mkdtemp` gives
* an unpredictable suffix and 0700 semantics.
*/
export function privateRoot(): string {
defaultRoot ??= mkdtempSync(join(tmpdir(), 'dsh-spill-'))
return defaultRoot
}
/**
* Encode an arbitrary string as one safe path segment, injectively over ALL JS
* (UTF-16) strings. A session id / suggested name is untrusted input, so this
* neutralizes `../`, absolute paths, NUL, and separators before any filesystem
* use. Each code unit is kept literal (`[A-Za-z0-9._-]`, minus `~`) or escaped
* as `~XXXX`; `~` is itself escaped, so the mapping is reversible and distinct
* inputs never collide. The whole-segment tokens `.`/`..` are escaped so they
* can never traverse. An empty string encodes to `~` (never an empty segment).
* (Mirrors the JSONL persistence backend's `encodeSegment`.)
*/
export function encodeSegment(raw: string): string {
if (raw.length === 0) return '~'
if (raw === '.') return '~002E'
if (raw === '..') return '~002E~002E'
let out = ''
for (let i = 0; i < raw.length; i++) {
const code = raw.charCodeAt(i)
const ch = String.fromCharCode(code)
if (ch !== '~' && /^[A-Za-z0-9._-]$/.test(ch)) {
out += ch
} else {
out += '~' + code.toString(16).toUpperCase().padStart(4, '0')
}
}
return out
}
/** The session-scoped directory: `<root>/session-<hash(sessionId)>`, a short stable hash. */
export function sessionDir(root: string, sessionId: string): string {
const hash = createHash('sha256').update(sessionId).digest('hex').slice(0, 12)
return join(root, `session-${hash}`)
}
/** Options for {@link saveTextFile} — the resolved root and the request fields the store needs. */
export interface SaveTextOptions {
/** The spill root directory (configured or the lazy private default). */
root: string
/** The owning session id (scopes the directory). */
sessionId: string
/** Caller-suggested base name; sanitized to one safe segment before use. */
suggestedName: string
/** The full text to persist. */
content: string
}
/** A written spill file. */
export interface SavedText {
path: string
bytes: number
}
/**
* Write `content` to a fresh file under the session-scoped directory and return
* its path + byte length. The filename is a random hex prefix plus the
* sanitized `suggestedName`, so it is unpredictable (defeats symlink planting in
* a shared root) AND stays readable. The open is exclusive + owner-only
* (`'wx', 0o600`): it fails on any existing path — symlink or not — so a
* pre-planted target cannot redirect the write.
*/
export async function saveTextFile(options: SaveTextOptions): Promise<SavedText> {
const dir = sessionDir(options.root, options.sessionId)
await mkdir(dir, { recursive: true, mode: 0o700 })
const safeName = encodeSegment(options.suggestedName)
const path = join(dir, `${randomBytes(6).toString('hex')}-${safeName}`)
const bytes = Buffer.byteLength(options.content, 'utf8')
const handle = await open(path, 'wx', 0o600)
try {
await handle.writeFile(options.content)
} finally {
await handle.close()
}
return { path, bytes }
}

View File

@@ -0,0 +1,138 @@
/**
* Tests for the LOCAL spill backend: `saveText` writes a session-scoped file and
* returns its path + byte length, filename sanitization neutralizes traversal,
* the configured `root` is honored (and the private default when omitted), and a
* storage failure rejects. The Cordis-free `store.ts` helpers are exercised
* directly for the naming/encoding edge cases.
*/
import { describe, expect, it, beforeEach, afterEach } from 'vitest'
import { Context } from 'cordis'
import { mkdtempSync, readFileSync, rmSync, statSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, isAbsolute, join } from 'node:path'
import { CallId } from '@deepseek-ai/dsh-llm'
import { SessionId } from '@deepseek-ai/dsh-session'
import type { SaveTextSpill } from '@deepseek-ai/dsh-spill'
import LocalSpillFiles, { encodeSegment, privateRoot, saveTextFile, sessionDir } from '@deepseek-ai/dsh-spill-local'
let root: string
beforeEach(() => {
root = mkdtempSync(join(tmpdir(), 'dsh-spill-test-'))
})
afterEach(() => {
rmSync(root, { recursive: true, force: true })
})
function request(overrides: Partial<SaveTextSpill> = {}): SaveTextSpill {
return {
owner: { sessionId: SessionId('sess-1') },
source: { toolName: 'web_fetch', callId: CallId('call-1'), label: 'result' },
suggestedName: 'web_fetch.txt',
content: 'the full body',
...overrides,
}
}
describe('encodeSegment', () => {
it('keeps the safe set literal', () => {
expect(encodeSegment('web_fetch.txt')).toBe('web_fetch.txt')
expect(encodeSegment('a-B_9.z')).toBe('a-B_9.z')
})
it('escapes separators and tilde (dots are literal except as whole-segment tokens)', () => {
// `.` is in the safe set, so `..` inside a longer string stays literal; the
// traversal defense is that separators escape, keeping the result ONE segment.
expect(encodeSegment('../etc/passwd')).toBe('..~002Fetc~002Fpasswd')
expect(encodeSegment('a/b')).toBe('a~002Fb')
expect(encodeSegment('~')).toBe('~007E')
})
it('escapes the whole-segment dot tokens', () => {
expect(encodeSegment('.')).toBe('~002E')
expect(encodeSegment('..')).toBe('~002E~002E')
})
it('encodes the empty string to a non-empty segment', () => {
expect(encodeSegment('')).toBe('~')
})
})
describe('sessionDir', () => {
it('is a stable per-session hash under the root', () => {
const dir = sessionDir('/spill', 'sess-1')
expect(dir).toBe(sessionDir('/spill', 'sess-1'))
expect(dir).toMatch(/\/spill\/session-[0-9a-f]{12}$/)
expect(sessionDir('/spill', 'sess-2')).not.toBe(dir)
})
})
describe('saveTextFile', () => {
it('writes the content under the session dir and reports bytes', async () => {
const saved = await saveTextFile({ root, sessionId: 'sess-1', suggestedName: 'r.txt', content: 'héllo' })
expect(readFileSync(saved.path, 'utf8')).toBe('héllo')
expect(saved.bytes).toBe(Buffer.byteLength('héllo', 'utf8'))
expect(dirname(saved.path)).toBe(sessionDir(root, 'sess-1'))
expect(saved.path).toMatch(/\/[0-9a-f]{12}-r\.txt$/)
})
it('sanitizes a traversal-shaped suggested name into one segment', async () => {
const saved = await saveTextFile({ root, sessionId: 'sess-1', suggestedName: '../../evil', content: 'x' })
// The separators escaped, so the whole name is one leaf under the session dir.
expect(dirname(saved.path)).toBe(sessionDir(root, 'sess-1'))
expect(saved.path.includes('/..')).toBe(false)
})
it('creates the session dir with owner-only permissions', async () => {
const saved = await saveTextFile({ root, sessionId: 'sess-1', suggestedName: 'r.txt', content: 'x' })
// 0o700 dir, 0o600 file (masked by umask, but the owner bits must hold).
expect(statSync(dirname(saved.path)).mode & 0o700).toBe(0o700)
expect(statSync(saved.path).mode & 0o600).toBe(0o600)
})
it('gives distinct paths to two saves of the same name', async () => {
const a = await saveTextFile({ root, sessionId: 'sess-1', suggestedName: 'r.txt', content: 'a' })
const b = await saveTextFile({ root, sessionId: 'sess-1', suggestedName: 'r.txt', content: 'b' })
expect(a.path).not.toBe(b.path)
})
})
describe('privateRoot', () => {
it('is a stable absolute directory under the temp dir', () => {
const first = privateRoot()
expect(isAbsolute(first)).toBe(true)
expect(privateRoot()).toBe(first)
})
})
describe('LocalSpillFiles service', () => {
it('registers as ctx.spillFiles and saves under the configured root', async () => {
const ctx = new Context()
await ctx.plugin(LocalSpillFiles, { root })
const ref = await ctx.spillFiles.saveText(request())
expect(dirname(ref.path)).toBe(sessionDir(root, 'sess-1'))
expect(readFileSync(ref.path, 'utf8')).toBe('the full body')
expect(ref.bytes).toBe(Buffer.byteLength('the full body', 'utf8'))
})
it('resolves a relative configured root to absolute', async () => {
const ctx = new Context()
await ctx.plugin(LocalSpillFiles, { root: '.' })
expect(isAbsolute((ctx.spillFiles as LocalSpillFiles).root)).toBe(true)
})
it('falls back to the private root when none is configured', async () => {
const ctx = new Context()
await ctx.plugin(LocalSpillFiles, {})
expect((ctx.spillFiles as LocalSpillFiles).root).toBe(privateRoot())
})
it('rejects when the root is not writable (missing parent, exclusive open)', async () => {
const ctx = new Context()
// A file (not a dir) as the root makes mkdir under it fail — a real storage error.
const filePath = (await saveTextFile({ root, sessionId: 's', suggestedName: 'f', content: 'x' })).path
await ctx.plugin(LocalSpillFiles, { root: filePath })
await expect(ctx.spillFiles.saveText(request())).rejects.toThrow()
})
})

View File

@@ -0,0 +1,14 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": ["src"],
"references": [
{ "path": "../../../vendor/cosmokit" },
{ "path": "../../../vendor/cordis" },
{ "path": "../../../vendor/schemastery" },
{ "path": "../spill" }
]
}

View File

@@ -0,0 +1,31 @@
# @deepseek-ai/dsh-spill-policy
The **tool-result spill policy**: a `tools/post-execute` transformer that keeps oversized plain-text tool results out of the model's context. When a final result exceeds `maxInlineBytes`, it saves the FULL text to a session-scoped spill file via [`ctx.spillFiles`](../spill) and replaces the model-facing result with a bounded head/tail preview plus the spill path — the model reads the complete result later with the existing `read` tool.
This plugin registers **no service** and owns no storage or preview mechanics: preview is [`@deepseek-ai/dsh-retention`](../../util/retention) (`TextRetainer`), storage is `ctx.spillFiles`. It only decides WHEN to spill and composes the notice.
## Config
| Key | Default | Meaning |
|---|---|---|
| `maxInlineBytes` | *(omitted)* | Model-facing context cap for a plain-text result, in UTF-8 bytes. **Omitted disables the policy entirely** (the plugin registers nothing). When set, a larger result is spilled and replaced with a preview derived from the same budget (head/tail split). |
## Behavior
1. Let the tool run (delegates via `next()`, so it bounds whatever a downstream hook accepted).
2. Skip `read` (avoids a `read → spill file → read again` loop) and any non-`accept` decision (a `block`'s corrective feedback passes through).
3. Flatten the accepted content only when it is **plain text** (all `text` blocks); a result with any non-text block is left untouched.
4. If its UTF-8 size is `≤ maxInlineBytes`, leave it unchanged.
5. Otherwise save the full text and replace the result with a preview + this notice:
```text
<retained head/tail preview>
(Omitted N bytes. Full formatted result saved to: /…/session-…/…-web_fetch.txt. Use read with offset/limit to inspect it.)
```
**Best-effort:** no session owner, no `ctx.spillFiles` backend, or a `saveText` rejection ⇒ the policy logs a warning and returns the original result. A spill failure never turns a successful call into an `isError` or hides the inline result.
## Scope
The policy sees only the FINAL formatted tool result — not a tool's internal resource. If a provider already truncated (e.g. `web-fetch-local.maxBodyChars`), the spill file holds the full formatted result the tool returned, not the full original source. Provider/resource caps stay mandatory and separate. Tool-owned early spill (bash streams, subagent rollouts) is future work — see the [tool output spill RFC](../../../docs/rfc/implemented/architecture/2026-07-08-tool-output-spill-files.md).

View File

@@ -0,0 +1,44 @@
{
"name": "@deepseek-ai/dsh-spill-policy",
"description": "Tool-result spill policy for the DeepSeek Harness — replaces oversized plain-text tool results with a retained preview plus a spill-file path (no service surface)",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-retention": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"@deepseek-ai/dsh-spill": "^0.0.1",
"@deepseek-ai/dsh-tools": "^0.0.1",
"cordis": "^4.0.0-rc.6"
},
"dependencies": {
"schemastery": "^3.18.0"
},
"devDependencies": {
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-retention": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-spill": "workspace:^",
"@deepseek-ai/dsh-tools": "workspace:^",
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,149 @@
/**
* The spill-policy PLUGIN: a `tools/post-execute` result transformer that keeps
* oversized plain-text tool results out of the model's context. When a final
* result's UTF-8 size exceeds `maxInlineBytes`, it saves the FULL text to a
* session-scoped spill file (`ctx.spillFiles`) and replaces the model-facing
* result with a bounded head/tail preview plus the spill path — the model reads
* the complete result later with the existing `read` tool.
*
* It registers NO service and owns NO storage or preview mechanics: preview is
* `@deepseek-ai/dsh-retention` (`TextRetainer`), storage is `ctx.spillFiles`.
* The policy only decides WHEN to spill and composes the notice.
*
* ## Deliberately narrow
*
* - Omitted `maxInlineBytes` ⇒ the plugin registers nothing (a true no-op).
* - Plain-text results only: a result carrying any non-text block is left
* untouched (the policy knows only the final formatted text, not tool
* internals).
* - `read` is skipped to avoid a `read → spill file → read again` loop.
* - Best-effort: no session owner, no `ctx.spillFiles` backend, or a save
* failure ⇒ log and return the original result. A spill failure must NEVER
* turn a successful tool call into an `isError` or hide the inline result.
*
* It COMPOSES with other post-execute listeners: it delegates via `next()` and
* bounds the resulting `accept` content, so a hook that replaced the content
* still has its replacement bounded, and a `block` decision passes through
* unchanged.
*
* @module @deepseek-ai/dsh-spill-policy
*/
import type { Context } from 'cordis'
import z from 'schemastery'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { TextRetainer, describeOmitted } from '@deepseek-ai/dsh-retention'
import type { Omitted } from '@deepseek-ai/dsh-retention'
import type { SaveTextSpill } from '@deepseek-ai/dsh-spill'
import type { SessionId } from '@deepseek-ai/dsh-session'
import type { PostToolDecision, ToolExecution } from '@deepseek-ai/dsh-tools'
import type { SpillPolicyExec } from './types.ts'
export type { SpillPolicyExec } from './types.ts'
/** Plugin config. */
export interface Config {
/**
* The model-facing context cap for a plain-text tool result, in UTF-8 bytes.
* Omitted disables the policy entirely (no-op). When set, a result larger than
* this is spilled and replaced with a preview derived from this same budget.
*/
maxInlineBytes?: number
}
/** Cordis plugin name used by loader diagnostics. */
export const name = 'spill-policy'
/** Require the tool registry (its `tools/post-execute` waterfall is the seam we transform). */
export const inject = ['tools']
export const Config: z<Config> = z.object({
maxInlineBytes: z.number(),
})
/** All-text content flattened to one UTF-8 string, or `undefined` if any block is non-text. */
function flattenPlainText(content: ContentBlock[]): string | undefined {
let text = ''
for (const block of content) {
if (block.type !== 'text') return undefined
text += block.text
}
return text
}
/** The owning session id, or `undefined` for a call with no agent (a direct/test call). */
function ownerSessionId(exec: ToolExecution): SessionId | undefined {
return (exec as SpillPolicyExec).agent?.session.header.id
}
/** Build the bounded head/tail preview for `text`, splitting `maxInlineBytes` across the two ends. */
function preview(text: string, maxInlineBytes: number): { text: string; omitted: Omitted } {
const headBytes = Math.ceil(maxInlineBytes / 2)
const tailBytes = Math.floor(maxInlineBytes / 2)
const retainer = new TextRetainer({ kind: 'headTail', headBytes, tailBytes })
retainer.push(text)
const kept = retainer.finish()
return { text: kept.text, omitted: kept.omittedBytes }
}
/**
* Compose the replacement text: the bounded preview, a blank line, then the
* spill notice. The omission clause comes from the retention library
* (`describeOmitted`); the recovery sentence names the concrete spill path.
*/
function replacementText(previewText: string, omitted: Omitted, spillPath: string): string {
const omission = describeOmitted(omitted, 'bytes')
const notice = `(${omission} Full formatted result saved to: ${spillPath}. Use read with offset/limit to inspect it.)`
return `${previewText}\n\n${notice}`
}
export function apply(ctx: Context, config: Config): void {
const maxInlineBytes = config.maxInlineBytes
// Omitted ⇒ no automatic spill policy: register nothing at all.
if (maxInlineBytes === undefined) return
ctx.on('tools/post-execute', async (exec, result, next): Promise<PostToolDecision> => {
// Delegate first so a downstream listener (e.g. a hook) settles the result;
// we bound whatever it accepted. A block passes through — spill only shapes
// accepted plain-text results, never corrective feedback.
const decision = await next()
// Skip `read` to avoid a read → spill file → read again loop.
if (decision.kind !== 'accept' || exec.name === 'read') return decision
const content = decision.content ?? result.content
const text = flattenPlainText(content)
if (text === undefined) return decision
if (Buffer.byteLength(text, 'utf8') <= maxInlineBytes) return decision
const sessionId = ownerSessionId(exec)
if (sessionId === undefined) {
ctx.logger.warn(`spill-policy: no session owner for ${exec.name} result; keeping the inline result`)
return decision
}
const spillFiles = ctx.get('spillFiles')
if (!spillFiles) {
ctx.logger.warn('spill-policy: no ctx.spillFiles backend loaded; keeping the inline result')
return decision
}
const save: SaveTextSpill = {
owner: { sessionId },
source: { toolName: exec.name, callId: exec.callId, label: 'result' },
suggestedName: `${exec.name}.txt`,
content: text,
}
let path: string
try {
({ path } = await spillFiles.saveText(save))
} catch (error: unknown) {
// Best-effort: a storage failure (permissions, ENOSPC, backend down) must
// never fail the call or hide the result — keep the original inline.
ctx.logger.warn(`spill-policy: saveText failed for ${exec.name}: ${String(error)}; keeping the inline result`)
return decision
}
const { text: previewText, omitted } = preview(text, maxInlineBytes)
const replaced: ContentBlock[] = [{ type: 'text', text: replacementText(previewText, omitted, path) }]
return { kind: 'accept', content: replaced, ...decision.additionalContext ? { additionalContext: decision.additionalContext } : {} }
})
}

View File

@@ -0,0 +1,26 @@
/**
* Vocabulary for the spill-policy plugin: the minimal structural view of a tool
* execution the policy needs to derive the owning session for a spill file.
*
* `@deepseek-ai/dsh-tools`' `ToolExecution` satisfies this shape, so the policy
* reads `exec` straight through without importing `dsh-tools` or `dsh-agent`.
* Only the session HEADER id is read — the same identity every other subsystem
* keys off (see `dsh-tool-bash`'s owner derivation).
*
* @module @deepseek-ai/dsh-spill-policy/types
*/
import type { SessionId } from '@deepseek-ai/dsh-session'
/** Minimal structural view of a tool execution: the owning session's header id, when present. */
export interface SpillPolicyExec {
/** The agent on whose behalf the call runs, when there is one. */
agent?: {
session: {
header: {
/** The canonical session identity — the spill owner. */
id: SessionId
}
}
}
}

View File

@@ -0,0 +1,196 @@
/**
* Tests for the spill-policy PLUGIN. It registers no service, only the
* `tools/post-execute` transformer. We drive real tools through
* `ctx.tools.execute(...)` and assert: disabled mode is a true no-op, an
* oversized plain-text result is spilled and replaced with a preview + path,
* a small result and a non-text result pass through, `read` is skipped, and a
* `saveText` failure / missing backend / missing owner all preserve the original
* result without an `isError`.
*/
import { describe, expect, it, vi } from 'vitest'
import { Context } from 'cordis'
import { CallId } from '@deepseek-ai/dsh-llm'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { SessionId } from '@deepseek-ai/dsh-session'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { defineTool } from '@deepseek-ai/dsh-tools'
import type { ToolExecution } from '@deepseek-ai/dsh-tools'
import { SpillFiles, SpillPath } from '@deepseek-ai/dsh-spill'
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
import * as SpillPolicy from '@deepseek-ai/dsh-spill-policy'
/** A stub spill backend recording its saves; `fail` exercises the best-effort fallback. */
class StubSpill extends SpillFiles {
saves: SaveTextSpill[] = []
fail = false
async saveText(input: SaveTextSpill): Promise<SpillRef> {
if (this.fail) throw new Error('disk full')
this.saves.push(input)
return { path: SpillPath(`/spill/${input.suggestedName}`), bytes: Buffer.byteLength(input.content, 'utf8') }
}
}
/** A tool returning `text` verbatim (name configurable so we can register `read`). */
function textTool(name: string, text: string) {
return defineTool({
name,
description: name,
parameters: {},
async execute(): Promise<ContentBlock[]> { return [{ type: 'text', text }] },
})
}
/** A minimal exec carrying a session header id (the spill owner). */
function exec(name: string, session = 's1'): ToolExecution {
// Only agent.session.header.id is read by the policy; a structural stub suffices.
const agent = { session: { header: { id: SessionId(session) } } }
return { callId: CallId(`call-${name}`), name, arguments: {}, agent } as unknown as ToolExecution
}
/**
* Build a context with tools + the policy, and optionally a spill backend.
* Returns the context and the backend handle (undefined when `withSpill` false).
*/
async function setup(config: SpillPolicy.Config, withSpill = true): Promise<{ ctx: Context; spill?: StubSpill }> {
const ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
let spill: StubSpill | undefined
if (withSpill) {
await ctx.plugin(StubSpill)
spill = ctx.spillFiles as StubSpill
}
await ctx.plugin(SpillPolicy, config)
return { ctx, ...spill ? { spill } : {} }
}
/** Flatten a result's text blocks. */
function textOf(content: ContentBlock[]): string {
return content.filter((b): b is Extract<ContentBlock, { type: 'text' }> => b.type === 'text').map(b => b.text).join('')
}
describe('disabled mode', () => {
it('registers no post-execute listener when maxInlineBytes is omitted', async () => {
const { ctx, spill } = await setup({})
ctx.tools.register(textTool('big', 'x'.repeat(1000)))
const result = await ctx.tools.execute(exec('big'))
expect(textOf(result.content)).toBe('x'.repeat(1000))
expect(result.isError).toBe(false)
expect(spill?.saves).toHaveLength(0)
})
})
describe('oversized plain-text replacement', () => {
it('spills the full text and replaces the result with a preview + path', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 20 })
const body = 'HEAD'.repeat(20) + 'TAIL'.repeat(20) // 160 bytes > 20
ctx.tools.register(textTool('big', body))
const result = await ctx.tools.execute(exec('big'))
expect(result.isError).toBe(false)
expect(spill?.saves).toHaveLength(1)
expect(spill?.saves[0]?.content).toBe(body)
expect(spill?.saves[0]?.source.toolName).toBe('big')
expect(spill?.saves[0]?.suggestedName).toBe('big.txt')
expect(spill?.saves[0]?.owner.sessionId).toBe('s1')
const text = textOf(result.content)
expect(text).not.toBe(body)
expect(text.startsWith('HEAD')).toBe(true)
expect(text).toContain('Full formatted result saved to: /spill/big.txt')
expect(text).toContain('Use read with offset/limit')
expect(text).toContain('Omitted')
})
it('leaves a small plain-text result unchanged', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 1000 })
ctx.tools.register(textTool('small', 'tiny'))
const result = await ctx.tools.execute(exec('small'))
expect(textOf(result.content)).toBe('tiny')
expect(spill?.saves).toHaveLength(0)
})
it('leaves a result with a non-text block unchanged', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 5 })
ctx.tools.register(defineTool({
name: 'mixed',
description: 'mixed',
parameters: {},
async execute(): Promise<ContentBlock[]> {
return [{ type: 'text', text: 'x'.repeat(100) }, { type: 'reasoning', text: 'why' }]
},
}))
const result = await ctx.tools.execute(exec('mixed'))
expect(spill?.saves).toHaveLength(0)
expect(result.content).toHaveLength(2)
})
})
describe('read skip', () => {
it('never spills the read tool result (avoids a read → spill → read loop)', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 10 })
ctx.tools.register(textTool('read', 'x'.repeat(1000)))
const result = await ctx.tools.execute(exec('read'))
expect(textOf(result.content)).toBe('x'.repeat(1000))
expect(spill?.saves).toHaveLength(0)
})
})
describe('best-effort fallback', () => {
it('keeps the original result when saveText fails', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 10 })
spill!.fail = true
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
ctx.tools.register(textTool('big', 'x'.repeat(1000)))
const result = await ctx.tools.execute(exec('big'))
expect(textOf(result.content)).toBe('x'.repeat(1000))
expect(result.isError).toBe(false)
expect(warn).toHaveBeenCalled()
})
it('keeps the original result when no spill backend is loaded', async () => {
const { ctx } = await setup({ maxInlineBytes: 10 }, false)
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
ctx.tools.register(textTool('big', 'x'.repeat(1000)))
const result = await ctx.tools.execute(exec('big'))
expect(textOf(result.content)).toBe('x'.repeat(1000))
expect(warn).toHaveBeenCalled()
})
it('keeps the original result when the call has no session owner', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 10 })
const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => {})
ctx.tools.register(textTool('big', 'x'.repeat(1000)))
const result = await ctx.tools.execute({ callId: CallId('c'), name: 'big', arguments: {} })
expect(textOf(result.content)).toBe('x'.repeat(1000))
expect(spill?.saves).toHaveLength(0)
expect(warn).toHaveBeenCalled()
})
})
describe('composition', () => {
it('bounds content a downstream post-execute listener replaced', async () => {
const { ctx, spill } = await setup({ maxInlineBytes: 10 })
// A later-registered listener replaces the (small) tool result with a big one;
// the policy delegated via next(), so it bounds the replacement.
ctx.on('tools/post-execute', async (_e, _r, _next) =>
({ kind: 'accept', content: [{ type: 'text', text: 'z'.repeat(500) }] }))
ctx.tools.register(textTool('small', 'tiny'))
const result = await ctx.tools.execute(exec('small'))
expect(spill?.saves[0]?.content).toBe('z'.repeat(500))
expect(textOf(result.content)).toContain('Full formatted result saved to')
})
it('preserves a downstream accept decision additionalContext when spilling', async () => {
const { ctx } = await setup({ maxInlineBytes: 10 })
const context = { content: [{ type: 'text' as const, text: 'note' }], source: { kind: 'plugin' as const, plugin: 'test' } }
ctx.on('tools/post-execute', async (_e, _r, _next) =>
({ kind: 'accept', additionalContext: context }))
ctx.tools.register(textTool('big', 'x'.repeat(1000)))
const result = await ctx.tools.execute(exec('big'))
expect(textOf(result.content)).toContain('Full formatted result saved to')
expect(result.additionalContext).toEqual(context)
})
})

View File

@@ -0,0 +1,18 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": ["src"],
"references": [
{ "path": "../../../vendor/cosmokit" },
{ "path": "../../../vendor/cordis" },
{ "path": "../../../vendor/schemastery" },
{ "path": "../../util/retention" },
{ "path": "../../llm/llm" },
{ "path": "../../core/session" },
{ "path": "../spill" },
{ "path": "../../core/tools" }
]
}

View File

@@ -0,0 +1,27 @@
# @deepseek-ai/dsh-spill
The **spill storage seam**: an abstract `SpillFiles` service (`ctx.spillFiles`) defining WHAT a spill backend does — persist a tool's oversized text to a session-scoped path the model can later `read` — without saying HOW.
This package is one third of the spill capability, split so each concern evolves (and swaps) independently:
| Package | Role |
|---|---|
| `@deepseek-ai/dsh-spill` (this) | the interface: abstract service + vocabulary types |
| `@deepseek-ai/dsh-spill-local` | an implementation: private session-scoped files on the host filesystem |
| `@deepseek-ai/dsh-spill-policy` | the tool-result policy that spills oversized final results |
The split mirrors the bash/fs seams. A future remote or virtual backend (e.g. a `spill://…` URI plus a read-only bridge for ACP or remote environments) implements this interface without touching the policy plugin.
## Service API (`ctx.spillFiles`)
| Member | Semantics |
|---|---|
| `saveText(input)` | Persist `input.content` verbatim to a session-scoped file; resolves with a `SpillRef` (path readable by the local `read` tool + exact bytes written). **Rejects on a real storage failure** (permissions, ENOSPC, backend unavailable) — the caller decides how to degrade. |
Storage is scoped by the request's `owner` session; the backend chooses a private (not world-readable) location and a collision-free name derived from — never equal to — the caller's `suggestedName`. The seam owns storage only: NO retention policy (that is [`@deepseek-ai/dsh-retention`](../../util/retention)), NO tool-result replacement (that is `@deepseek-ai/dsh-spill-policy`), NO file inspection (the model uses the existing `read` tool on the returned path).
## Vocabulary
`SaveTextSpill` (owner, source, suggestedName, content) is the request; `SpillRef` (path, bytes) is the result. `SpillPath` is [branded](../../util/brand) and rendered to the model as an ordinary path string in v1 — the brand records provenance (a runtime artifact, not a workspace file) so a future virtual backend can swap the path shape without a consumer change. `SpillOwner` scopes storage to a `SessionId`; unlike the bash executor's decoupled `OwnerToken`, spill is inherently session-scoped, so the seam imports `dsh-session`'s `SessionId` directly. `SpillSource` (toolName, callId, label) is descriptive provenance for the filename and future cleanup, not access control. See `src/types.ts` for the full contracts.
See the [tool output spill RFC](../../../docs/rfc/implemented/architecture/2026-07-08-tool-output-spill-files.md) for the design rationale, including why creation belongs to the runtime spill seam rather than the model-facing `write` tool.

View File

@@ -0,0 +1,36 @@
{
"name": "@deepseek-ai/dsh-spill",
"description": "Abstract spill storage seam (ctx.spillFiles) for the DeepSeek Harness — save oversized tool text to a session-scoped path",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "lib/index.js",
"types": "lib/types/index.d.ts",
"exports": {
".": {
"types": "./lib/types/index.d.ts",
"default": "./lib/index.js"
},
"./src/*": "./src/*",
"./package.json": "./package.json"
},
"files": [
"lib/index.js",
"lib/types/**/*.d.ts",
"lib/types/**/*.d.ts.map",
"src"
],
"license": "BSD-3-Clause",
"peerDependencies": {
"@deepseek-ai/dsh-brand": "^0.0.1",
"@deepseek-ai/dsh-llm": "^0.0.1",
"@deepseek-ai/dsh-session": "^0.0.1",
"cordis": "^4.0.0-rc.6"
},
"devDependencies": {
"@deepseek-ai/dsh-brand": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"cordis": "^4.0.0-rc.6"
}
}

View File

@@ -0,0 +1,60 @@
/**
* The spill storage seam (`ctx.spillFiles`): an abstract service defining WHAT a
* spill backend does — persist a tool's oversized text to a session-scoped path
* the model can later `read` — without saying HOW. Implementations subclass
* {@link SpillFiles} and register as the `spillFiles` service;
* `@deepseek-ai/dsh-spill-local` (host filesystem) is the first.
*
* The seam is deliberately minimal: `saveText` and nothing else. It owns NO
* retention policy (that is `@deepseek-ai/dsh-retention`), NO tool-result
* replacement (that is `@deepseek-ai/dsh-spill-policy`), and NO file inspection
* (the model uses the existing `read` tool on the returned path). A future
* remote/virtual backend may return a `spill://…` URI plus a read-only bridge;
* v1 keeps the path filesystem-shaped until such a backend exists.
*
* @module @deepseek-ai/dsh-spill
*/
import { Context, Service } from 'cordis'
import type { SaveTextSpill, SpillRef } from './types.ts'
export { SpillPath } from './types.ts'
export type { SaveTextSpill, SpillOwner, SpillRef, SpillSource } from './types.ts'
declare module 'cordis' {
interface Context {
spillFiles: SpillFiles
}
}
/**
* Abstract spill storage service. Subclass, implement {@link saveText}, and load
* the subclass as a plugin — it registers as `ctx.spillFiles` (one
* implementation per context; loading a second throws, cordis' standard
* duplicate-service behavior).
*
* Semantics every implementation must honor:
* - {@link saveText} persists the FULL `content` verbatim and returns a path
* the local `read` tool can open, plus the exact byte length written.
* - Storage is scoped by the request's {@link SaveTextSpill.owner} session; the
* backend chooses a private (not world-readable) location and a collision-free
* name derived from — never equal to — the caller's `suggestedName`.
* - `saveText` REJECTS on a real storage failure (permissions, ENOSPC, backend
* unavailable); the caller decides how to degrade (the spill policy treats a
* rejection as best-effort and keeps the inline result).
*/
export abstract class SpillFiles extends Service {
constructor(ctx: Context) {
super(ctx, 'spillFiles')
}
/**
* Persist `input.content` to a session-scoped spill file.
* @param input - the owner, provenance, suggested name, and full text to save.
* @returns the saved file's {@link SpillRef} (path + bytes written); rejects on
* a storage failure.
*/
abstract saveText(input: SaveTextSpill): Promise<SpillRef>
}
export default SpillFiles

View File

@@ -0,0 +1,68 @@
/**
* Vocabulary for the spill storage seam. Types only — the abstract service
* lives in `./index.ts`, implementations in sibling packages
* (`@deepseek-ai/dsh-spill-local` first).
*
* @module @deepseek-ai/dsh-spill/types
*/
import type { Branded } from '@deepseek-ai/dsh-brand'
import type { CallId } from '@deepseek-ai/dsh-llm'
import type { SessionId } from '@deepseek-ai/dsh-session'
/**
* A local filesystem path produced by the spill seam, intended for the model's
* `read` tool. The brand records that the path came from {@link SpillFiles.saveText}
* (a runtime artifact, not a workspace file); it is still rendered to the model
* as an ordinary path string in v1. A future remote/virtual backend may replace
* this with a `spill://…` URI, so consumers treat it as opaque.
*/
export type SpillPath = Branded<'SpillPath'>
/** Brand a string as a {@link SpillPath}. */
export function SpillPath(path: string): SpillPath {
return path as SpillPath
}
/**
* Who a spilled file belongs to: the session whose tool call produced it. The
* backend scopes storage per session (its directory layout, its cleanup unit),
* so the owner is the session id, not a decoupled token — spill is inherently
* session-scoped, unlike the bash executor's cross-session `OwnerToken`.
*/
export interface SpillOwner {
sessionId: SessionId
}
/**
* Provenance of one spilled artifact — recorded by the backend for a readable
* filename and future cleanup/inspection. Not interpreted for access control
* (the {@link SpillOwner} scopes storage); purely descriptive.
*/
export interface SpillSource {
/** The tool whose result was spilled (e.g. `web_fetch`). */
toolName: string
/** The model-issued call id the result belongs to. */
callId: CallId
/** A short human label for the artifact (e.g. `result`). */
label: string
}
/** One request to persist text to a spill file. */
export interface SaveTextSpill {
owner: SpillOwner
source: SpillSource
/**
* A caller-suggested base name (e.g. `web_fetch.txt`). The backend sanitizes
* it to a single safe path segment before use — it is a hint, never a path.
*/
suggestedName: string
/** The full text to persist (UTF-8). */
content: string
}
/** A saved spill file: its path plus the byte length written. */
export interface SpillRef {
path: SpillPath
bytes: number
}

View File

@@ -0,0 +1,56 @@
/**
* Tests for the spill seam INTERFACE: a minimal concrete subclass registers as
* `ctx.spillFiles`, a second load throws (duplicate service), and disposal
* releases the service. The storage behavior is the implementation's concern
* (`@deepseek-ai/dsh-spill-local`); here we only pin the seam contract.
*/
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { CallId } from '@deepseek-ai/dsh-llm'
import { SessionId } from '@deepseek-ai/dsh-session'
import { SpillFiles, SpillPath } from '@deepseek-ai/dsh-spill'
import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
/** Minimal concrete backend: records the last request, returns a fixed ref. */
class StubSpill extends SpillFiles {
last: SaveTextSpill | undefined
async saveText(input: SaveTextSpill): Promise<SpillRef> {
this.last = input
return { path: SpillPath(`/stub/${input.suggestedName}`), bytes: Buffer.byteLength(input.content, 'utf8') }
}
}
function request(content: string): SaveTextSpill {
return {
owner: { sessionId: SessionId('s1') },
source: { toolName: 'web_fetch', callId: CallId('c1'), label: 'result' },
suggestedName: 'web_fetch.txt',
content,
}
}
describe('spill seam', () => {
it('registers as ctx.spillFiles and saves text', async () => {
const ctx = new Context()
await ctx.plugin(StubSpill)
const ref = await ctx.spillFiles.saveText(request('hello'))
expect(ref).toEqual({ path: '/stub/web_fetch.txt', bytes: 5 })
expect((ctx.spillFiles as StubSpill).last?.content).toBe('hello')
})
it('rejects a second implementation (one per context)', async () => {
const ctx = new Context()
await ctx.plugin(StubSpill)
await expect(ctx.plugin(StubSpill)).rejects.toThrow()
})
it('releases the service on disposal', async () => {
const ctx = new Context()
const fiber = await ctx.plugin(StubSpill)
expect(ctx.spillFiles).toBeInstanceOf(StubSpill)
await fiber.dispose()
expect((ctx as Context & { spillFiles?: unknown }).spillFiles).toBeUndefined()
})
})

View File

@@ -0,0 +1,15 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "lib/types"
},
"include": ["src"],
"references": [
{ "path": "../../../vendor/cosmokit" },
{ "path": "../../../vendor/cordis" },
{ "path": "../../util/brand" },
{ "path": "../../llm/llm" },
{ "path": "../../core/session" }
]
}