fix(sandbox): evidence-gate runner failures (round 1)

This commit is contained in:
Hypatia May
2026-08-03 17:31:08 +08:00
parent 6a3caea277
commit 6343d8f6e6
51 changed files with 805 additions and 140 deletions

View File

@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/tool-fs-search/README.md
README.md: b12ffda9869c7d6bef5ea5b54594781ecf555ff4
README.zh.md: 7dd6cdf9a209f2fe357b4ffe48d20d574266ce60
README.md: 93df279e958c569cd0b30618d3c3e80f773d452b
README.zh.md: 5cec79577dc896c2a95f9539a9911adcdc23cfe1

View File

@@ -46,7 +46,7 @@ Raw `rg` stdout is an internal transport detail. Each search requests `stdoutMax
## Errors
Search failures carry the package-owned `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors.
Search-owned failures carry `SearchError` (a `HarnessError` subclass), surfaced as `{ name, code }` on `isError` results: `SEARCH_INVALID_PATTERN` (ripgrep rejected the regex/glob), `SEARCH_FAILED` (runtime `rg` disappearance after registration, inaccessible target, signal kill, malformed `--json` output), `SEARCH_RAW_OUTPUT_OVERFLOW` (raw output over `rawOutputMaxBytes`, or still truncated after the requested stdout capture budget), and `SEARCH_ABORTED` (tool timeout, caller cancellation, or the bash executor's own timeout). An existing structured `HarnessError` rejected by the bash executor, including `SANDBOX_UNAVAILABLE`, propagates unchanged; only an untyped spawn, cwd, or shell-start rejection becomes `SEARCH_FAILED`, while an aborted signal remains `SEARCH_ABORTED`. ripgrep exit semantics are tool-owned: exit 0 is success with results, exit 1 is a successful empty search (`No files found` / `No matches found`), and only other exits are failures. Model argument mistakes (blank pattern, a list-valued `include`) stay ordinary tool argument errors.
## Model Experience
@@ -114,7 +114,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
Failures are normalized as `Error: <message>` with structured `SEARCH_INVALID_PATTERN`, `SEARCH_FAILED`, `SEARCH_RAW_OUTPUT_OVERFLOW`, or `SEARCH_ABORTED` metadata for callers.
Search-owned failures render as `Error: <message>` with structured `SEARCH_INVALID_PATTERN`, `SEARCH_FAILED`, `SEARCH_RAW_OUTPUT_OVERFLOW`, or `SEARCH_ABORTED` metadata; structured bash-executor failures retain their owning name and code.
#### Token effect

View File

@@ -46,7 +46,7 @@ await ctx.plugin(LocalSpillStore) // @deepseek-ai/dsh-
## 错误
搜索失败携带本包拥有的 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(注册后 `rg` 在运行时消失、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍被截断)和 `SEARCH_ABORTED`(工具超时、调用方取消或 bash 执行器自身超时)。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。
搜索层自身的失败携带 `SearchError`(`HarnessError` 子类),以 `{ name, code }` 公开在 `isError` 结果上:`SEARCH_INVALID_PATTERN`(ripgrep 拒绝正则/glob)、`SEARCH_FAILED`(注册后 `rg` 在运行时消失、目标不可访问、信号终止、`--json` 输出格式错误)、`SEARCH_RAW_OUTPUT_OVERFLOW`(原始输出超过 `rawOutputMaxBytes`,或在请求 stdout 捕获预算后仍被截断)和 `SEARCH_ABORTED`(工具超时、调用方取消或 bash 执行器自身超时)。bash 执行器拒绝并返回的既有结构化 `HarnessError`(包括 `SANDBOX_UNAVAILABLE`)会原样传播;只有无类型的 spawn、cwd 或 shell 启动拒绝会转换为 `SEARCH_FAILED`,中止信号仍为 `SEARCH_ABORTED`。ripgrep 退出语义由工具拥有:退出 0 表示成功且有结果,退出 1 表示成功的空搜索(`No files found` / `No matches found`),只有其他退出值表示失败。模型参数错误(空白 pattern、列表值 `include`)仍是普通工具参数错误。
## 模型体验
@@ -114,7 +114,7 @@ glob 描述会说明配置所指定的超限结果排序方式。已生成的 [`
#### 模型看到的内容
失败会规范化为 `Error: <message>`,并向调用方提供结构化的 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据。
搜索层自身的失败会渲染为 `Error: <message>`,并附带结构化的 `SEARCH_INVALID_PATTERN`、`SEARCH_FAILED`、`SEARCH_RAW_OUTPUT_OVERFLOW` 或 `SEARCH_ABORTED` 元数据;来自 bash 执行器的结构化失败则保留其原有名称和错误码。
#### Token 影响

View File

@@ -44,10 +44,14 @@
"@deepseek-ai/dsh-agent": "workspace:^",
"@deepseek-ai/dsh-bash": "workspace:^",
"@deepseek-ai/dsh-bash-local": "workspace:^",
"@deepseek-ai/dsh-bash-sandbox": "workspace:^",
"@deepseek-ai/dsh-subprocess-local": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
"@deepseek-ai/dsh-retention": "workspace:^",
"@deepseek-ai/dsh-sandbox": "workspace:^",
"@deepseek-ai/dsh-sandbox-local": "workspace:^",
"@deepseek-ai/dsh-sandbox-policy": "workspace:^",
"@deepseek-ai/dsh-session": "workspace:^",
"@deepseek-ai/dsh-spill": "workspace:^",
"@deepseek-ai/dsh-system-prompt": "workspace:^",

View File

@@ -152,10 +152,9 @@ function completeStdout(toolName: string, result: BashRunResult, rawOutputMaxByt
* {@link SearchError} (abort/timeout → `SEARCH_ABORTED`, invalid pattern →
* `SEARCH_INVALID_PATTERN`, the rest → `SEARCH_FAILED` /
* `SEARCH_RAW_OUTPUT_OVERFLOW`). A `run()` REJECTION — the seam's
* infrastructure failures (pre-aborted signal, unusable workdir, missing
* shell) — is translated into the same taxonomy: a pre-aborted signal becomes
* `SEARCH_ABORTED`, everything else `SEARCH_FAILED`, with the original as
* `cause`.
* infrastructure failures becomes `SEARCH_ABORTED` when the forwarded signal
* aborted, propagates an existing structured {@link HarnessError} unchanged,
* and wraps only untyped spawn/workdir/shell-start errors as `SEARCH_FAILED`.
*
* @param ctx - the plugin context; execution uses its `bash` service.
* @param exec - the tool-execution context; supplies the session cwd and the abort signal.
@@ -182,12 +181,13 @@ export async function runRipgrep(
try {
result = await ctx.bash.run(spec)
} catch (error: unknown) {
// The seam contract: run() REJECTS only for infrastructure failures — a
// pre-aborted signal, an unusable workdir, a missing shell. Translate them
// so these failures stay machine-routable under the SEARCH_* taxonomy.
// Abort owns the outcome even when the executor rejects during teardown.
if (spec.signal?.aborted === true) {
throw new SearchError(`${toolName} was aborted before completion (tool timeout or caller cancellation)`, 'SEARCH_ABORTED', { cause: error })
}
// Infrastructure implementations may already provide a stable harness
// error (notably SANDBOX_UNAVAILABLE); preserve that owning taxonomy.
if (error instanceof HarnessError) throw error
throw new SearchError(`${toolName} could not start its search command (unusable working directory or missing shell)`, 'SEARCH_FAILED', { cause: error })
}
if (result.aborted) {

View File

@@ -18,6 +18,10 @@ import { CallId } from '@deepseek-ai/dsh-llm'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH } from '@deepseek-ai/dsh-tools'
import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local'
import { SANDBOX_UNAVAILABLE } from '@deepseek-ai/dsh-sandbox'
import { LocalSandboxProvider } from '@deepseek-ai/dsh-sandbox-local'
import { SandboxPolicyService } from '@deepseek-ai/dsh-sandbox-policy'
import { SandboxBashExecutor } from '@deepseek-ai/dsh-bash-sandbox'
import LocalSubprocessService from '@deepseek-ai/dsh-subprocess-local'
import * as ToolFsSearch from '@deepseek-ai/dsh-tool-fs-search'
@@ -68,6 +72,7 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
})
afterEach(async () => {
await ctx.fiber.dispose()
await rm(dir, { recursive: true, force: true })
})
@@ -192,4 +197,59 @@ describe.skipIf(!hasRg)('search tools over the real bash executor + real rg', ()
expect(text(result)).toContain('could not start')
})
})
it('preserves rg semantics through partial Landlock and propagates a real structured sandbox failure', async () => {
await ctx.fiber.dispose()
const launcher = join(dir, 'landlock-run')
const failMarker = join(dir, 'fail-runner')
await writeFile(launcher, `#!/bin/sh
while [ "$#" -gt 0 ]; do
case "$1" in
--ro|--rw) shift 2 ;;
--) shift; break ;;
*) printf '%s\\n' 'landlock-run: usage error: unexpected fake argument' >&2; exit 125 ;;
esac
done
printf '%s\\n' 'landlock-run: partial enforcement (older Landlock ABI)' >&2
if [ -e ${ToolFsSearch.singleQuote(failMarker)} ]; then
printf '%s\\n' 'landlock-run: landlock ruleset error: fixture failure' >&2
exit 125
fi
exec "$@"
`, { mode: 0o755 })
ctx = new Context()
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await ctx.plugin(LocalSandboxProvider, {})
;(ctx.sandbox as LocalSandboxProvider).internals = {
platform: 'linux',
probeBwrap: () => false,
probeLandlock: () => 'partial',
landlockLauncher: launcher,
}
await ctx.plugin(SandboxPolicyService, { mode: 'workspace-write', workspaceRoot: dir })
await ctx.plugin(LocalSubprocessService)
await ctx.plugin(SandboxBashExecutor, { cwd: dir, timeoutMs: 20_000 })
await ctx.plugin(ToolFsSearch, { sampleOverCapGlobResults: true })
const grepNoMatch = await call('grep', { pattern: 'does-not-exist' })
expect(grepNoMatch.isError).toBe(false)
expect(text(grepNoMatch)).toBe('No matches found')
const globNoFiles = await call('glob', { pattern: '*.does-not-exist' })
expect(globNoFiles.isError).toBe(false)
expect(text(globNoFiles)).toBe('No files found')
const invalidRegex = await call('grep', { pattern: '(unclosed' })
expect(invalidRegex.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_INVALID_PATTERN' } })
await writeFile(failMarker, '')
const sandboxFailure = await call('grep', { pattern: 'alpha' })
expect(sandboxFailure.error).toMatchObject({
info: { name: 'SandboxUnavailableError', code: SANDBOX_UNAVAILABLE },
})
expect(text(sandboxFailure)).toContain('Runner failure: landlock-run: landlock ruleset error: fixture failure')
expect(text(sandboxFailure)).not.toContain('could not start its search command')
})
})

View File

@@ -13,7 +13,7 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { join, sep } from 'node:path'
import { createUserMessage, CallId } from '@deepseek-ai/dsh-llm'
import { CallId, HarnessError, createUserMessage } from '@deepseek-ai/dsh-llm'
import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt'
import ToolRegistry, { TOOL_ABORTED_BEFORE_DISPATCH, type ToolExecutionToken } from '@deepseek-ai/dsh-tools'
import { BashExecutor } from '@deepseek-ai/dsh-bash'
@@ -373,7 +373,7 @@ describe('workdir derivation and signal forwarding', () => {
const controller = new AbortController()
bash.handler = () => {
controller.abort('cancel search')
throw new Error('executor stopped on abort')
throw new HarnessError('structured executor failure during abort', 'SANDBOX_UNAVAILABLE')
}
const result = await call(ctx, 'grep', { pattern: 'x' }, { signal: controller.signal })
@@ -403,6 +403,26 @@ describe('workdir derivation and signal forwarding', () => {
expect(result.error).toMatchObject({ info: { name: 'SearchError', code: 'SEARCH_FAILED' } })
expect(text(result)).toContain('could not start')
})
it('propagates an existing structured bash failure unchanged', async () => {
const { ctx, bash } = await setup()
const upstream = new HarnessError('sandbox runner failed', 'SANDBOX_UNAVAILABLE')
bash.handler = () => { throw upstream }
const rejection = ToolFsSearch.runRipgrep(
ctx,
{ signal: testToolSignal } as never,
'grep',
"rg --json --regexp='x'",
20_000_000,
)
await expect(rejection).rejects.toBe(upstream)
const result = await call(ctx, 'grep', { pattern: 'x' })
expect(result.error).toMatchObject({ info: { name: 'HarnessError', code: 'SANDBOX_UNAVAILABLE' } })
expect(text(result)).toContain('sandbox runner failed')
expect(text(result)).not.toContain('could not start')
})
})
describe('exit semantics and failure classification', () => {