Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: make search raw output recovery backend-neutral

Browse Source
This commit is contained in:
Dudu-0223
2026-07-10 11:53:02 +08:00
parent 1df9f3a84a
commit 3bb90bd4b6
16 changed files with 136 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
packages/bash/bash-local/README.md
View File

@@ -21,7 +21,7 @@ Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi;
- **Spawn per call, no shell state** — every call is a fresh non-login `bash -c` (deterministic; no rc files). All four surveyed tools spawn per call. `XXX(stateful-shell)` in `src/run.ts` records the two proven stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec sessions) for when real workflows demand them.
- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the `graceMs` grace (default 3s — OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools.
- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file.
- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. A foreground `BashExecRequest.stdoutMaxBytes` can raise stdout's capture budget for one trusted caller; stderr and background tasks still use `maxOutputBytes`. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file.
- **Model-friendly env + credential scrub** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`), then `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` (Codex's hardcoded set) so pagers and ANSI color don't garble results. This scrub is the security control that keeps the harness's *ambient* credentials out of a spawned command. A spec's `env` is merged LAST (after the scrub), so a caller's explicit entry — a value it already holds — wins even on a credential-shaped name. The spec's `stdin`, when supplied, is written to the child and closed; with none supplied, fd 0 is `/dev/null` — the exact pre-seam default, so a command that probes stdin's file type is unaffected. Both `env`/`stdin` are set by in-process plugins (the hooks bridges); the model-facing tool doesn't expose them. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).
- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. The spec's opaque `owner` token is stored on the tracked task and returned by `ownerOf(id)` — the executor never interprets it (the consumer's access policy does), and because it lives with the task here it survives a `tool-bash` HMR reload.

9
packages/bash/bash-local/src/index.ts
View File

@@ -121,10 +121,13 @@ export class LocalBashExecutor extends BashExecutor {
this.config.maxTimeoutMs,
'bash-local: request.timeoutMs',
)
const stdoutMaxBytes = request.stdoutMaxBytes ?? this.config.maxOutputBytes
assertPositiveFinite('request.stdoutMaxBytes', stdoutMaxBytes)
return {
command: request.command,
workdir: request.workdir ?? this.config.cwd ?? process.cwd(),
timeoutMs,
stdoutMaxBytes,
...request.signal ? { signal: request.signal } : {},
// Carry stdin/env through verbatim — optional, no config default (absent
// means none). env merges AFTER the scrub in run.ts.
@@ -144,7 +147,8 @@ export class LocalBashExecutor extends BashExecutor {
const outcome = await runBash({
command: spec.command,
cwd: spec.workdir,
maxOutputBytes: this.config.maxOutputBytes,
stdoutMaxBytes: spec.stdoutMaxBytes,
stderrMaxBytes: this.config.maxOutputBytes,
graceMs: this.config.graceMs,
signal: d.signal,
stdin: spec.stdin,
@@ -170,7 +174,8 @@ export class LocalBashExecutor extends BashExecutor {
const running = runBash({
command: spec.command,
cwd: spec.workdir,
maxOutputBytes: this.config.maxOutputBytes,
stdoutMaxBytes: this.config.maxOutputBytes,
stderrMaxBytes: this.config.maxOutputBytes,
graceMs: this.config.graceMs,
signal: spec.signal,
stdin: spec.stdin,

10
packages/bash/bash-local/src/run.ts
View File

@@ -77,8 +77,10 @@ export function childEnv(extra?: Record<string, string>): NodeJS.ProcessEnv {
export interface SpawnSpec {
command: string
cwd: string
/** Per-stream in-memory cap; overflow spills to disk (tail kept in memory). */
maxOutputBytes: number
/** Stdout in-memory cap; overflow spills to disk (tail kept in memory). */
stdoutMaxBytes: number
/** Stderr in-memory cap; overflow spills to disk (tail kept in memory). */
stderrMaxBytes: number
/** Grace period between the SIGTERM and the SIGKILL escalation on a kill. */
graceMs: number
/**
@@ -351,8 +353,8 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
? spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['pipe', 'pipe', 'pipe'], detached: true })
: spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['ignore', 'pipe', 'pipe'], detached: true })
const stdout = new OutputCollector(spec.maxOutputBytes, 'stdout', spillDir)
const stderr = new OutputCollector(spec.maxOutputBytes, 'stderr', spillDir)
const stdout = new OutputCollector(spec.stdoutMaxBytes, 'stdout', spillDir)
const stderr = new OutputCollector(spec.stderrMaxBytes, 'stderr', spillDir)
child.stdout.on('data', (chunk: Buffer) => { stdout.push(chunk) })
child.stderr.on('data', (chunk: Buffer) => { stderr.push(chunk) })

17
packages/bash/bash-local/tests/executor.spec.ts
View File

@@ -86,6 +86,23 @@ describe('LocalBashExecutor.run', () => {
const { bash } = await setup()
expect(() => bash.resolve({ command: 'true', timeoutMs: Number.NaN })).toThrow(/request\.timeoutMs/)
expect(() => bash.resolve({ command: 'true', timeoutMs: -1 })).toThrow(/request\.timeoutMs/)
expect(() => bash.resolve({ command: 'true', stdoutMaxBytes: Number.NaN })).toThrow(/request\.stdoutMaxBytes/)
expect(() => bash.resolve({ command: 'true', stdoutMaxBytes: -1 })).toThrow(/request\.stdoutMaxBytes/)
})
it('defaults stdoutMaxBytes to maxOutputBytes and lets foreground callers raise stdout only', async () => {
const { bash } = await setup({ maxOutputBytes: 100 })
expect(bash.resolve({ command: 'true' }).stdoutMaxBytes).toBe(100)
const result = await bash.run(bash.resolve({
command: 'printf "%.0sx" $(seq 1 500); printf "%.0se" $(seq 1 500) >&2',
stdoutMaxBytes: 500,
}))
expect(result.stdout.truncated).toBe(false)
expect(result.stdout.text).toBe('x'.repeat(500))
expect(result.stderr.truncated).toBe(true)
expect(result.stderr.text.length).toBeLessThanOrEqual(100)
})
it('kill escalation uses the configured graceMs (a TERM-trapping task dies by SIGKILL)', async () => {

27
packages/bash/bash-local/tests/run.spec.ts
View File

@@ -26,7 +26,8 @@ function spec(command: string, overrides: Partial<Parameters<typeof runBash>[0]>
return {
command,
cwd: process.cwd(),
maxOutputBytes: 64_000,
stdoutMaxBytes: 64_000,
stderrMaxBytes: 64_000,
graceMs: 3_000,
...overrides,
}
@@ -229,10 +230,24 @@ describe('stdin and extra env (set by in-process plugins)', () => {
})
describe('output truncation and spill', () => {
it('applies stdout and stderr caps independently', async () => {
const result = await runBash(
spec('printf "%.0sx" $(seq 1 500); printf "%.0se" $(seq 1 500) >&2', {
stdoutMaxBytes: 500,
stderrMaxBytes: 100,
}),
{ spillDir },
).done
expect(result.stdout.truncated).toBe(false)
expect(result.stdout.text).toBe('x'.repeat(500))
expect(result.stderr.truncated).toBe(true)
expect(result.stderr.text.length).toBeLessThanOrEqual(100)
})
it('keeps the tail and spills the full stream to disk', async () => {
// 200 numbered lines of ~10 bytes; cap at 500 bytes keeps a late tail.
const result = await runBash(
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }),
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }),
{ spillDir },
).done
expect(result.stdout.truncated).toBe(true)
@@ -247,7 +262,7 @@ describe('output truncation and spill', () => {
it('does not truncate output exactly at the cap', async () => {
const result = await runBash(
spec('printf "%.0sx" $(seq 1 500)', { maxOutputBytes: 500 }),
spec('printf "%.0sx" $(seq 1 500)', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }),
{ spillDir },
).done
expect(result.stdout.truncated).toBe(false)
@@ -258,7 +273,7 @@ describe('output truncation and spill', () => {
it('settles with the tail and no spill path when final spill close fails', async () => {
failNextClose.value = true
const result = await runBash(
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }),
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }),
{ spillDir },
).done
expect(failNextClose.value).toBe(false)
@@ -377,7 +392,7 @@ describe('review fixes: env scrubbing and spill hardening', () => {
it('creates spill files with owner-only permissions and random names', async () => {
const result = await runBash(
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }),
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }),
{ spillDir },
).done
const path = result.stdout.spillPath!
@@ -388,7 +403,7 @@ describe('review fixes: env scrubbing and spill hardening', () => {
it('defaults spills into a private per-process directory', async () => {
const result = await runBash(
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }),
spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { stdoutMaxBytes: 500, stderrMaxBytes: 500 }),
).done
const dir = dirname(result.stdout.spillPath!)
expect(dir).toMatch(/dsh-bash-/)

2
packages/bash/bash/README.md
View File

@@ -28,6 +28,6 @@ Implementations subclass `BashExecutor`, implement the abstract methods, and cal
## Vocabulary
`BashExecRequest` (command, workdir?, timeoutMs?, signal?, stdin?, env?, owner?) resolves to `BashExecSpec` (command, workdir, timeoutMs, signal?, stdin?, env?, owner) before execution; `owner` is optional on the request and **required-but-nullable** (`OwnerToken | undefined`) on the resolved spec, so a forgotten owner is a visible `undefined` rather than a silently-absent property. The task id (`BashTaskId`) and the `owner` token (`OwnerToken`) are [branded](../../util/brand) — `OwnerToken` is a DISTINCT brand from `SessionId` (the seam never imports `dsh-session`; the `dsh-tool-bash` consumer is the single boundary that casts its `SessionId` into one). `run()` returns `BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr as `CollectedOutput`) and `start()`/`readOutput()` use `BashTask`/`BashTaskRead` for the background side. See `src/types.ts` for the full contracts.
`BashExecRequest` (command, workdir?, timeoutMs?, stdoutMaxBytes?, signal?, stdin?, env?, owner?) resolves to `BashExecSpec` (command, workdir, timeoutMs, stdoutMaxBytes, signal?, stdin?, env?, owner) before execution; `owner` is optional on the request and **required-but-nullable** (`OwnerToken | undefined`) on the resolved spec, so a forgotten owner is a visible `undefined` rather than a silently-absent property. The task id (`BashTaskId`) and the `owner` token (`OwnerToken`) are [branded](../../util/brand) — `OwnerToken` is a DISTINCT brand from `SessionId` (the seam never imports `dsh-session`; the `dsh-tool-bash` consumer is the single boundary that casts its `SessionId` into one). `stdoutMaxBytes` is a trusted foreground-run capture budget for consumers that must parse complete stdout up to their own limit; the model-facing bash tool does not expose it. `run()` returns `BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr as `CollectedOutput`) and `start()`/`readOutput()` use `BashTask`/`BashTaskRead` for the background side. See `src/types.ts` for the full contracts.
`stdin` and `env` are set by in-process plugins (the hooks bridges, native plugins) to feed a hook command its JSON payload on stdin and its `CLAUDE_PROJECT_DIR`/`CLAUDE_PLUGIN_ROOT` env. The model-facing `dsh-tool-bash` tool does not expose them as parameters — a model already has equivalent power through shell syntax (`FOO=bar cmd`, a heredoc), so they would be redundant tool params. This is not a security boundary: the implementation's credential scrub (not these fields) is what keeps the harness's ambient secrets out of a spawned command. They are plain optionals on the resolved spec (unlike `owner`'s required-but-nullable): a missing one means "none", the safe default. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).

12
packages/bash/bash/src/types.ts
View File

@@ -52,6 +52,13 @@ export interface BashExecRequest {
workdir?: string | undefined
/** Timeout override in milliseconds (implementations cap it). */
timeoutMs?: number | undefined
/**
* Foreground stdout capture budget in bytes. Absent uses the executor's
* default output cap. Trusted in-process consumers use this when they must
* parse complete stdout up to their own bounded limit; the model-facing bash
* tool does not expose it as a parameter.
*/
stdoutMaxBytes?: number | undefined
/** Abort signal — implementations kill the command when it fires. */
signal?: AbortSignal | undefined
/**
@@ -95,6 +102,11 @@ export interface BashExecSpec {
command: string
workdir: string
timeoutMs: number
/**
* Resolved foreground stdout capture budget in bytes. `run()` uses it for
* stdout; background tasks and stderr keep the executor's own output cap.
*/
stdoutMaxBytes: number
/** Abort signal — implementations kill the command when it fires. */
signal?: AbortSignal | undefined
/**

1
packages/bash/bash/tests/service.spec.ts
View File

@@ -13,6 +13,7 @@ class StubExecutor extends BashExecutor {
command: request.command,
workdir: request.workdir ?? '/stub',
timeoutMs: request.timeoutMs ?? 1000,
stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000,
...request.signal ? { signal: request.signal } : {},
owner: request.owner,
}

2
packages/bash/tool-bash/README.md
View File

@@ -44,7 +44,7 @@ When a background task finishes, a short notice is injected into the owning agen
## The tool builds its request from named args only
The `BashExecRequest` seam carries optional `stdin` and `env`, used by the hooks bridges to feed a hook command its JSON payload and `CLAUDE_*` env. This tool does **not** expose them as parameters: its request is built from `command`/`workdir`/`timeoutMs`/`signal`/`owner` only, so a model that includes `env` or `stdin` keys in its tool arguments has them ignored. This is not a trust boundary — a model already has equivalent power through shell syntax (`FOO=bar cmd`, a heredoc), and the real defense against leaking the harness's ambient secrets is `dsh-bash-local`'s credential scrub, which works regardless. A regression guard drives the real tool with those extra args and asserts the resulting request carries neither field — its job is to catch a future refactor that blindly spreads `...args` into the request (which would silently forward model input into the post-scrub `env` merge), not to defend a wall. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).
The `BashExecRequest` seam carries optional trusted-plugin fields (`stdoutMaxBytes`, `stdin`, and `env`); hooks use `stdin`/`env` to feed a hook command its JSON payload and `CLAUDE_*` env. This tool does **not** expose them as parameters: its request is built from `command`/`workdir`/`timeoutMs`/`signal`/`owner` only, so a model that includes `env`, `stdin`, or `stdoutMaxBytes` keys in its tool arguments has them ignored. This is not a trust boundary — a model already has equivalent power through shell syntax (`FOO=bar cmd`, a heredoc), and the real defense against leaking the harness's ambient secrets is `dsh-bash-local`'s credential scrub, which works regardless. A regression guard drives the real tool with those extra args and asserts the resulting request carries none of those fields — its job is to catch a future refactor that blindly spreads `...args` into the request (which would silently forward model input into the post-scrub `env` merge), not to defend a wall. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).
## Permissions

23
packages/bash/tool-bash/tests/tools.spec.ts
View File

@@ -98,6 +98,7 @@ class LossyReadBashExecutor extends BashExecutor {
command: request.command,
workdir: request.workdir ?? process.cwd(),
timeoutMs: request.timeoutMs ?? 0,
stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000,
...request.signal ? { signal: request.signal } : {},
owner: request.owner,
}
@@ -873,11 +874,12 @@ describe('the model-facing bash tool builds its request from named args only (no
/**
* Records every {@link BashExecRequest} the consumer hands to `resolve()`, so a
* test can assert what the model-facing tool DID and DID NOT forward. The `bash`
* tool does not expose `stdin`/`env` as parameters (bash syntax already gives a
* model that power), so it must build its request from named args only and
* tool does not expose trusted-plugin fields (`stdoutMaxBytes`, `stdin`, or
* `env`) as parameters, so it must build its request from named args only and
* never spread unknown tool-call keys into it. This guard's job is to catch a
* future refactor that blindly forwards `...args` — which would silently thread
* model input into the post-scrub `env` merge — NOT to defend a trust boundary
* model input into the post-scrub `env` merge or per-run capture budget — NOT
* to defend a trust boundary
* (the credential scrub in dsh-bash-local is the security control; see the
* bash-stdin-env RFC). Foreground `run()` returns a canned result; `start()` is
* unused here.
@@ -890,6 +892,7 @@ describe('the model-facing bash tool builds its request from named args only (no
command: request.command,
workdir: request.workdir ?? process.cwd(),
timeoutMs: request.timeoutMs ?? 0,
stdoutMaxBytes: request.stdoutMaxBytes ?? 64_000,
...request.signal ? { signal: request.signal } : {},
...request.stdin !== undefined ? { stdin: request.stdin } : {},
...request.env !== undefined ? { env: request.env } : {},
@@ -920,12 +923,12 @@ describe('the model-facing bash tool builds its request from named args only (no
return { ctx, bash: ctx.bash as RecordingBashExecutor }
}
it('does not forward env/stdin even when the model includes them as extra arguments', async () => {
it('does not forward trusted-only fields even when the model includes them as extra arguments', async () => {
const { ctx, bash } = await setupRecording()
// Extra args: the model includes `env` and `stdin` keys hoping they reach the
// Extra args: the model includes trusted-plugin keys hoping they reach the
// executor. The bash tool's schema ignores unknown keys, and execute() builds
// the request from only command/workdir/timeoutMs/signal — so the recorded
// request carries NEITHER. (Not a security wall — the model could set an env
// request carries NONE. (Not a security wall — the model could set an env
// var or feed stdin via shell syntax anyway; this just keeps the request
// shape honest so a future `...args` spread can't silently forward input.)
await ctx.tools.execute({
@@ -936,6 +939,7 @@ describe('the model-facing bash tool builds its request from named args only (no
description: 'echo',
env: { SNEAKY_API_KEY: 'leak' },
stdin: 'malicious payload',
stdoutMaxBytes: 999_999,
},
})
expect(bash.requests).toHaveLength(1)
@@ -943,9 +947,10 @@ describe('the model-facing bash tool builds its request from named args only (no
expect(request.command).toBe('echo hi')
expect('env' in request).toBe(false)
expect('stdin' in request).toBe(false)
expect('stdoutMaxBytes' in request).toBe(false)
})
it('a background bash call likewise carries no env/stdin', async () => {
it('a background bash call likewise carries no trusted-only fields', async () => {
const { ctx, bash } = await setupRecording()
// start() throws in this recorder, but resolve() runs first and records the
// request — which is all this no-forward assertion needs.
@@ -958,15 +963,17 @@ describe('the model-facing bash tool builds its request from named args only (no
run_in_background: true,
env: { TOKEN: 'leak' },
stdin: 'x',
stdoutMaxBytes: 999_999,
},
})
expect(bash.requests).toHaveLength(1)
const request = bash.requests[0]!
expect('env' in request).toBe(false)
expect('stdin' in request).toBe(false)
expect('stdoutMaxBytes' in request).toBe(false)
// The owner token IS set on a background call (the isolation fence) — proving
// the recorder sees the real request the consumer built, so the absent
// env/stdin above is a real negative, not a recorder that drops everything.
// trusted-only fields above are a real negative, not a recorder that drops everything.
expect('owner' in request).toBe(true)
})
})