Merge remote-tracking branch 'origin/master' into codex/simp-prune-bash-surface
# Conflicts: # docs/config-catalog.md # packages/bash/tool-bash/src/index.ts
This commit is contained in:
@@ -24,7 +24,7 @@ Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi;
|
||||
- **Spawn per call, no shell state** — every call is a fresh non-login `bash -c` (deterministic; no rc files). All four surveyed tools spawn per call. `XXX(stateful-shell)` in `src/run.ts` records the two proven stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec sessions) for when real workflows demand them.
|
||||
- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the `graceMs` grace (default 3s — OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools.
|
||||
- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file.
|
||||
- **Model-friendly env + credential scrub** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`), then `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` (Codex's hardcoded set) so pagers and ANSI color don't garble results. This scrub is the security control that keeps the harness's *ambient* credentials out of a spawned command. A spec's `env` is merged LAST (after the scrub), so a caller's explicit entry — a value it already holds — wins even on a credential-shaped name. The spec's `stdin`, when supplied, is written to the child and closed; with none supplied, fd 0 is `/dev/null` — the exact pre-seam default, so a command that probes stdin's file type is unaffected. Both `env`/`stdin` are set by in-process plugins (the hooks bridges); the model-facing tool doesn't expose them. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).
|
||||
- **Model-friendly environment** — ambient credential-shaped variables are removed before noninteractive terminal defaults and explicit caller entries are applied. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. Trusted plugins use `env` and `stdin`, but the model-facing tool does not expose them. See the [bash stdin/env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md).
|
||||
- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. The spec's opaque `owner` token is stored on the tracked task and returned by `ownerOf(id)` — the executor never interprets it (the consumer's access policy does), and because it lives with the task here it survives a `tool-bash` HMR reload.
|
||||
|
||||
## Model Experience
|
||||
@@ -39,6 +39,5 @@ Indirectly, through `dsh-tool-bash`, which renders this executor's bounded stdou
|
||||
- **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work.
|
||||
- **Spill files are never deleted** — full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them.
|
||||
- **Finished background tasks are never evicted** — they stay in the task map, retaining their in-memory output tails, until executor disposal.
|
||||
- **`OutputCollector.snapshot()` / `totalBytes` are test-shaped residuals** — the live poll path uses `readFrom()` and a marked cleanup can inline the final snapshot and remove the unused public getter.
|
||||
|
||||
The raw process handling lives in `src/run.ts`; `src/index.ts` is the service wiring.
|
||||
|
||||
@@ -1,16 +1,8 @@
|
||||
/**
|
||||
* `LocalBashExecutor`: the local-subprocess implementation of the
|
||||
* `@deepseek-ai/dsh-bash` executor seam. Spawns `bash -c` per call in its
|
||||
* own process group (see `./run.ts` for the plumbing and the agent-tool
|
||||
* survey notes), tracks background tasks, and kills everything on dispose.
|
||||
*
|
||||
* TODO(permissions/sandbox): execution policy does NOT belong here — use
|
||||
* the `tools/pre-execute` deny/ask gate (see docs/architecture.md
|
||||
* § Extending The Harness) or implement a sandboxing `BashExecutor`.
|
||||
* Reference points:
|
||||
* Claude Code wraps commands in sandbox-exec/bubblewrap; Codex applies
|
||||
* seatbelt/landlock plus an execpolicy prefix-rule engine.
|
||||
*
|
||||
* Local-subprocess implementation of the bash seam. Each call runs in its own
|
||||
* process group, background tasks are tracked, and disposal kills and awaits
|
||||
* them. Execution policy belongs in `tools/pre-execute` or a sandboxing
|
||||
* executor, not this local process layer.
|
||||
* @module @deepseek-ai/dsh-bash-local
|
||||
*/
|
||||
|
||||
@@ -87,10 +79,9 @@ export class LocalBashExecutor extends BashExecutor {
|
||||
assertPositiveFinite('maxOutputBytes', this.config.maxOutputBytes)
|
||||
assertPositiveFinite('graceMs', this.config.graceMs)
|
||||
ctx.effect(() => async () => {
|
||||
// Kill every live process group and WAIT for the processes to close so
|
||||
// nothing outlives the fiber (HMR safety) — a TERM-trapping child is
|
||||
// held until the SIGKILL escalation lands. The base class already
|
||||
// silenced listeners, so these kills complete without notices.
|
||||
// Kill every live process group and WAIT for the processes to close so nothing outlives
|
||||
// the fiber (HMR safety) — a TERM-trapping child is held until the SIGKILL escalation
|
||||
// lands.
|
||||
const pending: Promise<void>[] = []
|
||||
for (const task of this.tasks.values()) {
|
||||
if (task.status === 'running') {
|
||||
@@ -151,23 +142,18 @@ export class LocalBashExecutor extends BashExecutor {
|
||||
stdin: spec.stdin,
|
||||
env: spec.env,
|
||||
}, this.internals).done
|
||||
// Classify the FIRST abort reason: a BASH_TIMEOUT TimeoutReason means our
|
||||
// timeout cut the command short; any other abort — an upstream cancel, or a
|
||||
// foreign (outer) deadline's timeout under nesting — is aborted. Scoping to
|
||||
// our own code keeps a nested outer deadline from reading as our timeout.
|
||||
// Mutually exclusive by construction — the fused signal reports one cause.
|
||||
// Classify the FIRST abort reason: a BASH_TIMEOUT TimeoutReason means our timeout cut the
|
||||
// command short; any other abort — an upstream cancel, or a foreign (outer) deadline's
|
||||
// timeout under nesting — is aborted.
|
||||
const timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined
|
||||
const aborted = d.signal.aborted && !timedOut
|
||||
return { ...outcome, timedOut, aborted, timeoutMs: spec.timeoutMs }
|
||||
}
|
||||
|
||||
start(spec: BashExecSpec): BashTask {
|
||||
// No timeout for background tasks (matches Claude Code, which detaches
|
||||
// the timeout when backgrounding); callers stop tasks via kill() — or
|
||||
// via spec.signal, which the seam contract honors for background runs
|
||||
// too (runBash wires it to the group kill). No deadline is created here,
|
||||
// so spec.timeoutMs is ignored by design — background tasks stay
|
||||
// timeout-free (see the timeout-library RFC).
|
||||
// No timeout for background tasks (matches Claude Code, which detaches the timeout when
|
||||
// backgrounding); callers stop tasks via kill() — or via spec.signal, which the seam
|
||||
// contract honors for background runs too (runBash wires it to the group kill).
|
||||
const running = runBash({
|
||||
command: spec.command,
|
||||
cwd: spec.workdir,
|
||||
|
||||
@@ -1,23 +1,7 @@
|
||||
/**
|
||||
* Process plumbing for the local bash executor: spawn, output collection
|
||||
* with tail-keep + spill-to-disk truncation, and process-group kill with
|
||||
* SIGTERM→SIGKILL escalation.
|
||||
*
|
||||
* Everything here is deliberately free of Cordis concepts so it can be unit
|
||||
* tested in isolation; `LocalBashExecutor` owns lifecycle and configuration.
|
||||
*
|
||||
* runBash owns NO timing: it kills the process group when its `spec.signal`
|
||||
* fires and does not distinguish a timeout from a cancel. The executor fuses
|
||||
* timeout + upstream cancellation into that one signal via
|
||||
* `@deepseek-ai/dsh-timeout`'s `deadline`, and classifies the outcome from the
|
||||
* signal afterward — the timing/classification half is shared, the kill is not.
|
||||
*
|
||||
* Design notes (surveyed against Claude Code, OpenCode, Codex, and pi — see
|
||||
* the package README): spawn-per-call with `detached: true` so the child
|
||||
* leads its own process group; kills target the group (`kill(-pid)`) so
|
||||
* pipelines and subshells die with the parent. SIGTERM first, SIGKILL after a
|
||||
* grace period (OpenCode's escalation; Codex/pi jump straight to SIGKILL).
|
||||
*
|
||||
* Process plumbing for the local bash executor: detached process-group spawn,
|
||||
* tail-keep output with spill files, and SIGTERM→SIGKILL escalation. This layer
|
||||
* reacts to an abort signal; the executor owns deadlines and classifies causes.
|
||||
* @module dsh-bash-local/run
|
||||
*/
|
||||
|
||||
@@ -50,18 +34,9 @@ export const ENV_OVERRIDES = {
|
||||
export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i
|
||||
|
||||
/**
|
||||
* `process.env` minus credential-shaped vars, plus the model-friendly
|
||||
* overrides, plus any caller-supplied `extra` entries.
|
||||
* Build a child environment by scrubbing credential-shaped ambient variables,
|
||||
* applying model-friendly overrides, then merging trusted caller entries last.
|
||||
*
|
||||
* Layering matters: the scrub drops `process.env` credentials, then
|
||||
* `ENV_OVERRIDES` forces the model-friendly terminal vars, then `extra` is
|
||||
* merged LAST so an explicit caller entry wins even when its name matches the
|
||||
* scrub pattern (the scrub is the control that stops the HARNESS's ambient
|
||||
* credentials leaking into a spawned command; a caller that explicitly sets a
|
||||
* var named a value it already holds, not that ambient secret). `extra` is set
|
||||
* by in-process plugins (the hooks bridges), not the model — `dsh-tool-bash`
|
||||
* builds its request from named fields only and does not forward model input
|
||||
* here (see its README, § "The tool builds its request from named args only").
|
||||
* @param extra - caller-supplied entries merged last; an explicit entry wins even against the scrub and the overrides.
|
||||
* @returns the environment to hand to `spawn` for the child process.
|
||||
*/
|
||||
@@ -241,10 +216,8 @@ export class OutputCollector {
|
||||
try {
|
||||
closeSync(this.spillFd)
|
||||
} catch {
|
||||
// close can surface delayed writeback failures (for example EIO/ENOSPC)
|
||||
// after writeSync appeared to succeed. Keep finalize total so runBash's
|
||||
// close handler still resolves, but stop advertising a spill file that
|
||||
// may be missing its tail.
|
||||
// A delayed writeback failure makes the spill unreliable; keep finalize
|
||||
// total but stop advertising that file.
|
||||
this.spillFile = undefined
|
||||
}
|
||||
this.spillFd = undefined
|
||||
@@ -258,13 +231,9 @@ export class OutputCollector {
|
||||
}
|
||||
|
||||
/**
|
||||
* Send `sig` to the process GROUP led by `pid` (requires the child to have
|
||||
* been spawned with `detached: true`). NEVER throws: kills race process exit
|
||||
* by design (ESRCH), and the other failure modes (EPERM from setuid
|
||||
* children, …) fire inside timer callbacks where a throw would crash the
|
||||
* host process — a kill that cannot be delivered is reported by the process
|
||||
* NOT dying, which callers already handle via escalation/timeouts. No-op for
|
||||
* non-positive pids (spawn never started a process).
|
||||
* Send `sig` to a detached process group. Never throws: delivery races process
|
||||
* exit and may run in a timer callback, so failures are contained and a
|
||||
* non-positive pid is a no-op.
|
||||
* @param pid - the group leader's pid; non-positive means the spawn failed and the call is a no-op.
|
||||
* @param sig - the signal to deliver to the whole group.
|
||||
*/
|
||||
@@ -294,24 +263,13 @@ export interface RunningBash {
|
||||
}
|
||||
|
||||
/**
|
||||
* Spawn `bash -c <command>` in its own process group and collect output.
|
||||
*
|
||||
* Outcome semantics: the returned promise REJECTS only for spawn-level
|
||||
* failures (bad cwd → ENOENT, missing binary, pre-aborted signal); every
|
||||
* runtime outcome — nonzero exit, timeout kill, abort kill, signal death —
|
||||
* RESOLVES with a {@link SpawnOutcome} describing what happened, so callers
|
||||
* shape one consistent report for the model.
|
||||
*
|
||||
* XXX(stateful-shell): per the agent-tool survey there are two proven
|
||||
* stateful designs worth revisiting — Claude Code persists ONLY cwd between
|
||||
* calls (captures `pwd -P` after each command), and Codex keeps whole PTY
|
||||
* exec sessions addressable via session ids + stdin writes. We deliberately
|
||||
* spawn a fresh non-login `bash -c` per call for determinism (no rc files,
|
||||
* no inherited shell state); revisit when real workflows demand it.
|
||||
* @param spec - the fully-resolved run (command, cwd, limits); no defaulting happens here.
|
||||
* @param internals - test-only knobs; omitted fields fall back to the private per-process spill dir.
|
||||
* @returns the live handle: pid, the two live collectors, the outcome promise, and `kill()`.
|
||||
* Spawn one isolated `bash -c` process group and collect its output.
|
||||
* Runtime exits resolve as {@link SpawnOutcome}; only spawn failures reject.
|
||||
* @param spec - fully resolved command, cwd, limits, and cancellation.
|
||||
* @param internals - test-only process and spill-directory overrides.
|
||||
* @returns live process handle and outcome promise.
|
||||
*/
|
||||
// XXX(stateful-shell): evaluate persistent cwd or PTY sessions when workflows require shell state.
|
||||
export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningBash {
|
||||
const spillDir = internals.spillDir ?? privateSpillDir()
|
||||
|
||||
@@ -319,16 +277,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
|
||||
throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`)
|
||||
}
|
||||
|
||||
// stdin is a pipe ONLY when the caller supplied bytes; with none it is `ignore`
|
||||
// (fd 0 → /dev/null) — the exact pre-seam default. This matters: a spawn pipe
|
||||
// and /dev/null are NOT observationally identical (node's pipe is an AF_UNIX
|
||||
// socket, so a command that probes stdin's type — `test -c /dev/stdin`, `stat
|
||||
// /proc/self/fd/0` — sees a char device vs a socket), so the no-stdin path
|
||||
// (every model-driven call) must keep /dev/null rather than regress to a socket.
|
||||
// Two LITERAL `stdio` tuples (not one variable tuple): only a literal lets the
|
||||
// typed `spawn` overload infer non-null stdout/stderr, which the
|
||||
// `ChildProcessByStdio` annotation captures (stdin `Writable | null`; stdout/
|
||||
// stderr the non-null `Readable` the collectors attach to without a cast).
|
||||
// Keep absent stdin as /dev/null; literal tuples preserve non-null output types.
|
||||
const env = childEnv(spec.env)
|
||||
const child: ChildProcessByStdio<Writable | null, Readable, Readable> = spec.stdin !== undefined
|
||||
? spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['pipe', 'pipe', 'pipe'], detached: true })
|
||||
@@ -341,8 +290,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
|
||||
|
||||
let graceTimer: NodeJS.Timeout | undefined
|
||||
|
||||
// pid is undefined when the spawn itself fails (bad cwd, missing binary);
|
||||
// the 'error' handler rejects `done` and kills become no-ops via pid -1.
|
||||
// Failed spawns use pid -1 so kill remains a no-op.
|
||||
const pid = child.pid ?? -1
|
||||
|
||||
const kill = (): void => {
|
||||
@@ -351,27 +299,11 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
|
||||
graceTimer = setTimeout(() => { killGroup(pid, 'SIGKILL') }, spec.graceMs)
|
||||
}
|
||||
|
||||
// runBash owns no timer: the executor's `run()` fuses timeout+cancel into one
|
||||
// deadline signal (`@deepseek-ai/dsh-timeout`) and passes it here; we only
|
||||
// listen and run the SIGTERM→grace→SIGKILL kill. Whether the abort was a
|
||||
// timeout or an upstream cancel is classified by the executor from that
|
||||
// signal, not tracked here.
|
||||
// The executor owns timeout classification; this layer only reacts to abort.
|
||||
const onAbort = (): void => { kill() }
|
||||
spec.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
|
||||
// Write stdin and close it, but ONLY when the caller supplied bytes — with no
|
||||
// stdin, fd 0 is `ignore` (/dev/null) and `child.stdin` is null. The error
|
||||
// handler must exist whenever we write: an unhandled 'error' on the stream
|
||||
// would throw and crash the host. We swallow the error rather than reject
|
||||
// `done`, and that is correct for ANY stdin-write error, not just the common
|
||||
// one — the stdin write is BEST-EFFORT, while the command's authoritative
|
||||
// outcome is its exit code + captured output, which the `close` handler reports
|
||||
// regardless of whether the write landed. The expected case is EPIPE (the child
|
||||
// exited without reading, so closing our end of a still-full pipe fails); a rare
|
||||
// non-EPIPE pipe fault means the command ran with incomplete stdin, and it
|
||||
// surfaces that itself through its own exit/output (e.g. a hook that gets
|
||||
// truncated JSON errors out) — rejecting here would instead discard that real
|
||||
// output and turn it into an opaque infrastructure error, which is worse.
|
||||
// Stdin writes are best-effort; process exit and captured output remain authoritative.
|
||||
if (child.stdin !== null) {
|
||||
child.stdin.on('error', () => { /* stdin write is best-effort; outcome rides on exit/output. */ })
|
||||
child.stdin.end(spec.stdin)
|
||||
@@ -379,8 +311,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
|
||||
|
||||
const done = new Promise<SpawnOutcome>((resolve, reject) => {
|
||||
child.on('error', (error) => {
|
||||
// Spawn-level failure (ENOENT cwd, EACCES, …): no close event with
|
||||
// meaningful output follows; clean up and reject.
|
||||
// No meaningful close outcome follows a spawn failure.
|
||||
cleanup()
|
||||
reject(error)
|
||||
})
|
||||
|
||||
@@ -337,7 +337,7 @@ describe('LocalBashExecutor background tasks', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes: lifecycle hardening', () => {
|
||||
describe('executor cancellation, callback, and disposal contracts', () => {
|
||||
it('start honors a pre-aborted or later-aborted AbortSignal', async () => {
|
||||
const { bash } = await setup()
|
||||
const controller = new AbortController()
|
||||
|
||||
@@ -189,12 +189,8 @@ describe('stdin and extra env (set by in-process plugins)', () => {
|
||||
})
|
||||
|
||||
it('gives fd 0 the exact pre-seam type: /dev/null when no stdin, a pipe when supplied', async () => {
|
||||
// The no-stdin path must stay observationally identical to the pre-seam
|
||||
// `ignore` default: a command that probes stdin's file type sees a char
|
||||
// device (/dev/null). Regressing to an always-open pipe would make fd 0 a
|
||||
// socket (node's spawn pipe is an AF_UNIX socket, not a FIFO), flipping
|
||||
// `test -c /dev/stdin` for every model-driven call. When bytes ARE supplied,
|
||||
// fd 0 is that pipe (a socket), as it must be to carry them.
|
||||
// With no bytes, fd 0 remains the pre-seam `ignore` default (/dev/null, a character device).
|
||||
// Supplied bytes use Node's spawn pipe, which is an AF_UNIX socket rather than a FIFO.
|
||||
const none = await runBash(spec('test -c /dev/stdin && echo char || echo other')).done
|
||||
expect(none.stdout.text).toBe('char\n')
|
||||
const piped = await runBash(spec('test -S /dev/stdin && echo socket || echo other', { stdin: 'x' })).done
|
||||
@@ -219,9 +215,8 @@ describe('stdin and extra env (set by in-process plugins)', () => {
|
||||
})
|
||||
|
||||
it('does not crash or reject when the child ignores a large stdin (EPIPE)', async () => {
|
||||
// The child exits immediately without reading; closing our end of a stdin
|
||||
// pipe still holding ~1MiB triggers EPIPE on the write. The handler must
|
||||
// swallow it: `done` resolves normally with the child's real exit.
|
||||
// The child exits without reading, so closing a stdin pipe holding ~1 MiB triggers EPIPE.
|
||||
// The handler swallows that write error and `done` reports the child's real exit.
|
||||
const big = 'x'.repeat(1024 * 1024)
|
||||
const result = await runBash(spec('exit 7', { stdin: big })).done
|
||||
expect(result.exitCode).toBe(7)
|
||||
@@ -352,7 +347,7 @@ describe('abort edge cases', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('review fixes: env scrubbing and spill hardening', () => {
|
||||
describe('environment and spill-file hardening', () => {
|
||||
it('scrubs credential-shaped env vars from child processes', async () => {
|
||||
process.env.DSH_TEST_API_KEY = 'super-secret'
|
||||
process.env.DSH_TEST_TOKEN = 'also-secret'
|
||||
|
||||
Reference in New Issue
Block a user