From 8b5a3ef7300e5b100c66268f9c761c660172b298 Mon Sep 17 00:00:00 2001 From: Tianyi Cui <53024+tianyicui@users.noreply.github.com> Date: Fri, 12 Jun 2026 23:28:44 +0800 Subject: [PATCH] Add bash execution: dsh-bash seam, dsh-bash-local impl, dsh-tool-bash tools MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three packages following the new capability-seam pattern (interface / implementation / consumer, now documented in docs/architecture.md): - dsh-bash: abstract BashExecutor service (ctx.bash) + vocabulary types. - dsh-bash-local: local subprocesses — bash -c per call in a detached process group, SIGTERM→SIGKILL group kills, tail-keep truncation with full-stream spill files, model-friendly env, background task registry. - dsh-tool-bash: the bash / bash_output / bash_kill tool schemas with runtime arg validation and background completion notices via agent.inject(). Non-zero exits are reported, not errored. Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi (notes in the package READMEs). Permissions/sandbox stay TODO on the tools/execute waterfall seam; stateful-shell alternatives recorded in run.ts. --- .gitignore | 1 + AGENTS.md | 22 + docs/architecture.md | 47 ++- packages/bash-local/README.md | 51 +++ packages/bash-local/package.json | 33 ++ packages/bash-local/src/index.ts | 204 ++++++++++ packages/bash-local/src/run.ts | 329 +++++++++++++++ packages/bash-local/tests/executor.spec.ts | 297 ++++++++++++++ packages/bash-local/tests/run.spec.ts | 295 ++++++++++++++ packages/bash-local/tsconfig.json | 14 + packages/bash/README.md | 42 ++ packages/bash/package.json | 28 ++ packages/bash/src/index.ts | 131 ++++++ packages/bash/src/types.ts | 97 +++++ packages/bash/tests/service.spec.ts | 138 +++++++ packages/bash/tsconfig.json | 12 + packages/tool-bash/README.md | 62 +++ packages/tool-bash/package.json | 40 ++ packages/tool-bash/src/index.ts | 226 ++++++++++ packages/tool-bash/tests/integration.spec.ts | 164 ++++++++ packages/tool-bash/tests/tools.spec.ts | 407 +++++++++++++++++++ packages/tool-bash/tsconfig.json | 16 + scripts/publint-all.ts | 3 + tsconfig.base.json | 5 +- tsconfig.build.json | 5 +- tsconfig.typecheck.json | 5 +- yarn.lock | 47 ++- 27 files changed, 2714 insertions(+), 7 deletions(-) create mode 100644 packages/bash-local/README.md create mode 100644 packages/bash-local/package.json create mode 100644 packages/bash-local/src/index.ts create mode 100644 packages/bash-local/src/run.ts create mode 100644 packages/bash-local/tests/executor.spec.ts create mode 100644 packages/bash-local/tests/run.spec.ts create mode 100644 packages/bash-local/tsconfig.json create mode 100644 packages/bash/README.md create mode 100644 packages/bash/package.json create mode 100644 packages/bash/src/index.ts create mode 100644 packages/bash/src/types.ts create mode 100644 packages/bash/tests/service.spec.ts create mode 100644 packages/bash/tsconfig.json create mode 100644 packages/tool-bash/README.md create mode 100644 packages/tool-bash/package.json create mode 100644 packages/tool-bash/src/index.ts create mode 100644 packages/tool-bash/tests/integration.spec.ts create mode 100644 packages/tool-bash/tests/tools.spec.ts create mode 100644 packages/tool-bash/tsconfig.json diff --git a/.gitignore b/.gitignore index ae91a454b8..20b383e21c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ CLAUDE.local.md +.env node_modules/ lib/ *.tsbuildinfo diff --git a/AGENTS.md b/AGENTS.md index 4df48812fc..f8f974ec38 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,6 +27,9 @@ packages/ Harness packages, all named @deepseek-ai/dsh-: tools/ tool registry + tools/execute waterfall agent/ Agent interface, registry, agent/* event vocabulary agent-loop/ THE concrete plugin: LoopAgent + the loop driver + bash/ abstract bash executor seam (ctx.bash) — interface only + bash-local/ local-subprocess BashExecutor implementation + tool-bash/ model-facing bash/bash_output/bash_kill tool schemas examples/ Runnable demos (not workspaces). echo-agent = mock model + echo tool + stdio UI + JSONL persistence, wired via cordis.yml. docs/ architecture.md — the design doc. adr/ — decision records (the @@ -86,6 +89,25 @@ unresolved-type `no-unsafe-*` errors. - **Plugins, not loop changes**: new behavior goes into a plugin on the documented extension seams (see the plugin sanity checklist in docs/architecture.md). Changing `agent-loop` requires updating that doc. +- **Capability seams are three packages**: when adding a swappable capability + (an execution backend, a provider integration, …), split it into + *interface* (abstract service + vocabulary types, e.g. `bash/`), + *implementation* (a concrete subclass, e.g. `bash-local/`), and + *consumer* (what the model/plugins see, e.g. `tool-bash/`). Implementations + and consumers then evolve independently — a sandboxed executor replaces + `bash-local` without touching tool schemas. The LLM seam follows the same + shape (`llm/` is interface + consumer surface; adapters are implementations). + See docs/architecture.md § "Capability seams" for when NOT to split. +- **Explicit > implicit at package seams**: interface/vocabulary types spell + out every field a consumer must supply — no optional field that the + implementation silently fills with a hidden `?? default`. Put defaulting in + the owning implementation as an explicit step (a `resolve(request): Spec` + method that turns the optional-field request into the required-field spec), + not smuggled inside `run()`/`start()`. Example: `dsh-bash` splits + `BashExecRequest` (optional `workdir`/`timeoutMs`, model-facing) from + `BashExecSpec` (required, what `run`/`start` act on); the tool layer calls + `ctx.bash.resolve()` between them. The reader of a `BashExecSpec` never has + to wonder where the working directory came from. - **Tests**: vitest, colocated under `packages//tests/*.spec.ts`. Every registry needs an HMR-safety test (dispose the contributing fiber, assert cleanup). **Excessive tests are welcome** — when in doubt, write the test; diff --git a/docs/architecture.md b/docs/architecture.md index 150f6acace..8e8ca19dd1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -21,15 +21,18 @@ Requirement context: [Coding Harness MVP 需求分析][mvp-doc]. ``` ┌─────────────────────────────────────────────────────────────┐ -│ future plugins: tools, hooks, compaction, sandbox, UI, MCP… │ +│ future plugins: hooks, compaction, sandbox, UI, MCP… │ ├─────────────────────────────────────────────────────────────┤ │ @deepseek-ai/dsh-agent-loop (the ONE concrete plugin) │ +│ @deepseek-ai/dsh-bash-local (bash impl) │ +│ @deepseek-ai/dsh-tool-bash (bash tool schemas) │ ├─────────────────────────────────────────────────────────────┤ │ @deepseek-ai/dsh-agent (vocabulary + registry) │ │ @deepseek-ai/dsh-tools (registry + exec waterfall)│ │ @deepseek-ai/dsh-system-prompt (assembly registry) │ │ @deepseek-ai/dsh-session (event-sourced log) │ │ @deepseek-ai/dsh-llm (abstract model service) │ +│ @deepseek-ai/dsh-bash (abstract bash executor) │ ├─────────────────────────────────────────────────────────────┤ │ vendor/: cordis, loader, include, group, timer, hmr, │ │ logger-console, cosmokit, schemastery │ @@ -50,11 +53,49 @@ working against the `dsh-agent` vocabulary if the loop is replaced. | `ctx.tools` | `ToolRegistry` | dsh-tools | tool definitions; `execute()` through waterfall | | `ctx.agents` | `AgentRegistry` | dsh-agent | live `Agent` handles | | `ctx.agentLoop` | `AgentLoop` | dsh-agent-loop | creates `LoopAgent`s and drives their loops | +| `ctx.bash` | `BashExecutor` (abstract) | dsh-bash | bash execution seam: foreground runs + background tasks | All registrations (`registerAdapter`, `section`, `tools`, `register`, …) go through `ctx.effect()` and return disposers, so plugin hot-reload (vendored HMR) and fiber disposal clean up automatically. +## Capability seams: interface / implementation / consumer + +Swappable capabilities are split into **three packages** so each part evolves +independently. The bash capability is the template: + +1. **Interface** (`dsh-bash`) — an abstract service plus the vocabulary types + (`BashExecutor`, `BashRunResult`, `BashTask`, …). Defines the contract, + owns the `ctx.bash` key, depends only on cordis. +2. **Implementation** (`dsh-bash-local`) — a concrete subclass loaded as a + plugin (local subprocesses, process-group kills, spill-file truncation). + Sandboxed, containerized, or remote backends are sibling packages + implementing the same interface. +3. **Consumer** (`dsh-tool-bash`) — what the model and other plugins program + against (the `bash`/`bash_output`/`bash_kill` tool schemas). Consumers + `inject` the interface's ctx key and never import implementation types. + +The LLM seam has the same topology folded differently: `dsh-llm` carries the +interface (`LlmAdapter`) AND the consumer surface (`ctx.llm.stream()`), with +adapters as implementation packages — there the consumer is the loop itself, +not a swappable schema surface. Use the full three-package split when the +consumer is independently replaceable; keep interface + consumer together +when they are one concern. Don't split preemptively: a capability with one +conceivable implementation and one consumer stays one package until proven +otherwise. + +> **"Capability" — two unrelated meanings.** (1) The *seam pattern* above +> ("one plugin provides a capability, another needs it") is realized by +> plain Cordis **services + `inject`**: a provider registers a service +> (`ctx.bash`, declared in `interface Context`); a consumer declares +> `inject: ['bash']` and its fiber stays pending until the service exists, +> tearing down via HMR if it later vanishes. No extra library is needed. +> (2) `@cordisjs/plugin-capability` is a different axis entirely — a +> **permission/capability-security** service (named permissions with +> inheritance/dependency, tested against a session via `ctx.capability.test`). +> It is a candidate for the deferred permissions/sandbox work (the +> `tools/execute` veto seam), NOT a mechanism for swapping implementations. + ## The vocabulary (dsh-llm) Messages are arrays of typed **content blocks** (`text`, `reasoning`, @@ -240,9 +281,9 @@ implements it **without modifying the loop**: | System prompt configurability | `ctx.systemPrompt.section()` with ordering | | AGENTS.md (root) | a section provider reading the file | | AGENTS.md (subdir, on-touch) + file-change notices | `agent.inject()` from a watcher / tool-result listener | -| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically | +| Built-in tools (Read/Write/Edit/Bash/…) | `ctx.tools.register()`; schemas flow into the assembly automatically. **Bash: implemented** — `dsh-bash` (seam) + `dsh-bash-local` (subprocesses) + `dsh-tool-bash` (`bash`/`bash_output`/`bash_kill`, incl. background tasks) | | ToolSearch / progressive disclosure | wrap `agent/request`, filter `req.tools` | -| Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute` | +| Tool sandbox (landlock / sandbox-exec) | wrap `tools/execute`, or implement a sandboxing `BashExecutor` (the dsh-bash seam) | | Permission system / AskUserQuestion | wrap `tools/execute` (veto or ask); register an ask tool | | Plan mode | wrap `tools/execute` (deny writes) + `agent/request` (inject mode prompt) | | Sub-agents (spawn / fork / steer) | TODO seam on `AgentLoop.create()`; fork = seed Session with parent events; `steer()` on the child handle | diff --git a/packages/bash-local/README.md b/packages/bash-local/README.md new file mode 100644 index 0000000000..3efab3ecfd --- /dev/null +++ b/packages/bash-local/README.md @@ -0,0 +1,51 @@ +# @deepseek-ai/dsh-bash-local + +Local-subprocess implementation of the `@deepseek-ai/dsh-bash` executor seam: +`LocalBashExecutor` spawns `bash -c ` per call in its own process +group, collects bounded output with full-stream spill files, and escalates +kills SIGTERM→SIGKILL across the whole group. + +## Config + +```yaml +- id: bash + name: '@deepseek-ai/dsh-bash-local' + config: + cwd: /path/to/workspace # default: process.cwd() + timeoutMs: 120000 # default foreground timeout + maxTimeoutMs: 600000 # cap for per-call overrides + maxOutputBytes: 64000 # per-stream in-memory cap; overflow spills to disk +``` + +## Behavior (and where it came from) + +Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and +pi; the notable choices: + +- **Spawn per call, no shell state** — every call is a fresh non-login + `bash -c` (deterministic; no rc files). All four surveyed tools spawn per + call. `TODO(stateful-shell)` in `src/run.ts` records the two proven + stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec + sessions) for when real workflows demand them. +- **Process-group kills with escalation** — children are spawned `detached` + (own process group); kills send SIGTERM to the group, then SIGKILL after a + 3s grace (OpenCode's escalation; pipelines and subshells die with the + parent). ESRCH is tolerated; daemons that re-parent away from the group can + still survive — same caveat as the surveyed tools. +- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` + keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode + rationale) while the FULL stream is appended to a temp file whose path is + reported. The model can `grep`/`tail` the spill file with bash itself. +- **Model-friendly env** — `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat` + (Codex's hardcoded set) so pagers and ANSI color don't garble results. +- **Background tasks** — `start()` returns immediately, no timeout applies + (Claude Code detaches timeouts when backgrounding), `readOutput()` is + incremental with whole-stream byte offsets, and disposal kills everything. + +## Sandboxing + +`TODO(permissions/sandbox)`: execution policy does NOT belong in this +package. Wrap the `tools/execute` waterfall (veto/ask) or implement a +sandboxing `BashExecutor` — see docs/architecture.md § plugin checklist. +Reference points: Claude Code wraps commands in sandbox-exec/bubblewrap; +Codex applies seatbelt/landlock plus an execpolicy prefix-rule engine. diff --git a/packages/bash-local/package.json b/packages/bash-local/package.json new file mode 100644 index 0000000000..b21b139485 --- /dev/null +++ b/packages/bash-local/package.json @@ -0,0 +1,33 @@ +{ + "name": "@deepseek-ai/dsh-bash-local", + "description": "Local-subprocess implementation of the DeepSeek Harness bash executor seam", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "exports": { + ".": { + "types": "./lib/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-bash": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "dependencies": { + "schemastery": "^3.18.0" + }, + "devDependencies": { + "@deepseek-ai/dsh-bash": "^0.0.1", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/bash-local/src/index.ts b/packages/bash-local/src/index.ts new file mode 100644 index 0000000000..1169160614 --- /dev/null +++ b/packages/bash-local/src/index.ts @@ -0,0 +1,204 @@ +/** + * `LocalBashExecutor`: the local-subprocess implementation of the + * `@deepseek-ai/dsh-bash` executor seam. Spawns `bash -c` per call in its + * own process group (see `./run.ts` for the plumbing and the agent-tool + * survey notes), tracks background tasks, and kills everything on dispose. + * + * TODO(permissions/sandbox): execution policy does NOT belong here — wrap + * the `tools/execute` waterfall (see docs/architecture.md § plugin + * checklist) or implement a sandboxing `BashExecutor`. Reference points: + * Claude Code wraps commands in sandbox-exec/bubblewrap; Codex applies + * seatbelt/landlock plus an execpolicy prefix-rule engine. + * + * @module @deepseek-ai/dsh-bash-local + */ + +import { Context } from 'cordis' +import z from 'schemastery' +import { BashExecutor } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead } from '@deepseek-ai/dsh-bash' +import { runBash } from './run.ts' +import type { RunInternals, RunningBash } from './run.ts' + +export { DEFAULT_GRACE_MS, ENV_OVERRIDES, killGroup, OutputCollector, runBash } from './run.ts' +export type { RunInternals, RunningBash, SpawnOutcome, SpawnSpec } from './run.ts' + +/** Plugin config (all optional — `static Config` supplies the defaults). */ +export interface Config { + /** Default working directory for commands (default: process.cwd()). */ + cwd?: string + /** Default foreground timeout in milliseconds. */ + timeoutMs?: number + /** Upper bound for per-call timeout overrides. */ + maxTimeoutMs?: number + /** Per-stream in-memory output cap; overflow spills to a temp file. */ + maxOutputBytes?: number +} + +/** The shape after schemastery applied the defaults (cwd has none). */ +type ResolvedConfig = Required> & Pick + +interface TrackedTask extends BashTask { + running: RunningBash + /** Whole-stream byte offsets already delivered via {@link LocalBashExecutor.readOutput}. */ + stdoutOffset: number + stderrOffset: number +} + +/** + * Local-subprocess bash executor. Defaults follow the agent-tool survey + * consensus: 120s default / 600s max timeout (Claude Code, OpenCode), 64KB + * in-memory output with full-stream spill files (pi, OpenCode), + * process-group SIGTERM→SIGKILL kills (OpenCode). + */ +export class LocalBashExecutor extends BashExecutor { + static Config: z = z.object({ + cwd: z.string(), + timeoutMs: z.number().default(120_000), + maxTimeoutMs: z.number().default(600_000), + maxOutputBytes: z.number().default(64_000), + }) + + private tasks = new Map() + private nextTaskId = 1 + /** Test seam: timer/spill knobs forwarded to runBash. */ + internals: RunInternals = {} + + /** Validated config (schemastery applied the defaults before construction). */ + readonly config: ResolvedConfig + + constructor(ctx: Context, config: Config) { + super(ctx) + // schemastery (static Config) has already filled the defaulted fields; + // the cast records that runtime fact for exactOptionalPropertyTypes. + this.config = config as ResolvedConfig + ctx.effect(() => async () => { + // Kill every live process group and WAIT for the processes to close so + // nothing outlives the fiber (HMR safety) — a TERM-trapping child is + // held until the SIGKILL escalation lands. The base class already + // silenced listeners, so these kills complete without notices. + const pending: Promise[] = [] + for (const task of this.tasks.values()) { + if (task.status === 'running') { + task.status = 'killed' + task.running.kill() + pending.push(task.done) + } + } + this.tasks.clear() + await Promise.all(pending) + }, 'local bash teardown') + } + + /** + * Resolve a request into a fully-specified spec: fill `workdir` from + * `config.cwd` (else `process.cwd()`), and `timeoutMs` from + * `config.timeoutMs`, capped at `config.maxTimeoutMs`. The tool layer calls + * this before {@link run}/{@link start}, so those methods receive explicit + * values and never re-default. + */ + resolve(request: BashExecRequest): BashExecSpec { + const timeoutMs = Math.min(request.timeoutMs ?? this.config.timeoutMs, this.config.maxTimeoutMs) + return { + command: request.command, + workdir: request.workdir ?? this.config.cwd ?? process.cwd(), + timeoutMs, + ...request.signal ? { signal: request.signal } : {}, + } + } + + async run(spec: BashExecSpec): Promise { + const outcome = await runBash({ + command: spec.command, + cwd: spec.workdir, + timeoutMs: spec.timeoutMs, + maxOutputBytes: this.config.maxOutputBytes, + signal: spec.signal, + }, this.internals).done + return { ...outcome, timeoutMs: spec.timeoutMs } + } + + start(spec: BashExecSpec): BashTask { + // No timeout for background tasks (matches Claude Code, which detaches + // the timeout when backgrounding); callers stop tasks via kill() — or + // via spec.signal, which the seam contract honors for background runs + // too (runBash wires it to the group kill). spec.timeoutMs is ignored + // here by design. + const running = runBash({ + command: spec.command, + cwd: spec.workdir, + timeoutMs: 0, + maxOutputBytes: this.config.maxOutputBytes, + signal: spec.signal, + }, this.internals) + + const id = `bash-${this.nextTaskId++}` + const task: TrackedTask = { + id, + command: spec.command, + status: 'running', + exitCode: null, + signal: null, + running, + stdoutOffset: 0, + stderrOffset: 0, + done: running.done.then((outcome) => { + // Abort-killed tasks report as killed, not completed. + if (task.status === 'running') task.status = outcome.aborted ? 'killed' : 'completed' + task.exitCode = outcome.exitCode + task.signal = outcome.signal + this.notifyTaskDone(task) + }, (error: unknown) => { + // Spawn-level failure (bad workdir, …): the task never ran. String() + // suffices — runBash only rejects with Error instances. + task.status = 'killed' + task.running.stderr.push(Buffer.from(`spawn failed: ${String(error)}`)) + this.notifyTaskDone(task) + }), + } + this.tasks.set(id, task) + return task + } + + get(id: string): BashTask | undefined { + return this.tasks.get(id) + } + + list(): BashTask[] { + return [...this.tasks.values()] + } + + readOutput(id: string): BashTaskRead { + const task = this.tasks.get(id) + if (!task) throw new Error(`unknown bash task "${id}"`) + + const out = task.running.stdout.readFrom(task.stdoutOffset) + const err = task.running.stderr.readFrom(task.stderrOffset) + task.stdoutOffset = out.nextOffset + task.stderrOffset = err.nextOffset + + // Single newline between sections: stdout chunks usually end with one + // already; add it only when missing. + const separator = out.text.length > 0 && !out.text.endsWith('\n') ? '\n' : '' + const delta = out.text + + (err.text.length > 0 ? `${separator}[stderr]\n${err.text}` : '') + return { + task, + delta, + lossy: out.lossy || err.lossy, + ...out.spillPath !== undefined ? { stdoutSpillPath: out.spillPath } : {}, + ...err.spillPath !== undefined ? { stderrSpillPath: err.spillPath } : {}, + } + } + + kill(id: string): boolean { + const task = this.tasks.get(id) + if (!task) throw new Error(`unknown bash task "${id}"`) + if (task.status !== 'running') return false + task.status = 'killed' + task.running.kill() + return true + } +} + +export default LocalBashExecutor diff --git a/packages/bash-local/src/run.ts b/packages/bash-local/src/run.ts new file mode 100644 index 0000000000..93551c8e6b --- /dev/null +++ b/packages/bash-local/src/run.ts @@ -0,0 +1,329 @@ +/** + * Process plumbing for the local bash executor: spawn, output collection + * with tail-keep + spill-to-disk truncation, and process-group kill with + * SIGTERM→SIGKILL escalation. + * + * Everything here is deliberately free of Cordis concepts so it can be unit + * tested in isolation; `LocalBashExecutor` owns lifecycle and configuration. + * + * Design notes (surveyed against Claude Code, OpenCode, Codex, and pi — see + * the package README): spawn-per-call with `detached: true` so the child + * leads its own process group; kills target the group (`kill(-pid)`) so + * pipelines and subshells die with the parent. SIGTERM first, SIGKILL after a + * grace period (OpenCode's escalation; Codex/pi jump straight to SIGKILL). + * + * @module dsh-bash-local/run + */ + +import { spawn } from 'node:child_process' +import { randomBytes } from 'node:crypto' +import { closeSync, mkdtempSync, openSync, writeSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import type { CollectedOutput } from '@deepseek-ai/dsh-bash' + +/** + * Model-friendly environment overrides: disable colors, pagers, and + * interactive terminal features that would garble tool output (the same set + * Codex hardcodes; Claude Code achieves it via TERM=dumb). + */ +export const ENV_OVERRIDES = { + NO_COLOR: '1', + TERM: 'dumb', + PAGER: 'cat', + GIT_PAGER: 'cat', +} as const + +/** + * Credential-shaped env vars are NOT forwarded to commands (the harness's + * own DEEPSEEK_API_KEY must not leak into `env` output, tool results, or + * spill files). Same default pattern as Codex's env policy; a future config + * can whitelist specific vars when a workflow genuinely needs one. + */ +export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i + +/** process.env minus credential-shaped vars, plus the model-friendly overrides. */ +export function childEnv(): NodeJS.ProcessEnv { + const env: NodeJS.ProcessEnv = {} + for (const [key, value] of Object.entries(process.env)) { + if (!SENSITIVE_ENV_PATTERN.test(key)) env[key] = value + } + return { ...env, ...ENV_OVERRIDES } +} + +/** What to run and under which limits (resolved — no defaults in here). */ +export interface SpawnSpec { + command: string + cwd: string + /** Kill the process group after this many milliseconds. 0 = no timeout. */ + timeoutMs: number + /** Per-stream in-memory cap; overflow spills to disk (tail kept in memory). */ + maxOutputBytes: number + /** Abort signal — kills the process group when fired. */ + signal?: AbortSignal | undefined +} + +/** Raw outcome of one closed process (before result shaping). */ +export interface SpawnOutcome { + exitCode: number | null + signal: NodeJS.Signals | null + timedOut: boolean + aborted: boolean + stdout: CollectedOutput + stderr: CollectedOutput +} + +/** Injectable knobs so tests can exercise escalation/spill without long waits. */ +export interface RunInternals { + /** Grace period between SIGTERM and SIGKILL on the process group. */ + graceMs?: number + /** Directory for spill files (defaults to the OS temp dir). */ + spillDir?: string +} + +/** Default SIGTERM→SIGKILL grace period (matches OpenCode's 3s). */ +export const DEFAULT_GRACE_MS = 3_000 + +let spillCounter = 0 +let defaultSpillDir: string | undefined + +/** + * The default spill location: a private (0700) per-process directory under + * the OS tmpdir, created lazily. Predictable world-readable paths would let + * other local users read command output or pre-create symlinks. + */ +function privateSpillDir(): string { + defaultSpillDir ??= mkdtempSync(join(tmpdir(), 'dsh-bash-')) + return defaultSpillDir +} + +/** + * Collects one stream with a bounded in-memory tail. The FULL stream is + * always recoverable: on first overflow a spill file is created and every + * chunk (including those already collected) is appended there. + * + * Tail-keep rationale (pi/OpenCode): errors and final results cluster at the + * end of command output; the spill file covers the head. + */ +export class OutputCollector { + private chunks: Buffer[] = [] + private bytes = 0 + private dropped = false + private spillFd: number | undefined + private spillFile: string | undefined + /** Total bytes ever pushed (not just retained). */ + private total = 0 + + constructor( + private readonly maxBytes: number, + private readonly label: string, + private readonly spillDir: string, + ) {} + + push(chunk: Buffer): void { + this.total += chunk.length + const overflows = this.bytes + chunk.length > this.maxBytes + if (overflows || this.spillFd !== undefined) this.spillAll(chunk) + this.chunks.push(chunk) + this.bytes += chunk.length + while (this.bytes > this.maxBytes && this.chunks.length > 1) { + // Drop whole chunks from the head; pipe chunks are small (≤64KiB), so + // the retained tail tracks the cap closely enough for a model-facing + // truncation boundary. (length > 1 was just checked — shift() returns.) + const head = this.chunks.shift() as Buffer + this.bytes -= head.length + this.dropped = true + } + if (this.bytes > this.maxBytes && this.chunks.length === 1) { + // A single chunk larger than the cap: keep its tail. + const only = this.chunks[0] as Buffer + this.chunks[0] = only.subarray(only.length - this.maxBytes) + this.bytes = this.maxBytes + this.dropped = true + } + } + + /** Open the spill file lazily and append `chunk` (and any prior chunks once). */ + private spillAll(chunk: Buffer): void { + if (this.spillFd === undefined) { + // Random suffix + O_EXCL + no-follow-equivalent ('wx' fails on any + // existing path, symlink or not) + owner-only mode: defeats spill-path + // prediction and symlink planting in shared tmp dirs. + this.spillFile = join( + this.spillDir, + `dsh-bash-${process.pid}-${++spillCounter}-${randomBytes(6).toString('hex')}-${this.label}.log`, + ) + this.spillFd = openSync(this.spillFile, 'wx', 0o600) + for (const prior of this.chunks) writeSync(this.spillFd, prior) + } + writeSync(this.spillFd, chunk) + } + + /** Read the collected tail without finalizing (used by background polling). */ + snapshot(): CollectedOutput { + return { + text: Buffer.concat(this.chunks).toString('utf8'), + truncated: this.dropped, + ...this.spillFile !== undefined ? { spillPath: this.spillFile } : {}, + } + } + + /** Total bytes ever pushed (including bytes dropped from memory). */ + get totalBytes(): number { + return this.total + } + + /** + * Incremental read in whole-stream byte coordinates: returns everything + * pushed since `fromByte`. When `fromByte` has already slid out of the + * in-memory tail window, the read is `lossy` — it returns the whole + * retained tail and the gap is only recoverable from the spill file. + */ + readFrom(fromByte: number): { text: string; nextOffset: number; lossy: boolean; spillPath?: string } { + const windowStart = this.total - this.bytes + const buffer = Buffer.concat(this.chunks) + const lossy = fromByte < windowStart + const slice = lossy ? buffer : buffer.subarray(fromByte - windowStart) + return { + text: slice.toString('utf8'), + nextOffset: this.total, + lossy, + ...this.spillFile !== undefined ? { spillPath: this.spillFile } : {}, + } + } + + /** Close the spill file (if any) and return the final output. */ + finalize(): CollectedOutput { + if (this.spillFd !== undefined) { + closeSync(this.spillFd) + this.spillFd = undefined + } + return this.snapshot() + } +} + +/** + * Send `sig` to the process GROUP led by `pid` (requires the child to have + * been spawned with `detached: true`). NEVER throws: kills race process exit + * by design (ESRCH), and the other failure modes (EPERM from setuid + * children, …) fire inside timer callbacks where a throw would crash the + * host process — a kill that cannot be delivered is reported by the process + * NOT dying, which callers already handle via escalation/timeouts. No-op for + * non-positive pids (spawn never started a process). + */ +export function killGroup(pid: number, sig: NodeJS.Signals): void { + if (pid <= 0) return + try { + process.kill(-pid, sig) + } catch { + // Swallow: see contract above. + } +} + +/** + * A live bash child process: the promise resolves when the process closes; + * `kill()` starts the SIGTERM→grace→SIGKILL escalation on its group. + */ +export interface RunningBash { + /** Process id (group leader); -1 when the spawn itself failed. */ + readonly pid: number + /** stdout/stderr collectors (live — background polling reads incrementally). */ + readonly stdout: OutputCollector + readonly stderr: OutputCollector + /** Resolves when the process closes; rejects only for spawn-level failures. */ + readonly done: Promise + /** Begin SIGTERM→grace→SIGKILL on the process group. Idempotent. */ + kill(): void +} + +/** + * Spawn `bash -c ` in its own process group and collect output. + * + * Outcome semantics: the returned promise REJECTS only for spawn-level + * failures (bad cwd → ENOENT, missing binary, pre-aborted signal); every + * runtime outcome — nonzero exit, timeout kill, abort kill, signal death — + * RESOLVES with a {@link SpawnOutcome} describing what happened, so callers + * shape one consistent report for the model. + * + * TODO(stateful-shell): per the agent-tool survey there are two proven + * stateful designs worth revisiting — Claude Code persists ONLY cwd between + * calls (captures `pwd -P` after each command), and Codex keeps whole PTY + * exec sessions addressable via session ids + stdin writes. We deliberately + * spawn a fresh non-login `bash -c` per call for determinism (no rc files, + * no inherited shell state); revisit when real workflows demand it. + */ +export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningBash { + const graceMs = internals.graceMs ?? DEFAULT_GRACE_MS + const spillDir = internals.spillDir ?? privateSpillDir() + + if (spec.signal?.aborted) { + throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`) + } + + const child = spawn('bash', ['-c', spec.command], { + cwd: spec.cwd, + env: childEnv(), + stdio: ['ignore', 'pipe', 'pipe'], + detached: true, + }) + + const stdout = new OutputCollector(spec.maxOutputBytes, 'stdout', spillDir) + const stderr = new OutputCollector(spec.maxOutputBytes, 'stderr', spillDir) + child.stdout.on('data', (chunk: Buffer) => { stdout.push(chunk) }) + child.stderr.on('data', (chunk: Buffer) => { stderr.push(chunk) }) + + let timedOut = false + let aborted = false + let killTimer: NodeJS.Timeout | undefined + let graceTimer: NodeJS.Timeout | undefined + + // pid is undefined when the spawn itself fails (bad cwd, missing binary); + // the 'error' handler rejects `done` and kills become no-ops via pid -1. + const pid = child.pid ?? -1 + + const kill = (): void => { + if (graceTimer !== undefined) return // escalation already in flight + killGroup(pid, 'SIGTERM') + graceTimer = setTimeout(() => { killGroup(pid, 'SIGKILL') }, graceMs) + } + + if (spec.timeoutMs > 0) { + killTimer = setTimeout(() => { + timedOut = true + kill() + }, spec.timeoutMs) + } + + const onAbort = (): void => { + aborted = true + kill() + } + spec.signal?.addEventListener('abort', onAbort, { once: true }) + + const done = new Promise((resolve, reject) => { + child.on('error', (error) => { + // Spawn-level failure (ENOENT cwd, EACCES, …): no close event with + // meaningful output follows; clean up and reject. + cleanup() + reject(error) + }) + child.on('close', (exitCode, signal) => { + cleanup() + resolve({ + exitCode, + signal, + timedOut, + aborted, + stdout: stdout.finalize(), + stderr: stderr.finalize(), + }) + }) + function cleanup(): void { + if (killTimer !== undefined) clearTimeout(killTimer) + if (graceTimer !== undefined) clearTimeout(graceTimer) + spec.signal?.removeEventListener('abort', onAbort) + } + }) + + return { pid, stdout, stderr, done, kill } +} diff --git a/packages/bash-local/tests/executor.spec.ts b/packages/bash-local/tests/executor.spec.ts new file mode 100644 index 0000000000..72383d0427 --- /dev/null +++ b/packages/bash-local/tests/executor.spec.ts @@ -0,0 +1,297 @@ +import { mkdtempSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import type {} from '@deepseek-ai/dsh-bash' + +const spillDir = mkdtempSync(join(tmpdir(), 'dsh-bash-exec-spec-')) + +async function setup(config: ConstructorParameters[1] = {}) { + const ctx = new Context() + await ctx.plugin(LocalBashExecutor, config) + const bash = ctx.bash as LocalBashExecutor + bash.internals = { spillDir, graceMs: 200 } + return { ctx, bash } +} + +/** Poll until a pid no longer exists. */ +async function waitGone(pid: number, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + try { + process.kill(pid, 0) + } catch { + return + } + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) +} + +describe('LocalBashExecutor.run', () => { + it('resolves with output and the effective timeout', async () => { + const { bash } = await setup({ timeoutMs: 5_000 }) + const result = await bash.run(bash.resolve({ command: 'echo hi' })) + expect(result.exitCode).toBe(0) + expect(result.stdout.text).toBe('hi\n') + expect(result.timeoutMs).toBe(5_000) + }) + + it('uses config cwd, overridable per call', async () => { + const { bash } = await setup({ cwd: '/tmp' }) + const fromConfig = await bash.run(bash.resolve({ command: 'pwd' })) + expect(fromConfig.stdout.text.trim()).toMatch(/\/tmp$/) + const fromCall = await bash.run(bash.resolve({ command: 'pwd', workdir: '/' })) + expect(fromCall.stdout.text.trim()).toBe('/') + }) + + it('defaults cwd to process.cwd()', async () => { + const { bash } = await setup() + const result = await bash.run(bash.resolve({ command: 'pwd' })) + expect(result.stdout.text.trim()).toBe(process.cwd()) + }) + + it('caps per-call timeouts at maxTimeoutMs', async () => { + const { bash } = await setup({ timeoutMs: 1_000, maxTimeoutMs: 2_000 }) + const result = await bash.run(bash.resolve({ command: 'true', timeoutMs: 99_999 })) + expect(result.timeoutMs).toBe(2_000) + }) + + it('per-call timeout takes precedence under the cap and kills on expiry', async () => { + const { bash } = await setup({ timeoutMs: 60_000 }) + const result = await bash.run(bash.resolve({ command: 'sleep 60', timeoutMs: 100 })) + expect(result.timedOut).toBe(true) + expect(result.timeoutMs).toBe(100) + }) + + it('propagates abort signals', async () => { + const { bash } = await setup() + const controller = new AbortController() + const pending = bash.run(bash.resolve({ command: 'sleep 60', signal: controller.signal })) + setTimeout(() => { controller.abort() }, 50) + const result = await pending + expect(result.aborted).toBe(true) + }) + + it('rejects on spawn failure (bad workdir)', async () => { + const { bash } = await setup() + await expect(bash.run(bash.resolve({ command: 'true', workdir: '/nonexistent-dsh' }))).rejects.toThrow(/ENOENT/) + }) +}) + +describe('LocalBashExecutor background tasks', () => { + it('start returns immediately with a registered running task', async () => { + const { bash } = await setup() + const before = Date.now() + const task = bash.start(bash.resolve({ command: 'sleep 0.2; echo done' })) + expect(Date.now() - before).toBeLessThan(150) + expect(task.status).toBe('running') + expect(bash.get(task.id)).toBe(task) + expect(bash.list()).toContain(task) + await task.done + expect(task.status).toBe('completed') + expect(task.exitCode).toBe(0) + }) + + it('assigns sequential ids', async () => { + const { bash } = await setup() + const first = bash.start(bash.resolve({ command: 'true' })) + const second = bash.start(bash.resolve({ command: 'true' })) + expect(first.id).toBe('bash-1') + expect(second.id).toBe('bash-2') + await Promise.all([first.done, second.done]) + }) + + it('readOutput returns increments without re-delivery', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'echo first; sleep 0.3; echo second' })) + await new Promise(resolve => setTimeout(resolve, 150)) + const first = bash.readOutput(task.id) + expect(first.delta).toBe('first\n') + expect(first.lossy).toBe(false) + await task.done + const second = bash.readOutput(task.id) + expect(second.delta).toBe('second\n') + const third = bash.readOutput(task.id) + expect(third.delta).toBe('') + }) + + it('readOutput marks stderr sections', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'echo out; echo err >&2' })) + await task.done + const read = bash.readOutput(task.id) + expect(read.delta).toBe('out\n[stderr]\nerr\n') + }) + + it('readOutput reports stderr-only deltas without a leading newline', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'echo err >&2' })) + await task.done + expect(bash.readOutput(task.id).delta).toBe('[stderr]\nerr\n') + }) + + it('readOutput flags lossy reads and reports spill paths', async () => { + const { bash } = await setup({ maxOutputBytes: 100 }) + const task = bash.start(bash.resolve({ command: 'for i in $(seq 1 100); do printf "line-%04d\\n" $i; done' })) + await task.done + const read = bash.readOutput(task.id) + // Window slid past offset 0 → lossy, spill path points at the full stream. + expect(read.lossy).toBe(true) + expect(read.stdoutSpillPath).toBeDefined() + }) + + it('readOutput throws for unknown ids', async () => { + const { bash } = await setup() + expect(() => bash.readOutput('nope')).toThrow(/unknown bash task "nope"/) + }) + + it('kill terminates the process group and reports status killed', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'sleep 60' })) + expect(bash.kill(task.id)).toBe(true) + await task.done + expect(task.status).toBe('killed') + expect(task.signal).toBe('SIGTERM') + }) + + it('kill returns false for finished tasks and throws for unknown ids', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'true' })) + await task.done + expect(bash.kill(task.id)).toBe(false) + expect(() => bash.kill('nope')).toThrow(/unknown bash task "nope"/) + }) + + it('notifies onTaskDone listeners on completion', async () => { + const { bash } = await setup() + const seen: [string, string][] = [] + bash.onTaskDone(task => void seen.push([task.id, task.status])) + const task = bash.start(bash.resolve({ command: 'true' })) + await task.done + expect(seen).toEqual([[task.id, 'completed']]) + }) + + it('notifies onTaskDone for killed tasks too', async () => { + const { bash } = await setup() + const listener = vi.fn() + bash.onTaskDone(listener) + const task = bash.start(bash.resolve({ command: 'sleep 60' })) + bash.kill(task.id) + await task.done + expect(listener).toHaveBeenCalledWith(task) + expect(task.status).toBe('killed') + }) + + it('marks tasks killed when the background spawn itself fails', async () => { + const { bash } = await setup() + const listener = vi.fn() + bash.onTaskDone(listener) + const task = bash.start(bash.resolve({ command: 'true', workdir: '/nonexistent-dsh' })) + await task.done + expect(task.status).toBe('killed') + expect(listener).toHaveBeenCalledWith(task) + expect(bash.readOutput(task.id).delta).toContain('spawn failed') + }) + + it('readOutput adds a separator only when stdout lacks a trailing newline', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'printf out; echo err >&2' })) + await task.done + expect(bash.readOutput(task.id).delta).toBe('out\n[stderr]\nerr\n') + }) + + it('readOutput reports stderr spill paths', async () => { + const { bash } = await setup({ maxOutputBytes: 100 }) + const task = bash.start(bash.resolve({ command: 'for i in $(seq 1 100); do printf "line-%04d\\n" $i >&2; done' })) + await task.done + const read = bash.readOutput(task.id) + expect(read.lossy).toBe(true) + expect(read.stderrSpillPath).toBeDefined() + expect(read.delta).toContain('[stderr]') + }) + + it('disposing with already-finished tasks only kills the running ones', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalBashExecutor, {}) + const bash = ctx.bash as LocalBashExecutor + bash.internals = { spillDir, graceMs: 200 } + + const finished = bash.start(bash.resolve({ command: 'true' })) + await finished.done + const running = bash.start(bash.resolve({ command: 'sleep 60' })) + + await fiber.dispose() + await running.done + expect(finished.status).toBe('completed') + expect(running.signal).toBe('SIGTERM') + expect(bash.list()).toEqual([]) + }) + + it('disposing the executor fiber kills running tasks (no orphans)', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalBashExecutor, {}) + const bash = ctx.bash as LocalBashExecutor + bash.internals = { spillDir, graceMs: 200 } + const listener = vi.fn() + bash.onTaskDone(listener) + + const task = bash.start(bash.resolve({ command: 'sleep 60' })) + const running = bash.get(task.id)! + await new Promise(resolve => setTimeout(resolve, 50)) + + // Grab the pid before dispose clears the registry. + const pid = (running as unknown as { running: { pid: number } }).running.pid + await fiber.dispose() + await waitGone(pid) + expect(bash.list()).toEqual([]) + // Listener silenced by base-class teardown — no late notifications. + expect(listener).not.toHaveBeenCalled() + }) +}) + +describe('review fixes: lifecycle hardening', () => { + it('start honors a pre-aborted or later-aborted AbortSignal', async () => { + const { bash } = await setup() + const controller = new AbortController() + const task = bash.start(bash.resolve({ command: 'sleep 60', signal: controller.signal })) + controller.abort() + await task.done + expect(task.status).toBe('killed') + expect(task.signal).toBe('SIGTERM') + }) + + it('a throwing onTaskDone listener does not reject task.done or starve later listeners', async () => { + const { bash } = await setup() + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => undefined) + const second = vi.fn() + try { + bash.onTaskDone(() => { throw new Error('listener bug') }) + bash.onTaskDone(second) + const task = bash.start(bash.resolve({ command: 'true' })) + await expect(task.done).resolves.toBeUndefined() + expect(second).toHaveBeenCalledWith(task) + expect(errorSpy).toHaveBeenCalled() + } finally { + errorSpy.mockRestore() + } + }) + + it('dispose AWAITS a TERM-trapping process (SIGKILL escalation included)', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(LocalBashExecutor, {}) + const bash = ctx.bash as LocalBashExecutor + bash.internals = { spillDir, graceMs: 200 } + + const task = bash.start(bash.resolve({ command: 'trap \'\' TERM; sleep 60' })) + await new Promise(resolve => setTimeout(resolve, 100)) + const pid = (task as unknown as { running: { pid: number } }).running.pid + + await fiber.dispose() + // Disposal itself waited: the pid must already be gone, no grace left. + expect(() => process.kill(pid, 0)).toThrow() + expect(task.status).toBe('killed') + }) +}) diff --git a/packages/bash-local/tests/run.spec.ts b/packages/bash-local/tests/run.spec.ts new file mode 100644 index 0000000000..7f17ba1a94 --- /dev/null +++ b/packages/bash-local/tests/run.spec.ts @@ -0,0 +1,295 @@ +import { mkdtempSync, readFileSync, statSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { describe, expect, it, vi } from 'vitest' +import { killGroup, OutputCollector, runBash } from '@deepseek-ai/dsh-bash-local' + +const spillDir = mkdtempSync(join(tmpdir(), 'dsh-bash-spec-')) + +function spec(command: string, overrides: Partial[0]> = {}) { + return { + command, + cwd: process.cwd(), + timeoutMs: 0, + maxOutputBytes: 64_000, + ...overrides, + } +} + +/** Poll until a pid no longer exists (kill(pid, 0) throws ESRCH). */ +async function waitGone(pid: number, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs + while (Date.now() < deadline) { + try { + process.kill(pid, 0) + } catch { + return + } + await new Promise(resolve => setTimeout(resolve, 20)) + } + throw new Error(`pid ${pid} still alive after ${timeoutMs}ms`) +} + +describe('runBash', () => { + it('captures stdout on success', async () => { + const result = await runBash(spec('echo hello')).done + expect(result.exitCode).toBe(0) + expect(result.signal).toBeNull() + expect(result.timedOut).toBe(false) + expect(result.aborted).toBe(false) + expect(result.stdout.text).toBe('hello\n') + expect(result.stdout.truncated).toBe(false) + expect(result.stderr.text).toBe('') + }) + + it('captures stderr separately', async () => { + const result = await runBash(spec('echo oops >&2')).done + expect(result.exitCode).toBe(0) + expect(result.stdout.text).toBe('') + expect(result.stderr.text).toBe('oops\n') + }) + + it('captures both streams', async () => { + const result = await runBash(spec('echo out; echo err >&2')).done + expect(result.stdout.text).toBe('out\n') + expect(result.stderr.text).toBe('err\n') + }) + + it('reports non-zero exit codes', async () => { + const result = await runBash(spec('exit 42')).done + expect(result.exitCode).toBe(42) + expect(result.signal).toBeNull() + }) + + it('applies model-friendly env overrides', async () => { + const result = await runBash(spec('echo "$NO_COLOR/$TERM/$PAGER"')).done + expect(result.stdout.text).toBe('1/dumb/cat\n') + }) + + it('runs in the requested cwd', async () => { + const result = await runBash(spec('pwd', { cwd: '/tmp' })).done + expect(result.stdout.text.trim()).toMatch(/\/tmp$/) + }) + + it('kills with SIGTERM on timeout', async () => { + const start = Date.now() + const result = await runBash(spec('sleep 60', { timeoutMs: 100 })).done + expect(Date.now() - start).toBeLessThan(5_000) + expect(result.timedOut).toBe(true) + expect(result.signal).toBe('SIGTERM') + expect(result.exitCode).toBeNull() + }) + + it('escalates to SIGKILL when SIGTERM is trapped', async () => { + const result = await runBash( + spec('trap \'\' TERM; sleep 60', { timeoutMs: 100 }), + { graceMs: 200 }, + ).done + expect(result.timedOut).toBe(true) + expect(result.signal).toBe('SIGKILL') + }) + + it('kills the whole process group (grandchildren die too)', async () => { + // The subshell writes the sleep's pid then waits on it; killing the + // group must take the sleep down with bash. + const pidFile = join(spillDir, `grandchild-${Date.now()}.pid`) + const running = runBash(spec(`sleep 60 & echo $! > ${pidFile}; wait`)) + await new Promise(resolve => setTimeout(resolve, 300)) + const grandchild = Number(readFileSync(pidFile, 'utf8').trim()) + expect(grandchild).toBeGreaterThan(0) + + running.kill() + const result = await running.done + expect(result.signal).toBe('SIGTERM') + await waitGone(grandchild) + }) + + it('aborts via AbortSignal mid-run', async () => { + const controller = new AbortController() + const running = runBash(spec('sleep 60', { signal: controller.signal })) + setTimeout(() => { controller.abort('user cancelled') }, 50) + const result = await running.done + expect(result.aborted).toBe(true) + expect(result.signal).toBe('SIGTERM') + }) + + it('throws when the signal is already aborted before spawn', () => { + const controller = new AbortController() + controller.abort('too late') + expect(() => runBash(spec('echo hi', { signal: controller.signal }))) + .toThrow(/aborted before spawn: too late/) + }) + + it('rejects with a spawn error for a nonexistent cwd', async () => { + await expect(runBash(spec('echo hi', { cwd: '/nonexistent-dir-dsh-test' })).done) + .rejects.toThrow(/ENOENT/) + }) + + it('kill() is idempotent (second call does not restart escalation)', async () => { + const running = runBash(spec('sleep 60')) + running.kill() + running.kill() + const result = await running.done + expect(result.signal).toBe('SIGTERM') + }) +}) + +describe('output truncation and spill', () => { + it('keeps the tail and spills the full stream to disk', async () => { + // 200 numbered lines of ~10 bytes; cap at 500 bytes keeps a late tail. + const result = await runBash( + spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }), + { spillDir }, + ).done + expect(result.stdout.truncated).toBe(true) + expect(result.stdout.text.length).toBeLessThanOrEqual(500) + expect(result.stdout.text).toContain('line-0200') + expect(result.stdout.text).not.toContain('line-0001') + expect(result.stdout.spillPath).toBeDefined() + const full = readFileSync(result.stdout.spillPath!, 'utf8') + expect(full).toContain('line-0001') + expect(full).toContain('line-0200') + }) + + it('does not truncate output exactly at the cap', async () => { + const result = await runBash( + spec('printf "%.0sx" $(seq 1 500)', { maxOutputBytes: 500 }), + { spillDir }, + ).done + expect(result.stdout.truncated).toBe(false) + expect(result.stdout.text.length).toBe(500) + expect(result.stdout.spillPath).toBeUndefined() + }) +}) + +describe('OutputCollector', () => { + it('keeps the tail of a single oversized chunk', () => { + const collector = new OutputCollector(10, 'test', spillDir) + collector.push(Buffer.from('0123456789abcdef')) + const out = collector.finalize() + expect(out.text).toBe('6789abcdef') + expect(out.truncated).toBe(true) + expect(readFileSync(out.spillPath!, 'utf8')).toBe('0123456789abcdef') + }) + + it('readFrom returns increments and flags lossy reads', () => { + const collector = new OutputCollector(10, 'test', spillDir) + collector.push(Buffer.from('aaaaa')) + const first = collector.readFrom(0) + expect(first.text).toBe('aaaaa') + expect(first.lossy).toBe(false) + expect(first.nextOffset).toBe(5) + + collector.push(Buffer.from('bbbbb')) + const second = collector.readFrom(first.nextOffset) + expect(second.text).toBe('bbbbb') + expect(second.lossy).toBe(false) + + // Push enough to slide the window past the last offset. + collector.push(Buffer.from('c'.repeat(20))) + const third = collector.readFrom(second.nextOffset) + expect(third.lossy).toBe(true) + expect(third.text).toBe('c'.repeat(10)) + expect(third.spillPath).toBeDefined() + }) + + it('tracks totalBytes across drops', () => { + const collector = new OutputCollector(4, 'test', spillDir) + collector.push(Buffer.from('aaaa')) + collector.push(Buffer.from('bbbb')) + expect(collector.totalBytes).toBe(8) + expect(collector.finalize().text).toBe('bbbb') + }) +}) + +describe('killGroup', () => { + it('ignores non-positive pids', () => { + expect(() => { killGroup(-1, 'SIGTERM') }).not.toThrow() + expect(() => { killGroup(0, 'SIGTERM') }).not.toThrow() + }) + + it('swallows ESRCH for vanished groups', async () => { + const running = runBash(spec('true')) + await running.done + expect(() => { killGroup(running.pid, 'SIGTERM') }).not.toThrow() + }) +}) + +describe('abort edge cases', () => { + it('reports a fallback reason for reason-less pre-aborted signals', () => { + // Real AbortControllers always set a DOMException reason; signal-like + // objects from other libraries may not — the fallback covers them. + const bare = { + aborted: true, + reason: undefined, + addEventListener() {}, + removeEventListener() {}, + } as unknown as AbortSignal + expect(() => runBash(spec('echo hi', { signal: bare }))) + .toThrow(/aborted before spawn: aborted/) + }) + + it('reports an externally self-killed command without the timeout marker', async () => { + const result = await runBash(spec('kill -TERM $$')).done + expect(result.signal).toBe('SIGTERM') + expect(result.timedOut).toBe(false) + expect(result.aborted).toBe(false) + }) +}) + +describe('review fixes: env scrubbing and spill hardening', () => { + it('scrubs credential-shaped env vars from child processes', async () => { + process.env.DSH_TEST_API_KEY = 'super-secret' + process.env.DSH_TEST_TOKEN = 'also-secret' + process.env.DSH_TEST_PLAIN = 'visible' + try { + const result = await runBash(spec('echo "[${DSH_TEST_API_KEY:-absent}|${DSH_TEST_TOKEN:-absent}|${DSH_TEST_PLAIN:-absent}]"')).done + expect(result.stdout.text.trim()).toBe('[absent|absent|visible]') + } finally { + delete process.env.DSH_TEST_API_KEY + delete process.env.DSH_TEST_TOKEN + delete process.env.DSH_TEST_PLAIN + } + }) + + it('creates spill files with owner-only permissions and random names', async () => { + const result = await runBash( + spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }), + { spillDir }, + ).done + const path = result.stdout.spillPath! + expect(path).toMatch(/dsh-bash-\d+-\d+-[0-9a-f]{12}-stdout\.log$/) + const mode = statSync(path).mode & 0o777 + expect(mode).toBe(0o600) + }) + + it('defaults spills into a private per-process directory', async () => { + const result = await runBash( + spec('for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', { maxOutputBytes: 500 }), + ).done + const dir = dirname(result.stdout.spillPath!) + expect(dir).toMatch(/dsh-bash-/) + const mode = statSync(dir).mode & 0o777 + expect(mode).toBe(0o700) + }) + + it('killGroup never throws, even for EPERM-style failures', () => { + const spy = vi.spyOn(process, 'kill').mockImplementation(() => { + throw Object.assign(new Error('EPERM'), { code: 'EPERM' }) + }) + try { + expect(() => { killGroup(12345, 'SIGTERM') }).not.toThrow() + } finally { + spy.mockRestore() + } + }) + + it('honors AbortSignal on background-style runs (no timeout)', async () => { + const controller = new AbortController() + const running = runBash(spec('sleep 60', { timeoutMs: 0, signal: controller.signal })) + setTimeout(() => { controller.abort() }, 50) + const result = await running.done + expect(result.aborted).toBe(true) + expect(result.signal).toBe('SIGTERM') + }) +}) diff --git a/packages/bash-local/tsconfig.json b/packages/bash-local/tsconfig.json new file mode 100644 index 0000000000..a657d8bf8e --- /dev/null +++ b/packages/bash-local/tsconfig.json @@ -0,0 +1,14 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib" + }, + "include": ["src"], + "references": [ + { "path": "../../vendor/cosmokit" }, + { "path": "../../vendor/cordis" }, + { "path": "../../vendor/schemastery" }, + { "path": "../bash" } + ] +} diff --git a/packages/bash/README.md b/packages/bash/README.md new file mode 100644 index 0000000000..2480240c77 --- /dev/null +++ b/packages/bash/README.md @@ -0,0 +1,42 @@ +# @deepseek-ai/dsh-bash + +The **bash executor seam**: an abstract `BashExecutor` service (`ctx.bash`) +defining WHAT a bash backend does — run commands, manage background tasks — +without saying HOW. + +This package is one third of the bash capability, split so each concern can +evolve (and be swapped) independently: + +| Package | Role | +|---|---| +| `@deepseek-ai/dsh-bash` (this) | the interface: abstract service + vocabulary types | +| `@deepseek-ai/dsh-bash-local` | an implementation: local subprocesses | +| `@deepseek-ai/dsh-tool-bash` | the model-facing tool schemas over `ctx.bash` | + +The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the agent-tool +survey: pi hides execution behind a `BashOperations` interface (local shell / +SSH / VM backends), Codex behind an exec-server protocol. A future sandboxed, +containerized, or remote executor implements this interface and the tool +schemas don't change. + +## Service API (`ctx.bash`) + +| Member | Semantics | +|---|---| +| `run(spec)` | Foreground execution. Resolves when the command finishes. **Rejects only for infrastructure failures** (unusable workdir, missing shell, pre-aborted signal); nonzero exits, timeout kills, and abort kills resolve with a descriptive `BashRunResult`. | +| `start(spec)` | Background execution. Returns a `BashTask` handle immediately; **no timeout applies** (stop tasks via `kill`). | +| `get(id)` / `list()` | Task lookup. | +| `readOutput(id)` | **Incremental** output read — consecutive reads never re-deliver. Reads that lost data to buffer bounds flag `lossy` and point at full-stream spill files. Throws for unknown ids. | +| `kill(id)` | Kill a running task. Returns `false` when it already finished; throws for unknown ids. | +| `onTaskDone(listener)` | Completion listener (effect-based, disposer returned). Fires exactly once per task; never after the service is disposed. | + +Implementations subclass `BashExecutor`, implement the abstract methods, and +call `notifyTaskDone(task)` on background completion. Disposal must kill every +running task (no orphan processes) — see the HMR-safety tests. + +## Vocabulary + +`BashExecSpec` (command, workdir?, timeoutMs?, signal?) → +`BashRunResult` (exitCode, signal, timedOut, aborted, timeoutMs, stdout/stderr +as `CollectedOutput`) and `BashTask`/`BashTaskRead` for the background side. +See `src/types.ts` for the full contracts. diff --git a/packages/bash/package.json b/packages/bash/package.json new file mode 100644 index 0000000000..52bf80282f --- /dev/null +++ b/packages/bash/package.json @@ -0,0 +1,28 @@ +{ + "name": "@deepseek-ai/dsh-bash", + "description": "Abstract bash executor seam (ctx.bash) for the DeepSeek Harness", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "exports": { + ".": { + "types": "./lib/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/bash/src/index.ts b/packages/bash/src/index.ts new file mode 100644 index 0000000000..e22aad5ff3 --- /dev/null +++ b/packages/bash/src/index.ts @@ -0,0 +1,131 @@ +/** + * The bash executor seam (`ctx.bash`): an abstract service defining WHAT a + * bash backend does — run commands, manage background tasks — without saying + * HOW. Implementations subclass {@link BashExecutor} and register themselves + * as the `bash` service; `@deepseek-ai/dsh-bash-local` (local subprocesses) + * is the first. Future implementations swap in sandboxes, containers, or + * remote exec servers without touching the tool schemas that consume them + * (`@deepseek-ai/dsh-tool-bash`). + * + * The split mirrors the LLM seam (`LlmService`/`LlmAdapter`) and the + * surveyed agents: pi hides execution behind a `BashOperations` interface + * (local shell / SSH / VM backends), Codex behind an exec-server protocol. + * + * @module @deepseek-ai/dsh-bash + */ + +import { Context, Service } from 'cordis' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskListener, BashTaskRead } from './types.ts' + +export type { + BashExecRequest, + BashExecSpec, + BashRunResult, + BashTask, + BashTaskListener, + BashTaskRead, + BashTaskStatus, + CollectedOutput, +} from './types.ts' + +declare module 'cordis' { + interface Context { + bash: BashExecutor + } +} + +/** + * Abstract bash execution service. Subclass, implement the abstract methods, + * and load the subclass as a plugin — it registers as `ctx.bash` (one + * implementation per context; loading a second throws, which is cordis' + * standard duplicate-service behavior). + * + * Semantics every implementation must honor: + * - {@link run} REJECTS only for infrastructure failures (unusable workdir, + * missing shell, pre-aborted signal). Nonzero exits, timeout kills, and + * abort kills RESOLVE with a descriptive {@link BashRunResult} — reporting + * a failed command is the tool layer's job, not an exception. + * - {@link start} returns immediately; no timeout applies to background + * tasks (callers stop them via {@link kill} or the spec's AbortSignal). + * Completion must fire the {@link onTaskDone} listeners exactly once per + * task, and must NOT fire after the service is disposed. + * - {@link readOutput} is incremental: consecutive reads never re-deliver + * output. Implementations bound their buffers; reads that lost data flag + * `lossy` and point at full-stream spill files when available. + * - Disposal kills every running task and awaits their exit (no orphan + * processes survive `fiber.dispose()`). + */ +export abstract class BashExecutor extends Service { + private listeners = new Set() + private listenersClosed = false + + constructor(ctx: Context) { + super(ctx, 'bash') + ctx.effect(() => () => { + // Close the listener registry before subclass teardown so late task + // completions (e.g. from kills issued during dispose) stay silent. + this.listenersClosed = true + this.listeners.clear() + }, 'bash listener teardown') + } + + /** + * Resolve a caller's {@link BashExecRequest} into a fully-specified + * {@link BashExecSpec}, applying this implementation's config defaults and + * caps (working directory, default/max timeout). Consumers (tool layer) + * call this, then pass the result to {@link run}/{@link start} — keeping + * defaulting in the implementation that owns the config while the seam type + * stays explicit (no hidden `?? default` inside run/start). + */ + abstract resolve(request: BashExecRequest): BashExecSpec + + /** Run a command in the foreground; resolves when it finishes. */ + abstract run(spec: BashExecSpec): Promise + + /** Start a background task and return its handle immediately. */ + abstract start(spec: BashExecSpec): BashTask + + /** Look up a background task by id. */ + abstract get(id: string): BashTask | undefined + + /** All tracked background tasks (insertion order). */ + abstract list(): BashTask[] + + /** Read output produced since the previous read. Throws for unknown ids. */ + abstract readOutput(id: string): BashTaskRead + + /** + * Kill a running background task. Returns false when it had already + * finished (no-op). Throws for unknown ids. + */ + abstract kill(id: string): boolean + + /** + * Register a background-task completion listener (disposed with the + * calling fiber). Listeners never fire after this service is disposed. + */ + onTaskDone(listener: BashTaskListener): () => void { + const dispose = this.ctx.effect(() => { + this.listeners.add(listener) + return () => this.listeners.delete(listener) + }, 'bash.onTaskDone()') + return () => void dispose() + } + + /** For implementations: notify listeners that `task` completed. Listener + * exceptions are contained (logged) — one bad listener must not reject + * `BashTask.done` or starve the listeners after it. */ + protected notifyTaskDone(task: BashTask): void { + if (this.listenersClosed) return + for (const listener of this.listeners) { + try { + listener(task) + } catch (error: unknown) { + // Listener bugs are reported, never propagated into task.done. + console.error('bash onTaskDone listener threw:', error) + } + } + } +} + +export default BashExecutor diff --git a/packages/bash/src/types.ts b/packages/bash/src/types.ts new file mode 100644 index 0000000000..1860401ee2 --- /dev/null +++ b/packages/bash/src/types.ts @@ -0,0 +1,97 @@ +/** + * Execution vocabulary for the bash executor seam. Types only — the abstract + * service lives in `./index.ts`, implementations in sibling packages + * (`@deepseek-ai/dsh-bash-local` first). + * + * @module dsh-bash/types + */ + +/** + * A caller's execution REQUEST: `workdir` and `timeoutMs` are optional and + * filled by {@link BashExecutor.resolve} from the implementation's config. + * This is the model-/plugin-facing shape; pass it to `resolve()` to obtain a + * fully-resolved {@link BashExecSpec}. + */ +export interface BashExecRequest { + command: string + /** Working directory override (default: implementation-configured). */ + workdir?: string | undefined + /** Timeout override in milliseconds (implementations cap it). */ + timeoutMs?: number | undefined + /** Abort signal — implementations kill the command when it fires. */ + signal?: AbortSignal | undefined +} + +/** + * A fully-resolved execution SPEC — exactly what {@link BashExecutor.run} / + * {@link BashExecutor.start} act on. `workdir` and `timeoutMs` are REQUIRED: + * defaulting and capping already happened in {@link BashExecutor.resolve}, so + * the executor never hides a `?? config` fallback (explicit > implicit). For + * background tasks, `start()` ignores `timeoutMs` (background runs have no + * timeout) — the field is still required because the type is shared. + */ +export interface BashExecSpec { + command: string + workdir: string + timeoutMs: number + /** Abort signal — implementations kill the command when it fires. */ + signal?: AbortSignal | undefined +} + +/** One captured stream: the (possibly truncated) text plus recovery info. */ +export interface CollectedOutput { + /** Collected text — the TAIL of the stream when truncated. */ + text: string + /** True when bytes were dropped from `text`. */ + truncated: boolean + /** Path to a file holding the COMPLETE stream, when truncated. */ + spillPath?: string +} + +/** The outcome of one completed (or killed) foreground run. */ +export interface BashRunResult { + /** Exit code; null when the process died from a signal. */ + exitCode: number | null + /** Terminating signal (e.g. 'SIGTERM'); null on normal exit. */ + signal: NodeJS.Signals | null + /** True when the executor's own timeout killed the command. */ + timedOut: boolean + /** True when the caller's AbortSignal killed the command. */ + aborted: boolean + /** The effective timeout applied to this run (after defaulting/capping). */ + timeoutMs: number + stdout: CollectedOutput + stderr: CollectedOutput +} + +/** Lifecycle of a background task. */ +export type BashTaskStatus = 'running' | 'completed' | 'killed' + +/** A tracked background task handle. */ +export interface BashTask { + readonly id: string + readonly command: string + status: BashTaskStatus + /** Exit code once finished (null = killed by signal / still running). */ + exitCode: number | null + /** Terminating signal name, when signal-killed. */ + signal: NodeJS.Signals | null + /** Resolves when the underlying process closes (never rejects). */ + readonly done: Promise +} + +/** One incremental {@link BashExecutor.readOutput} read. */ +export interface BashTaskRead { + task: BashTask + /** Output produced since the previous read (stderr in a marked section). */ + delta: string + /** True when truncation dropped unread bytes the delta cannot include. */ + lossy: boolean + /** Full stdout spill file, when stdout truncation occurred. */ + stdoutSpillPath?: string + /** Full stderr spill file, when stderr truncation occurred. */ + stderrSpillPath?: string +} + +/** Completion callback for background tasks. */ +export type BashTaskListener = (task: BashTask) => void diff --git a/packages/bash/tests/service.spec.ts b/packages/bash/tests/service.spec.ts new file mode 100644 index 0000000000..751bd80015 --- /dev/null +++ b/packages/bash/tests/service.spec.ts @@ -0,0 +1,138 @@ +import { describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import { BashExecutor } from '@deepseek-ai/dsh-bash' +import type { BashExecRequest, BashExecSpec, BashRunResult, BashTask, BashTaskRead } from '@deepseek-ai/dsh-bash' + +/** Minimal concrete executor: records calls, lets tests drive completions. */ +class StubExecutor extends BashExecutor { + tasks = new Map() + + resolve(request: BashExecRequest): BashExecSpec { + return { + command: request.command, + workdir: request.workdir ?? '/stub', + timeoutMs: request.timeoutMs ?? 1000, + ...request.signal ? { signal: request.signal } : {}, + } + } + + async run(_spec: BashExecSpec): Promise { + return { + exitCode: 0, + signal: null, + timedOut: false, + aborted: false, + timeoutMs: 1000, + stdout: { text: 'ok', truncated: false }, + stderr: { text: '', truncated: false }, + } + } + + start(spec: BashExecSpec): BashTask { + const task: BashTask = { + id: `stub-${this.tasks.size + 1}`, + command: spec.command, + status: 'running', + exitCode: null, + signal: null, + done: Promise.resolve(), + } + this.tasks.set(task.id, task) + return task + } + + get(id: string): BashTask | undefined { + return this.tasks.get(id) + } + + list(): BashTask[] { + return [...this.tasks.values()] + } + + readOutput(id: string): BashTaskRead { + const task = this.tasks.get(id) + if (!task) throw new Error(`unknown bash task "${id}"`) + return { task, delta: '', lossy: false } + } + + kill(id: string): boolean { + const task = this.tasks.get(id) + if (!task) throw new Error(`unknown bash task "${id}"`) + if (task.status !== 'running') return false + task.status = 'killed' + return true + } + + /** Expose the protected notifier for tests. */ + fire(task: BashTask): void { + this.notifyTaskDone(task) + } +} + +async function setup() { + const ctx = new Context() + await ctx.plugin(StubExecutor) + // ctx.bash resolves to the registered implementation. + const bash = ctx.bash as StubExecutor + return { ctx, bash } +} + +describe('BashExecutor service seam', () => { + it('registers as ctx.bash and serves the abstract API', async () => { + const { bash } = await setup() + const task = bash.start(bash.resolve({ command: 'sleep 1' })) + expect(bash.get(task.id)).toBe(task) + expect(bash.list()).toEqual([task]) + expect(bash.kill(task.id)).toBe(true) + expect(bash.kill(task.id)).toBe(false) + const result = await bash.run(bash.resolve({ command: 'true' })) + expect(result.exitCode).toBe(0) + }) + + it('onTaskDone delivers completions to registered listeners', async () => { + const { bash } = await setup() + const seen: string[] = [] + bash.onTaskDone(task => void seen.push(task.id)) + const task = bash.start(bash.resolve({ command: 'x' })) + bash.fire(task) + expect(seen).toEqual([task.id]) + }) + + it('onTaskDone disposer unsubscribes the listener', async () => { + const { bash } = await setup() + const listener = vi.fn() + const dispose = bash.onTaskDone(listener) + dispose() + bash.fire(bash.start(bash.resolve({ command: 'x' }))) + expect(listener).not.toHaveBeenCalled() + }) + + it('listeners registered from a fiber are removed on dispose (HMR safety)', async () => { + const { ctx, bash } = await setup() + const listener = vi.fn() + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + inner.bash.onTaskDone(listener) + }, { inject: ['bash'] })) + bash.fire(bash.start(bash.resolve({ command: 'one' }))) + expect(listener).toHaveBeenCalledTimes(1) + + await fiber.dispose() + bash.fire(bash.start(bash.resolve({ command: 'two' }))) + expect(listener).toHaveBeenCalledTimes(1) + }) + + it('silences listeners once the service fiber is disposed', async () => { + const ctx = new Context() + const fiber = await ctx.plugin(Object.assign(async (inner: Context) => { + await inner.plugin(StubExecutor) + }, {})) + const bash = ctx.bash as StubExecutor + const listener = vi.fn() + bash.onTaskDone(listener) + const task = bash.start(bash.resolve({ command: 'x' })) + + await fiber.dispose() + bash.fire(task) + expect(listener).not.toHaveBeenCalled() + }) +}) diff --git a/packages/bash/tsconfig.json b/packages/bash/tsconfig.json new file mode 100644 index 0000000000..2617271c44 --- /dev/null +++ b/packages/bash/tsconfig.json @@ -0,0 +1,12 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib" + }, + "include": ["src"], + "references": [ + { "path": "../../vendor/cosmokit" }, + { "path": "../../vendor/cordis" } + ] +} diff --git a/packages/tool-bash/README.md b/packages/tool-bash/README.md new file mode 100644 index 0000000000..35f5de0c46 --- /dev/null +++ b/packages/tool-bash/README.md @@ -0,0 +1,62 @@ +# @deepseek-ai/dsh-tool-bash + +The model-facing bash tools — `bash`, `bash_output`, `bash_kill` — registered +over the `ctx.bash` executor seam (`@deepseek-ai/dsh-bash`). Pure schema + +text shaping; every process concern lives behind the seam, so sandboxed or +remote executor implementations swap in without changing what the model sees. + +Requires a loaded executor implementation (e.g. +`@deepseek-ai/dsh-bash-local`); the plugin stays pending until `ctx.bash` +exists (`inject: ['tools', 'bash']`). + +## Tools + +### `bash` + +| Arg | Type | Notes | +|---|---|---| +| `command` | string (required) | Run via `bash -c`. No state persists between calls — use `workdir`, not `cd`. | +| `description` | string (required) | One-line, active-voice summary of the command (5-10 words), for UI/log display only — no effect on execution. | +| `timeoutMs` | number | Default/max from executor config (120s/600s for bash-local). | +| `workdir` | string | Working directory for this call. | +| `run_in_background` | boolean | Return a task id immediately; no timeout applies. | + +`command`, `workdir`, and `timeoutMs` are resolved against the executor's +config defaults via `ctx.bash.resolve()` before execution, so the executor +seam (`BashExecSpec`) receives explicit `workdir`/`timeoutMs` values. + +Result text: stdout, then a `[stderr]` section, then status markers — +`[timed out after Nms]` whenever the executor's timer fired (reported +independently of how the process ended, so a command that traps SIGTERM and +exits 0 still shows it), `[killed by signal: …]` for a signal death, +`[exit code: N]` for a non-zero exit (reported, **not** `isError`: the model +decides how to react), and `[output truncated; full output: ]` when the +tail was kept. Only infrastructure failures (spawn errors, aborts) surface as +`isError` results. + +### `bash_output` + +`task_id` → output produced **since the previous `bash_output` call** plus a +status line (`running` / `completed, exit code: N` / `killed`). Reads that +lost data to buffer bounds say so and point at the full-output spill file. + +### `bash_kill` + +`task_id` → SIGTERM→SIGKILL on the task's process group. Killing an +already-finished task is a reported no-op; unknown ids are errors. + +## Background completion notices + +When a background task finishes, a short notice is injected into the owning +agent's session (`agent.inject()`, source `{kind: 'plugin', plugin: +'tool-bash'}`). Injection is **durable context for the next model request, +not a wake-up** — an idle agent stays idle until something sends a message. +That's why the tool descriptions tell the model to poll with `bash_output`. + +## Permissions + +`TODO(permissions)`: commands run with the executor's full authority. The +permission/sandbox seam is the `tools/execute` waterfall (veto or ask) plus +sandboxing `BashExecutor` implementations — see docs/architecture.md. +`@cordisjs/plugin-capability` (a named-permission service with a session +`test()`) is a candidate building block for that work. diff --git a/packages/tool-bash/package.json b/packages/tool-bash/package.json new file mode 100644 index 0000000000..3bf4f64de0 --- /dev/null +++ b/packages/tool-bash/package.json @@ -0,0 +1,40 @@ +{ + "name": "@deepseek-ai/dsh-tool-bash", + "description": "Model-facing bash tools (bash, bash_output, bash_kill) over the DeepSeek Harness bash executor seam", + "version": "0.0.1", + "private": true, + "type": "module", + "main": "lib/index.js", + "types": "lib/index.d.ts", + "exports": { + ".": { + "types": "./lib/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib", + "src" + ], + "license": "BSD-3-Clause", + "peerDependencies": { + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-bash": "^0.0.1", + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", + "cordis": "^4.0.0-rc.6" + }, + "devDependencies": { + "@deepseek-ai/dsh-agent": "^0.0.1", + "@deepseek-ai/dsh-agent-loop": "^0.0.1", + "@deepseek-ai/dsh-bash": "^0.0.1", + "@deepseek-ai/dsh-bash-local": "^0.0.1", + "@deepseek-ai/dsh-llm": "^0.0.1", + "@deepseek-ai/dsh-session": "^0.0.1", + "@deepseek-ai/dsh-system-prompt": "^0.0.1", + "@deepseek-ai/dsh-tools": "^0.0.1", + "cordis": "^4.0.0-rc.6" + } +} diff --git a/packages/tool-bash/src/index.ts b/packages/tool-bash/src/index.ts new file mode 100644 index 0000000000..30edb8639b --- /dev/null +++ b/packages/tool-bash/src/index.ts @@ -0,0 +1,226 @@ +/** + * The model-facing bash tools: `bash`, `bash_output`, `bash_kill`. Pure + * schema + text shaping — every process concern lives behind the `ctx.bash` + * executor seam (`@deepseek-ai/dsh-bash`), so sandbox/permission/remote + * executor implementations swap in without touching what the model sees. + * + * Background notifications: when a background task completes, a short notice + * is injected into the owning agent's session (`agent.inject()` — the + * documented context seam). Injection is durable context for the NEXT model + * request, not a wake-up: an idle agent stays idle until something sends a + * message, which is why the tool descriptions tell the model to poll with + * `bash_output`. + * + * TODO(permissions): commands run with the executor's full authority. The + * permission/sandbox seam is the `tools/execute` waterfall (veto/ask) plus + * sandboxing `BashExecutor` implementations — see docs/architecture.md + * § plugin checklist. + * + * @module @deepseek-ai/dsh-tool-bash + */ + +import type { Context } from 'cordis' +import { defineTool } from '@deepseek-ai/dsh-tools' +import type { Agent } from '@deepseek-ai/dsh-agent' +import type { BashRunResult, BashTask, CollectedOutput } from '@deepseek-ai/dsh-bash' + +export const name = 'tool-bash' +export const inject = ['tools', 'bash'] + +/** + * Validate model-produced arguments. `defineTool`'s `InferArgs` typing is + * compile-time only — at runtime `arguments` is whatever JSON the model + * emitted, so every field is checked before it reaches the executor. + * + * TODO(RFC 005): this hand-rolled validation is the per-tool stopgap until + * `defineTool` validates parsed args against the SchemaSpec itself (the + * converter already encodes the structure). When that lands, delete this and + * let the registry reject malformed calls — see docs/rfc/005. + */ +function validateBashArgs(args: { + command: string + description: string + timeoutMs?: number + workdir?: string + run_in_background?: boolean +}): void { + if (typeof args.command !== 'string' || args.command.trim().length === 0) { + throw new Error('invalid command: expected a non-empty string') + } + if (typeof args.description !== 'string' || args.description.trim().length === 0) { + throw new Error('invalid description: expected a non-empty string') + } + if (args.timeoutMs !== undefined + && (typeof args.timeoutMs !== 'number' || !Number.isFinite(args.timeoutMs) || args.timeoutMs <= 0)) { + throw new Error(`invalid timeoutMs: expected a positive number, got ${JSON.stringify(args.timeoutMs)}`) + } + if (args.workdir !== undefined && typeof args.workdir !== 'string') { + throw new Error(`invalid workdir: expected a string, got ${JSON.stringify(args.workdir)}`) + } + if (args.run_in_background !== undefined && typeof args.run_in_background !== 'boolean') { + throw new Error(`invalid run_in_background: expected a boolean, got ${JSON.stringify(args.run_in_background)}`) + } +} + +/** Require a string `task_id` (model-produced, so runtime-checked). */ +function validateTaskId(value: unknown): string { + if (typeof value !== 'string' || value.length === 0) { + throw new Error(`invalid task_id: expected a string, got ${JSON.stringify(value)}`) + } + return value +} + +/** Append the truncation notice (with the full-output spill path) to a stream's text. */ +function streamText(output: CollectedOutput): string { + if (!output.truncated) return output.text + return `${output.text}\n[output truncated; full output: ${output.spillPath ?? '(unavailable)'}]` +} + +/** + * Shape one finished run into the text the model sees: stdout, then a marked + * stderr section, then exit-status markers. Non-zero exits are REPORTED, not + * errored — the model decides how to react; only infrastructure failures + * (spawn errors, aborts) surface as isError results. + */ +export function renderResult(result: BashRunResult): string { + const out = streamText(result.stdout) + const err = streamText(result.stderr) + + let body = out + if (err.length > 0) { + // Single newline between sections (stdout usually ends with one already). + if (body.length > 0 && !body.endsWith('\n')) body += '\n' + body += `[stderr]\n${err}` + } + if (body.length === 0) body = '(no output)' + + const markers: string[] = [] + // Timeout is reported independently of how the process actually ended: a + // command can trap SIGTERM and exit 0 after our timer fired (e.g. + // `trap "exit 0" TERM; sleep 60`), giving timedOut:true / exitCode:0 / + // signal:null — the model must still see that the command was cut short. + if (result.timedOut) markers.push(`[timed out after ${result.timeoutMs}ms]`) + if (result.signal !== null) { + markers.push(`[killed by signal: ${result.signal}]`) + } else if (result.exitCode !== 0) { + markers.push(`[exit code: ${result.exitCode}]`) + } + if (markers.length === 0) return body + + if (!body.endsWith('\n')) body += '\n' + return body + markers.join('\n') +} + +/** Status line for background task reads. */ +function statusLine(task: BashTask): string { + switch (task.status) { + case 'running': return '[status: running]' + case 'killed': return `[status: killed${task.signal !== null ? ` by ${task.signal}` : ''}]` + case 'completed': return `[status: completed, exit code: ${task.exitCode ?? 0}]` + } +} + +export function apply(ctx: Context): void { + // Background completion → inject a notice into the owning agent's session. + // Tracks the agent per task id; entries drop once notified. + const owners = new Map() + ctx.bash.onTaskDone((task) => { + const agent = owners.get(task.id) + owners.delete(task.id) + if (!agent) return + try { + agent.inject( + [{ type: 'text', text: `background bash task ${task.id} finished ${statusLine(task)}. Read its output with bash_output.` }], + { source: { kind: 'plugin', plugin: 'tool-bash' } }, + ) + } catch (error: unknown) { + // The ONE expected failure: the agent was disposed between task + // completion and this injection (LoopAgent.inject throws + // `agent "" is disposed`). That race is benign — drop the notice. + // Anything else is a real bug and must surface, not be swallowed. + if (error instanceof Error && error.message.includes('is disposed')) return + throw error + } + }) + + ctx.tools.register(defineTool({ + name: 'bash', + description: 'Execute a bash command (`bash -c`) and return its stdout/stderr. ' + + 'Each call runs in a fresh shell: no state (cwd, variables, functions) persists between calls — ' + + 'pass `workdir` instead of using `cd`. Non-zero exits are reported as `[exit code: N]`. ' + + 'Long output is truncated to its tail; the full output is saved to a file whose path is reported. ' + + 'Set `run_in_background: true` for long-running commands: the call returns a task id immediately; ' + + 'poll it with `bash_output` and stop it with `bash_kill`.', + parameters: { + command: { type: 'string', required: true, description: 'The bash command to execute.' }, + description: { + type: 'string', + required: true, + description: 'Clear, concise description of what this command does in active voice, ' + + '5-10 words (shown in the UI). Examples: "ls" → "List files in current directory"; ' + + '"git status" → "Show working tree status"; "npm install" → "Install package dependencies".', + }, + timeoutMs: { type: 'number', description: 'Timeout in milliseconds (default 120000, max 600000). The command is killed on expiry.' }, + workdir: { type: 'string', description: 'Working directory for this command.' }, + run_in_background: { type: 'boolean', description: 'Run in the background and return a task id immediately. No timeout applies.' }, + }, + async execute(args, exec) { + validateBashArgs(args) + // `description` is display/logging metadata only (surfaced to UIs via + // the tool/call session event); it is intentionally NOT forwarded to + // ctx.bash and has no effect on execution. + const request = { + command: args.command, + ...args.workdir !== undefined ? { workdir: args.workdir } : {}, + ...args.timeoutMs !== undefined ? { timeoutMs: args.timeoutMs } : {}, + ...exec.signal ? { signal: exec.signal } : {}, + } + if (args.run_in_background === true) { + const task = ctx.bash.start(ctx.bash.resolve(request)) + if (exec.agent) owners.set(task.id, exec.agent) + return [{ type: 'text', text: `started background task ${task.id}` }] + } + const result = await ctx.bash.run(ctx.bash.resolve(request)) + if (result.aborted) throw new Error('command aborted') + return [{ type: 'text', text: renderResult(result) }] + }, + })) + + ctx.tools.register(defineTool({ + name: 'bash_output', + description: 'Read new output from a background bash task started with `bash` + `run_in_background`. ' + + 'Returns only output produced since the previous bash_output call, plus the task status. ' + + 'Tasks keep running while you do other work; poll again later for more output.', + parameters: { + task_id: { type: 'string', required: true, description: 'Task id returned by the bash tool.' }, + }, + // execute is synchronous (registry reads + string shaping) but the + // ToolDefinition contract wants a Promise — hence resolve(), not async. + execute(args) { + const read = ctx.bash.readOutput(validateTaskId(args.task_id)) + let text = read.delta.length > 0 ? read.delta : '(no new output)' + if (read.lossy) { + const paths = [read.stdoutSpillPath, read.stderrSpillPath].filter((p): p is string => p !== undefined) + text += `\n[some output was dropped from memory; full output: ${paths.join(', ')}]` + } + text += `\n${statusLine(read.task)}` + return Promise.resolve([{ type: 'text', text }]) + }, + })) + + ctx.tools.register(defineTool({ + name: 'bash_kill', + description: 'Kill a running background bash task (SIGTERM, then SIGKILL) by task id.', + parameters: { + task_id: { type: 'string', required: true, description: 'Task id returned by the bash tool.' }, + }, + execute(args) { + const id = validateTaskId(args.task_id) + const killed = ctx.bash.kill(id) + return Promise.resolve([{ + type: 'text', + text: killed ? `killed background task ${id}` : `task ${id} had already finished`, + }]) + }, + })) +} diff --git a/packages/tool-bash/tests/integration.spec.ts b/packages/tool-bash/tests/integration.spec.ts new file mode 100644 index 0000000000..2683f7ec55 --- /dev/null +++ b/packages/tool-bash/tests/integration.spec.ts @@ -0,0 +1,164 @@ +import { describe, expect, it } from 'vitest' +import { Context } from 'cordis' +import LlmService from '@deepseek-ai/dsh-llm' +import SessionStore from '@deepseek-ai/dsh-session' +import type { SessionEvent } from '@deepseek-ai/dsh-session' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import AgentRegistry from '@deepseek-ai/dsh-agent' +import AgentLoop, { LoopAgent } from '@deepseek-ai/dsh-agent-loop' +import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import * as ToolBash from '@deepseek-ai/dsh-tool-bash' +import { MockAdapter, textResponse, toolCallResponse } from '../../agent-loop/tests/mock-adapter.ts' + +/** + * Full-loop integration: a scripted mock model drives the REAL bash tool + * through the agent loop, exercising the same seams a live model would + * (tool/call + tool/result session events, agent.inject notifications). + */ +async function harness(adapter: MockAdapter) { + const ctx = new Context() + await ctx.plugin(LlmService) + await ctx.plugin(SessionStore) + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(AgentRegistry) + await ctx.plugin(AgentLoop, { agents: [] }) + await ctx.plugin(LocalBashExecutor, { timeoutMs: 10_000 }) + await ctx.plugin(ToolBash) + ctx.llm.registerAdapter(['mock'], adapter) + return ctx +} + +function waitForIdle(ctx: Context, agent: LoopAgent): Promise { + return new Promise((resolve) => { + const dispose = ctx.on('agent/status', (subject, status) => { + if (subject === agent && status === 'idle') { + dispose() + resolve() + } + }) + }) +} + +function events(agent: LoopAgent): SessionEvent[] { + return [...agent.session.events] +} + +/** Find a session event by type, narrowed; throws when absent. */ +function findEvent( + log: SessionEvent[], + type: T, + position: 'first' | 'last' = 'first', +): Extract { + const found = position === 'first' + ? log.find(event => event.type === type) + : log.findLast(event => event.type === type) + if (!found) throw new Error(`no ${type} event in the session log`) + return found as Extract +} + +function resultText(event: SessionEvent): string { + if (event.type !== 'tool/result') return '' + return event.data.content + .filter(block => block.type === 'text') + .map(block => block.text) + .join('') +} + +describe('bash tool through the agent loop', () => { + it('foreground: model calls bash, sees the result, replies', async () => { + const adapter = new MockAdapter([ + toolCallResponse('call-1', 'bash', { command: 'echo integration-ok', description: 'test command' }, 'Running it.'), + textResponse('The command printed integration-ok.'), + ]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create('it-fg', { model: 'mock' }) + + agent.send([{ type: 'text', text: 'run echo integration-ok' }]) + await waitForIdle(ctx, agent) + + const log = events(agent) + const toolCall = findEvent(log, 'tool/call') + expect(toolCall.data.name).toBe('bash') + + const toolResult = findEvent(log, 'tool/result') + expect(toolResult.data.isError).toBe(false) + expect(resultText(toolResult)).toBe('integration-ok\n') + + // The second model call saw the tool result in its derived history. + const lastRequest = adapter.requests.at(-1) + const toolResultBlocks = (lastRequest?.messages ?? []) + .flatMap(message => message.content) + .filter(block => block.type === 'tool-result') + expect(toolResultBlocks).toHaveLength(1) + + const finalMessage = findEvent(log, 'assistant/message', 'last') + expect(finalMessage.data.content.some( + block => block.type === 'text' && block.text.includes('integration-ok'), + )).toBe(true) + }) + + it('foreground: non-zero exit is reported in the result text, not as isError', async () => { + const adapter = new MockAdapter([ + toolCallResponse('call-1', 'bash', { command: 'exit 9', description: 'test command' }), + textResponse('It failed with code 9.'), + ]) + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create('it-exit', { model: 'mock' }) + + agent.send([{ type: 'text', text: 'run exit 9' }]) + await waitForIdle(ctx, agent) + + const toolResult = findEvent(events(agent), 'tool/result') + expect(toolResult.data.isError).toBe(false) + expect(resultText(toolResult)).toContain('[exit code: 9]') + }) + + it('background: start → poll → completion notice lands as context/message', async () => { + const adapter = new MockAdapter([ + toolCallResponse('call-1', 'bash', { command: 'echo bg-ok', description: 'test command', run_in_background: true }), + toolCallResponse('call-2', 'bash_output', {}, undefined), + textResponse('Background task finished.'), + ]) + // The second tool call needs the REAL task id from the first result; + // a tools/execute waterfall listener rewrites the scripted arguments. + let taskId = '' + + const ctx = await harness(adapter) + const agent = ctx.agentLoop.create('it-bg', { model: 'mock' }) + + // Intercept the first tool result to capture the generated task id, then + // rewrite the second scripted call's arguments to use it. + ctx.on('session/event', (_session, event) => { + if (event.type === 'tool/result' && taskId === '') { + const match = /task (bash-\d+)/.exec(resultText(event)) + if (match) taskId = match[1]! + } + }) + ctx.on('tools/execute', async (exec, next) => { + if (exec.name === 'bash_output') { + exec.arguments = { task_id: taskId } + } + return next() + }) + + agent.send([{ type: 'text', text: 'run echo bg-ok in the background' }]) + await waitForIdle(ctx, agent) + + // Wait for the background task itself (completion may race turn end). + const task = ctx.bash.get(taskId) + if (!task) throw new Error(`task ${taskId} not registered`) + await task.done + + const log = events(agent) + const firstResult = findEvent(log, 'tool/result') + expect(resultText(firstResult)).toBe(`started background task ${taskId}`) + + const notice = findEvent(log, 'context/message') + expect(notice.data.content.some( + block => block.type === 'text' && block.text.includes(`background bash task ${taskId} finished`), + )).toBe(true) + expect(notice.data.source).toEqual({ kind: 'plugin', plugin: 'tool-bash' }) + }) +}) diff --git a/packages/tool-bash/tests/tools.spec.ts b/packages/tool-bash/tests/tools.spec.ts new file mode 100644 index 0000000000..f35fc18ba6 --- /dev/null +++ b/packages/tool-bash/tests/tools.spec.ts @@ -0,0 +1,407 @@ +import { mkdtempSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { describe, expect, it, vi } from 'vitest' +import { Context } from 'cordis' +import { CallId } from '@deepseek-ai/dsh-llm' +import SystemPrompt from '@deepseek-ai/dsh-system-prompt' +import ToolRegistry from '@deepseek-ai/dsh-tools' +import { LocalBashExecutor } from '@deepseek-ai/dsh-bash-local' +import * as ToolBash from '@deepseek-ai/dsh-tool-bash' +import { renderResult } from '@deepseek-ai/dsh-tool-bash' + +const spillDir = mkdtempSync(join(tmpdir(), 'dsh-tool-bash-spec-')) + +async function setup() { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalBashExecutor, { timeoutMs: 10_000 }) + ;(ctx.bash as LocalBashExecutor).internals = { spillDir, graceMs: 200 } + await ctx.plugin(ToolBash) + return ctx +} + +let callCounter = 0 +function call(ctx: Context, name: string, args: unknown) { + return ctx.tools.execute({ callId: CallId(`call-${++callCounter}`), name, arguments: args }) +} + +function text(result: { content: { type: string; text?: string }[] }): string { + return result.content.filter(block => block.type === 'text').map(block => block.text).join('') +} + +describe('bash tool', () => { + it('returns stdout for a successful command', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'echo hello', description: 'test command' }) + expect(result.isError).toBe(false) + expect(text(result)).toBe('hello\n') + }) + + it('reports (no output) for silent commands', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'true', description: 'test command' }) + expect(text(result)).toBe('(no output)') + }) + + it('marks stderr sections', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'echo out; echo err >&2', description: 'test command' }) + expect(text(result)).toBe('out\n[stderr]\nerr\n') + expect(result.isError).toBe(false) + }) + + it('reports non-zero exits without isError', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'echo failing; exit 3', description: 'test command' }) + expect(result.isError).toBe(false) + expect(text(result)).toBe('failing\n[exit code: 3]') + }) + + it('reports timeout kills with both markers (timeout first)', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'sleep 60', description: 'test command', timeoutMs: 100 }) + expect(result.isError).toBe(false) + expect(text(result)).toBe('(no output)\n[timed out after 100ms]\n[killed by signal: SIGTERM]') + }) + + it('reports a timeout even when the command traps the signal and exits 0', async () => { + // The signal-independent timeout marker: a trapped SIGTERM that exits 0 + // after our timer fired must NOT look like a clean success. (bash may + // print "Terminated" to stderr for the killed sleep — environment + // dependent — so assert the marker, not the exact body.) + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'trap "exit 0" TERM; sleep 60', description: 'test command', timeoutMs: 100 }) + expect(result.isError).toBe(false) + expect(text(result)).toContain('[timed out after 100ms]') + expect(text(result)).not.toContain('[exit code:') + }) + + it('reports truncation with the spill path', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalBashExecutor, { maxOutputBytes: 100 }) + ;(ctx.bash as LocalBashExecutor).internals = { spillDir, graceMs: 200 } + await ctx.plugin(ToolBash) + const result = await call(ctx, 'bash', { command: 'for i in $(seq 1 100); do printf "line-%04d\\n" $i; done', description: 'test command' }) + expect(text(result)).toContain('[output truncated; full output: ') + expect(text(result)).toContain('line-0100') + }) + + it('honors workdir', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'pwd', description: 'test command', workdir: '/tmp' }) + expect(text(result).trim()).toMatch(/\/tmp$/) + }) + + it('surfaces spawn failures as isError', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'true', description: 'test command', workdir: '/nonexistent-dsh' }) + expect(result.isError).toBe(true) + expect(text(result)).toMatch(/ENOENT/) + }) + + it('surfaces aborts as isError', async () => { + const ctx = await setup() + const controller = new AbortController() + const pending = ctx.tools.execute({ + callId: CallId('call-abort'), + name: 'bash', + arguments: { command: 'sleep 60', description: 'test command' }, + signal: controller.signal, + }) + setTimeout(() => { controller.abort() }, 50) + const result = await pending + expect(result.isError).toBe(true) + expect(text(result)).toMatch(/aborted/) + }) + + it.each([ + [{}, /invalid command/], + [{ command: 42 }, /invalid command/], + [{ command: ' ' }, /invalid command/], + [{ command: 'x' }, /invalid description/], + [{ command: 'x', description: '' }, /invalid description/], + [{ command: 'x', description: 7 }, /invalid description/], + [{ command: 'x', description: 'd', timeoutMs: 'soon' }, /invalid timeoutMs/], + [{ command: 'x', description: 'd', timeoutMs: -1 }, /invalid timeoutMs/], + [{ command: 'x', description: 'd', timeoutMs: Number.NaN }, /invalid timeoutMs/], + [{ command: 'x', description: 'd', workdir: 7 }, /invalid workdir/], + [{ command: 'x', description: 'd', run_in_background: 'yes' }, /invalid run_in_background/], + ])('rejects invalid args %j', async (args, pattern) => { + const ctx = await setup() + const result = await call(ctx, 'bash', args) + expect(result.isError).toBe(true) + expect(text(result)).toMatch(pattern) + }) + + it('registers all three schemas in the system prompt assembly', async () => { + const ctx = await setup() + const names = ctx.tools.schemas().map(schema => schema.name) + expect(names).toEqual(['bash', 'bash_output', 'bash_kill']) + const bashSchema = ctx.tools.schemas()[0]! + expect(bashSchema.parameters).toMatchObject({ + type: 'object', + required: ['command', 'description'], + }) + }) + + it('unregisters everything when the plugin fiber is disposed (HMR safety)', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalBashExecutor, {}) + const fiber = await ctx.plugin(ToolBash) + expect(ctx.tools.schemas()).toHaveLength(3) + await fiber.dispose() + expect(ctx.tools.schemas()).toHaveLength(0) + }) + + it('tools depend on the executor: no registration without ctx.bash', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + // inject: ['tools', 'bash'] keeps the plugin pending until bash exists. + await ctx.plugin(ToolBash) + expect(ctx.tools.schemas()).toHaveLength(0) + await ctx.plugin(LocalBashExecutor, {}) + await new Promise(resolve => setTimeout(resolve, 0)) + expect(ctx.tools.schemas()).toHaveLength(3) + }) +}) + +describe('background tools', () => { + it('bash with run_in_background returns a task id immediately', async () => { + const ctx = await setup() + const result = await call(ctx, 'bash', { command: 'sleep 0.2; echo bg-done', description: 'test command', run_in_background: true }) + expect(result.isError).toBe(false) + expect(text(result)).toMatch(/^started background task bash-\d+$/) + }) + + it('bash_output polls incrementally and reports status', async () => { + const ctx = await setup() + const started = await call(ctx, 'bash', { command: 'echo first; sleep 0.3; echo second', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + + await new Promise(resolve => setTimeout(resolve, 150)) + const first = await call(ctx, 'bash_output', { task_id: id }) + expect(text(first)).toContain('first') + expect(text(first)).toContain('[status: running]') + + await ctx.bash.get(id)!.done + const second = await call(ctx, 'bash_output', { task_id: id }) + expect(text(second)).toContain('second') + expect(text(second)).not.toContain('first') + expect(text(second)).toContain('[status: completed, exit code: 0]') + + const third = await call(ctx, 'bash_output', { task_id: id }) + expect(text(third)).toContain('(no new output)') + }) + + it('bash_output flags lossy reads with spill paths', async () => { + const ctx = new Context() + await ctx.plugin(SystemPrompt) + await ctx.plugin(ToolRegistry) + await ctx.plugin(LocalBashExecutor, { maxOutputBytes: 100 }) + ;(ctx.bash as LocalBashExecutor).internals = { spillDir, graceMs: 200 } + await ctx.plugin(ToolBash) + + const started = await call(ctx, 'bash', { command: 'for i in $(seq 1 200); do printf "line-%04d\\n" $i; done', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + await ctx.bash.get(id)!.done + const read = await call(ctx, 'bash_output', { task_id: id }) + expect(text(read)).toContain('[some output was dropped from memory; full output: ') + }) + + it('bash_kill stops a running task; repeat reports already-finished', async () => { + const ctx = await setup() + const started = await call(ctx, 'bash', { command: 'sleep 60', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + + const killed = await call(ctx, 'bash_kill', { task_id: id }) + expect(text(killed)).toBe(`killed background task ${id}`) + await ctx.bash.get(id)!.done + + const again = await call(ctx, 'bash_kill', { task_id: id }) + expect(text(again)).toBe(`task ${id} had already finished`) + + const status = await call(ctx, 'bash_output', { task_id: id }) + expect(text(status)).toContain('[status: killed by SIGTERM]') + }) + + it('unknown task ids are isError for both tools', async () => { + const ctx = await setup() + const read = await call(ctx, 'bash_output', { task_id: 'bash-999' }) + expect(read.isError).toBe(true) + expect(text(read)).toMatch(/unknown bash task/) + const kill = await call(ctx, 'bash_kill', { task_id: 'bash-999' }) + expect(kill.isError).toBe(true) + }) + + it.each([ + ['bash_output', {}], + ['bash_output', { task_id: 9 }], + ['bash_kill', { task_id: '' }], + ])('%s rejects invalid task_id %j', async (tool, args) => { + const ctx = await setup() + const result = await call(ctx, tool, args) + expect(result.isError).toBe(true) + expect(text(result)).toMatch(/invalid task_id/) + }) + + it('injects a completion notice into the owning agent', async () => { + const ctx = await setup() + const inject = vi.fn() + const agent = { inject } as unknown as import('@deepseek-ai/dsh-agent').Agent + + const started = await ctx.tools.execute({ + callId: CallId('call-bg'), + name: 'bash', + arguments: { command: 'true', description: 'test command', run_in_background: true }, + agent, + }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + await ctx.bash.get(id)!.done + + expect(inject).toHaveBeenCalledTimes(1) + const [content, options] = inject.mock.calls[0] as [ + { type: string; text: string }[], + { source: { kind: string; plugin: string } }, + ] + expect(content[0]!.text).toContain(`background bash task ${id} finished`) + expect(content[0]!.text).toContain('bash_output') + expect(options.source).toEqual({ kind: 'plugin', plugin: 'tool-bash' }) + }) + + it('swallows ONLY the disposed-agent inject error', async () => { + const ctx = await setup() + const agent = { + inject: () => { throw new Error('agent "x" is disposed') }, + } as unknown as import('@deepseek-ai/dsh-agent').Agent + + const started = await ctx.tools.execute({ + callId: CallId('call-bg2'), + name: 'bash', + arguments: { command: 'true', description: 'test command', run_in_background: true }, + agent, + }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + await expect(ctx.bash.get(id)!.done).resolves.toBeUndefined() + }) + + it('rethrows a non-disposed inject failure (not blindly swallowed)', async () => { + const ctx = await setup() + // A real bug in inject (not the benign disposed race) must surface — the + // base-class notifier contains it (logs, does not reject task.done), but + // the listener itself must have thrown rather than silently eaten it. + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => undefined) + try { + const agent = { + inject: () => { throw new Error('unexpected inject bug') }, + } as unknown as import('@deepseek-ai/dsh-agent').Agent + + const started = await ctx.tools.execute({ + callId: CallId('call-bg3'), + name: 'bash', + arguments: { command: 'true', description: 'test command', run_in_background: true }, + agent, + }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + await ctx.bash.get(id)!.done + // notifyTaskDone caught and logged the rethrown error. + expect(errorSpy).toHaveBeenCalled() + const logged = errorSpy.mock.calls.flat().some(arg => arg instanceof Error && arg.message === 'unexpected inject bug') + expect(logged).toBe(true) + } finally { + errorSpy.mockRestore() + } + }) + + it('does not notify when no agent owned the task', async () => { + const ctx = await setup() + const started = await call(ctx, 'bash', { command: 'true', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + await expect(ctx.bash.get(id)!.done).resolves.toBeUndefined() + }) +}) + +describe('renderResult', () => { + const base = { + exitCode: 0 as number | null, + signal: null as NodeJS.Signals | null, + timedOut: false, + aborted: false, + timeoutMs: 1000, + stdout: { text: '', truncated: false }, + stderr: { text: '', truncated: false }, + } + + it('renders stderr-only output without a stdout prefix', () => { + expect(renderResult({ ...base, stderr: { text: 'err\n', truncated: false } })) + .toBe('[stderr]\nerr\n') + }) + + it('adds a separator when stdout does not end with a newline', () => { + expect(renderResult({ + ...base, + stdout: { text: 'out', truncated: false }, + stderr: { text: 'err', truncated: false }, + })).toBe('out\n[stderr]\nerr') + }) + + it('appends exit-code markers after a newline for unterminated output', () => { + expect(renderResult({ ...base, exitCode: 7, stdout: { text: 'x', truncated: false } })) + .toBe('x\n[exit code: 7]') + }) + + it('renders signal kills without the timeout marker when not timed out', () => { + expect(renderResult({ ...base, exitCode: null, signal: 'SIGKILL' })) + .toBe('(no output)\n[killed by signal: SIGKILL]') + }) + + it('reports a timeout that exited 0 (trapped signal) without a kill marker', () => { + expect(renderResult({ ...base, exitCode: 0, signal: null, timedOut: true })) + .toBe('(no output)\n[timed out after 1000ms]') + }) + + it('orders the timeout marker before a kill marker', () => { + expect(renderResult({ ...base, exitCode: null, signal: 'SIGTERM', timedOut: true })) + .toBe('(no output)\n[timed out after 1000ms]\n[killed by signal: SIGTERM]') + }) + + it('notes truncation with a fallback when the spill path is missing', () => { + expect(renderResult({ ...base, stdout: { text: 'tail', truncated: true } })) + .toBe('tail\n[output truncated; full output: (unavailable)]') + }) +}) + +describe('status lines', () => { + it('reports kills without a recorded signal (executor raced process exit)', async () => { + const ctx = await setup() + const started = await call(ctx, 'bash', { command: 'sleep 60', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + const task = ctx.bash.get(id)! + + await call(ctx, 'bash_kill', { task_id: id }) + await task.done + // Simulate the variant where the close event carried no signal. + task.signal = null + const read = await call(ctx, 'bash_output', { task_id: id }) + expect(text(read)).toContain('[status: killed]') + }) + + it('reports completed tasks with a null exit code as exit 0', async () => { + const ctx = await setup() + const started = await call(ctx, 'bash', { command: 'true', description: 'test command', run_in_background: true }) + const id = /task (bash-\d+)/.exec(text(started))![1]! + const task = ctx.bash.get(id)! + await task.done + // Defensive: completed tasks always carry an exit code in practice; the + // ?? 0 fallback covers task shapes from other executor implementations. + task.exitCode = null + const read = await call(ctx, 'bash_output', { task_id: id }) + expect(text(read)).toContain('[status: completed, exit code: 0]') + }) +}) diff --git a/packages/tool-bash/tsconfig.json b/packages/tool-bash/tsconfig.json new file mode 100644 index 0000000000..4741cb67f3 --- /dev/null +++ b/packages/tool-bash/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib" + }, + "include": ["src"], + "references": [ + { "path": "../../vendor/cosmokit" }, + { "path": "../../vendor/cordis" }, + { "path": "../llm" }, + { "path": "../tools" }, + { "path": "../agent" }, + { "path": "../bash" } + ] +} diff --git a/scripts/publint-all.ts b/scripts/publint-all.ts index b460875946..3ee0a40b73 100644 --- a/scripts/publint-all.ts +++ b/scripts/publint-all.ts @@ -10,6 +10,9 @@ const packages = [ 'packages/tools', 'packages/agent', 'packages/agent-loop', + 'packages/bash', + 'packages/bash-local', + 'packages/tool-bash', ] const root = resolve(import.meta.dirname, '..') diff --git a/tsconfig.base.json b/tsconfig.base.json index ef29d9ffbe..c9b2951e9d 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -39,7 +39,10 @@ "@deepseek-ai/dsh-system-prompt": ["./packages/system-prompt/src"], "@deepseek-ai/dsh-tools": ["./packages/tools/src"], "@deepseek-ai/dsh-agent": ["./packages/agent/src"], - "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"] + "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"], + "@deepseek-ai/dsh-bash": ["./packages/bash/src"], + "@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"], + "@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"] } } } diff --git a/tsconfig.build.json b/tsconfig.build.json index 580ff92da1..34ead19825 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -15,6 +15,9 @@ { "path": "./packages/system-prompt" }, { "path": "./packages/agent" }, { "path": "./packages/tools" }, - { "path": "./packages/agent-loop" } + { "path": "./packages/agent-loop" }, + { "path": "./packages/bash" }, + { "path": "./packages/bash-local" }, + { "path": "./packages/tool-bash" } ] } diff --git a/tsconfig.typecheck.json b/tsconfig.typecheck.json index 472560f1ea..b419d097f4 100644 --- a/tsconfig.typecheck.json +++ b/tsconfig.typecheck.json @@ -21,7 +21,10 @@ "@deepseek-ai/dsh-system-prompt": ["./packages/system-prompt/src"], "@deepseek-ai/dsh-tools": ["./packages/tools/src"], "@deepseek-ai/dsh-agent": ["./packages/agent/src"], - "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"] + "@deepseek-ai/dsh-agent-loop": ["./packages/agent-loop/src"], + "@deepseek-ai/dsh-bash": ["./packages/bash/src"], + "@deepseek-ai/dsh-bash-local": ["./packages/bash-local/src"], + "@deepseek-ai/dsh-tool-bash": ["./packages/tool-bash/src"] } }, "include": ["packages/*/src", "packages/*/tests", "examples", "scripts"] diff --git a/yarn.lock b/yarn.lock index c5c7284003..36f2b10494 100644 --- a/yarn.lock +++ b/yarn.lock @@ -177,7 +177,7 @@ __metadata: languageName: unknown linkType: soft -"@deepseek-ai/dsh-agent-loop@workspace:packages/agent-loop": +"@deepseek-ai/dsh-agent-loop@npm:^0.0.1, @deepseek-ai/dsh-agent-loop@workspace:packages/agent-loop": version: 0.0.0-use.local resolution: "@deepseek-ai/dsh-agent-loop@workspace:packages/agent-loop" dependencies: @@ -212,6 +212,29 @@ __metadata: languageName: unknown linkType: soft +"@deepseek-ai/dsh-bash-local@npm:^0.0.1, @deepseek-ai/dsh-bash-local@workspace:packages/bash-local": + version: 0.0.0-use.local + resolution: "@deepseek-ai/dsh-bash-local@workspace:packages/bash-local" + dependencies: + "@deepseek-ai/dsh-bash": "npm:^0.0.1" + cordis: "npm:^4.0.0-rc.6" + schemastery: "npm:^3.18.0" + peerDependencies: + "@deepseek-ai/dsh-bash": ^0.0.1 + cordis: ^4.0.0-rc.6 + languageName: unknown + linkType: soft + +"@deepseek-ai/dsh-bash@npm:^0.0.1, @deepseek-ai/dsh-bash@workspace:packages/bash": + version: 0.0.0-use.local + resolution: "@deepseek-ai/dsh-bash@workspace:packages/bash" + dependencies: + cordis: "npm:^4.0.0-rc.6" + peerDependencies: + cordis: ^4.0.0-rc.6 + languageName: unknown + linkType: soft + "@deepseek-ai/dsh-llm@npm:^0.0.1, @deepseek-ai/dsh-llm@workspace:packages/llm": version: 0.0.0-use.local resolution: "@deepseek-ai/dsh-llm@workspace:packages/llm" @@ -267,6 +290,28 @@ __metadata: languageName: unknown linkType: soft +"@deepseek-ai/dsh-tool-bash@workspace:packages/tool-bash": + version: 0.0.0-use.local + resolution: "@deepseek-ai/dsh-tool-bash@workspace:packages/tool-bash" + dependencies: + "@deepseek-ai/dsh-agent": "npm:^0.0.1" + "@deepseek-ai/dsh-agent-loop": "npm:^0.0.1" + "@deepseek-ai/dsh-bash": "npm:^0.0.1" + "@deepseek-ai/dsh-bash-local": "npm:^0.0.1" + "@deepseek-ai/dsh-llm": "npm:^0.0.1" + "@deepseek-ai/dsh-session": "npm:^0.0.1" + "@deepseek-ai/dsh-system-prompt": "npm:^0.0.1" + "@deepseek-ai/dsh-tools": "npm:^0.0.1" + cordis: "npm:^4.0.0-rc.6" + peerDependencies: + "@deepseek-ai/dsh-agent": ^0.0.1 + "@deepseek-ai/dsh-bash": ^0.0.1 + "@deepseek-ai/dsh-llm": ^0.0.1 + "@deepseek-ai/dsh-tools": ^0.0.1 + cordis: ^4.0.0-rc.6 + languageName: unknown + linkType: soft + "@deepseek-ai/dsh-tools@npm:^0.0.1, @deepseek-ai/dsh-tools@workspace:packages/tools": version: 0.0.0-use.local resolution: "@deepseek-ai/dsh-tools@workspace:packages/tools"