Coder/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(subprocess): migrate lsp-local, subagent-acp, and the env scrubs onto the seam

Browse Source 
Review direction (tianyicui, PR #660): in a stacked PR, change all other
process-running places to use the new service.

- lsp-local: LspConnection spawns through ctx.subprocess (piped protocol
  streams + a no-spill collected stderr tail); its private process-tree
  helpers (POSIX group signalling, Windows taskkill, liveness polling) are
  deleted in favor of the seam's handle verbs, and its buildChildEnv now
  rides scrubbedParentEnv (LSP children also stop inheriting stale DSH_*).
  The plugin injects 'subprocess'; compositions/tests mount
  dsh-subprocess-local.
- subagent-acp: the ACP child spawns through the seam (piped ndjson streams,
  inherited stderr); spawn failure surfaces through done-rejection into the
  same startup race; disposal is handle.dispose with the plugin's configured
  graces. dsh-subagent-subprocess is DELETED — its dispose ladder and scrub
  are the seam's, and the isolated-config-dir helper had no consumer.
- mcp-client, pty-local, sdk-helper: adopt scrubbedParentEnv as the one
  scrub definition (their spawns stay put by ownership: the MCP SDK and
  node-pty own those calls; the SDK wizard runs outside any composition).
- Coverage: per-file 100% over every touched src file, with each v8 ignore
  carrying a platform or contract reason; new suites cover stdio
  dispositions, the dispose ladder tiers, injected-win32 tree semantics,
  waitForExit, settled-kill/terminate no-ops, and spawn-failure disposal.
- Docs: consumer-migration Agent Note (en; zh follows in this PR), seam note
  updated in place, subprocess.md rewritten for the reshaped vocabulary
  (type-equiv re-registered), READMEs and SERVICE_ROLES updated, taskkill
  added to knip ignoreBinaries.
This commit is contained in:
Tianyi Cui
2026-07-26 15:27:59 +08:00
parent fd7e6d4d11
commit 3672cd25b4
57 changed files with 750 additions and 1225 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
packages/subprocess/README.md
View File

@@ -1,10 +1,10 @@
# subprocess/ — subprocess capability family
The shared home for spawning managed child-process groups: fully-specified spawn specs, bounded tail-keep output with spill files, credential-scrubbed environments, offset-based incremental reads, and SIGTERM→grace→SIGKILL group kills. Command defaulting, shell semantics, deadlines, and presentation stay with consumers — the [bash executor family](../bash/README.md) is the first and owning consumer. See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md).
The shared home for spawning managed child-process trees: fully-specified spawn specs with Node-shaped per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with spill files), the one credential scrub every harness spawner uses, offset-based incremental reads, tree-scoped signalling with SIGTERM→grace→SIGKILL escalation, and the cooperative dispose ladder. Command defaulting, shell semantics, deadlines, protocol framing, and presentation stay with consumers — the [bash executors](../bash/README.md), the [LSP host](../lsp/README.md), and the [ACP subagent backend](../subagent/README.md). See the [subprocess seam Agent Note](../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md).
| Package | ctx key | Role |
|---|---|---|
| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec`, `SubprocessHandle` with offset-based readers, and the shared `DSH_*` managed-environment and `CollectedOutput` vocabulary |
| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process groups, tail-keep truncation with bounded private spill files, the credential scrub and `DSH_*` merge order, kill escalation, and kill-and-join disposal |
| [`subprocess`](subprocess/README.md) (`@deepseek-ai/dsh-subprocess`) | `ctx.subprocess` | The seam: abstract `SubprocessService.spawn(spec)`, the fully-explicit `SubprocessSpawnSpec` with per-stream stdio dispositions, `SubprocessHandle` (streams, offset-based readers, kill/terminate/waitForExit/dispose), and the shared scrub + `DSH_*`/`CollectedOutput` vocabulary |
| [`subprocess-local`](subprocess-local/README.md) (`@deepseek-ai/dsh-subprocess-local`) | — | The local implementation: detached process trees, per-disposition stream wiring, tail-keep truncation with bounded private spill files, the `DSH_*` merge order, tree signalling with escalation, the dispose ladder, and terminate-and-join disposal |
The service owns process lifetime across consumer reloads; consumers own what a process means (a bash command, a future non-shell runner) and every default that shapes one.

10
packages/subprocess/subprocess-local/README.md
View File

@@ -1,14 +1,14 @@
# @deepseek-ai/dsh-subprocess-local
Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam: `LocalSubprocessService` spawns each spec's argv as a detached process group, collects bounded output with size-limited full-stream spill files, and escalates kills SIGTERM→SIGKILL across the whole group. It has no config: every limit and directory arrives on the spawn spec, so the deployment-varying knobs stay with the calling seam's config ([`dsh-bash-local`](../../bash/bash-local/README.md) today).
Local implementation of the [`@deepseek-ai/dsh-subprocess`](../subprocess/README.md) seam: `LocalSubprocessService` spawns each spec's argv as a detached process tree, wires the spec's per-stream stdio dispositions (raw pipes, inherit, bounded tail-keep collection with optional spill files), and signals tree-scoped with SIGTERM→SIGKILL escalation. It has no config: every disposition, limit, and directory arrives on the spawn spec, so the deployment-varying knobs stay with the calling seams' configs ([`dsh-bash-local`](../../bash/bash-local/README.md), [`dsh-lsp-local`](../../lsp/lsp-local/README.md), [`dsh-subagent-acp`](../../subagent/subagent-acp/README.md)).
## Behavior (and where it came from)
- **Detached process groups with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent). After the leader exits, inherited stdout/stderr pipes receive the same bounded drain grace so a surviving descendant cannot hold the spawn open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools.
- **Tail-keep truncation + bounded spill files** — output beyond a stream's cap keeps the in-memory TAIL (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file whose path is reported when available. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory.
- **Detached process trees with platform-correct signalling** — POSIX children are spawned `detached` (own process group) and signalled by negative pgid with a direct-child fallback; Windows terminates the tree via `taskkill /PID <pid> /T /F` (injectable for tests). `terminate()` sends SIGTERM then SIGKILL after the spec's grace (OpenCode's escalation; pipelines and subshells die with the parent); `kill(signal)` sends exactly one signal and is a no-op after settlement; `dispose(graces)` runs stdin-EOF → SIGTERM → SIGKILL with caller-supplied windows and one memoized disposal per handle. After the leader exits, still-open pipes receive the same bounded drain grace so a surviving descendant cannot hold the outcome open indefinitely. ESRCH is tolerated; daemons that re-parent away from the group can still survive — the same caveat as the surveyed tools.
- **Per-stream dispositions** — `'pipe'` hands the raw stream to the caller untouched (protocol framing stays consumer-owned); `'inherit'` passes the parent descriptor through; collect mode keeps the in-memory TAIL beyond its cap (errors and results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a private temp file when a spill cap is configured — omitting `spill` keeps only the tail, the diagnostic shape. A stream larger than the spill cap discards its now-incomplete spill and returns only the marked truncated tail; spill fds are sealed at settlement, and a failed final close withholds the path rather than advertising an incomplete file. Spill files are `0600` with random names under a lazily-created `0700` per-process directory.
- **Credential scrub + managed `DSH_*` merge** — `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and all ambient `DSH_*` names; a spec's ordinary `env` merges after the scrub but rejects `DSH_*`; managed `dshEnv` rejects ordinary names and merges last, preventing stale nested-harness identity. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env Agent Note](../../../.agents/notes/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment Agent Note](../../../.agents/notes/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md).
- **Offset-based reads** — `SubprocessHandle` readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist.
- **Kill-and-join disposal** — the service retains live handles only so its own disposal can kill every running group and await its exit; settled and spawn-failed handles leave the live set on settlement.
- **Offset-based reads** — collect-mode readers return deltas in whole-stream byte coordinates; the service never holds a cursor, so consumer-owned cursors (the bash background read path) and full-stream re-reads coexist, before and after settlement.
- **Terminate-and-join disposal** — the service retains live handles only so its own disposal can escalate every running tree and await its exit; settled and spawn-failed handles leave the live set on settlement.
## Model Experience

18
packages/subprocess/subprocess-local/src/spawn.ts
View File

@@ -258,8 +258,9 @@ export function killGroup(pid: number, sig: NodeJS.Signals): void {
*/
export function taskkillProcessTree(pid: number): void {
if (pid <= 0) return
// Outcome deliberately unchecked: an already-absent tree (status 128) and
// exit races are as tolerable here as ESRCH is for a POSIX group signal.
// Outcome deliberately unchecked: an already-absent tree (status 128), exit
// races, and a missing taskkill binary (spawnSync reports, never throws) are
// as tolerable here as ESRCH is for a POSIX group signal.
spawnSync('taskkill', ['/PID', String(pid), '/T', '/F'], { stdio: 'ignore' })
}
@@ -284,11 +285,14 @@ function signalTree(
try {
process.kill(-pid, sig)
} catch {
/* v8 ignore start -- the fallback needs a live child whose group signal fails
(EPERM-style), which POSIX CI cannot stage; the swallow keeps teardown idempotent. */
try {
child.kill(sig)
} catch {
// The direct child already exited; teardown remains idempotent.
}
/* v8 ignore stop */
}
}
@@ -360,6 +364,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter
if (settled) return
signalTree(platform, pid, 'SIGTERM', child, taskkill)
graceTimer = setTimeout(() => {
/* v8 ignore next -- the timer is cleared at settlement; only an in-flight fire racing the close event sees settled=true. */
if (!settled) signalTree(platform, pid, 'SIGKILL', child, taskkill)
}, spec.graceMs)
}
@@ -422,6 +427,8 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter
return true
} catch (error) {
const code = (error as NodeJS.ErrnoException).code
/* v8 ignore next -- POSIX reports an absent group as ESRCH; child-reaping timing
makes observing the other arm platform-dependent. */
if (code === 'ESRCH') return false
/* v8 ignore start -- EPERM and non-POSIX negative-pid failures are platform defenses; CI runs
tree-lifecycle tests on POSIX hosts where absence reports ESRCH. */
@@ -442,7 +449,8 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter
/** Race settlement against a timer without leaving listeners or live timers behind. */
const settlesWithin = async (ms: number): Promise<boolean> => {
if (settled) return true
let timer: NodeJS.Timeout | undefined
// The executor runs synchronously, so the timer is assigned before the race.
let timer!: NodeJS.Timeout
const timeout = new Promise<false>((resolve) => {
// `.unref()` so a pending grace timer never keeps the parent's loop alive.
timer = setTimeout(() => { resolve(false) }, ms)
@@ -451,7 +459,7 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter
try {
return await Promise.race([done.then(() => true, () => true), timeout])
} finally {
if (timer !== undefined) clearTimeout(timer)
clearTimeout(timer)
}
}
@@ -474,9 +482,11 @@ export function spawnSubprocess(spec: SubprocessSpawnSpec, internals: SpawnInter
return {
pid,
/* v8 ignore start -- pipe-mode fds exist on every spawn Node returns; the null-coalesces guard a nonconforming ChildProcess only. */
stdin: stdinMode === 'pipe' ? child.stdin ?? undefined : undefined,
stdout: outMode === 'pipe' ? child.stdout ?? undefined : undefined,
stderr: errMode === 'pipe' ? child.stderr ?? undefined : undefined,
/* v8 ignore stop */
collected: {
...stdoutCollector !== undefined ? { stdout: stdoutCollector } : {},
...stderrCollector !== undefined ? { stderr: stderrCollector } : {},

143
packages/subprocess/subprocess-local/tests/spawn.spec.ts
View File

@@ -3,7 +3,7 @@ import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { describe, expect, it, vi } from 'vitest'
import type { DshEnvironment } from '@deepseek-ai/dsh-subprocess'
import { killGroup, OutputCollector, spawnSubprocess } from '../src/spawn.ts'
import { killGroup, OutputCollector, spawnSubprocess, taskkillProcessTree } from '../src/spawn.ts'
import type { SubprocessHandle, SubprocessOutputReader } from '@deepseek-ai/dsh-subprocess'
const { failNextClose, failNextUnlink } = vi.hoisted(() => ({
@@ -575,6 +575,147 @@ describe('waitForExit', () => {
})
})
describe('coverage seams', () => {
it('taskkillProcessTree ignores non-positive pids and contains a missing binary', () => {
expect(() => { taskkillProcessTree(-1) }).not.toThrow()
expect(() => { taskkillProcessTree(0) }).not.toThrow()
// On POSIX there is no taskkill; spawnSync reports the failure in its
// result and the function stays silent — the same containment Windows
// relies on for an already-absent tree.
expect(() => { taskkillProcessTree(2 ** 30) }).not.toThrow()
})
it('dispose on a spawn-failed handle observes the rejection and returns', async () => {
const running = spawnSubprocess(spec('true', { cwd: '/nonexistent-dir-dsh-dispose-test' }))
const disposal = running.dispose({ eofGraceMs: 1_000, graceMs: 1_000 })
await expect(running.done).rejects.toThrow()
await expect(disposal).resolves.toBeUndefined()
})
it("an 'inherit' stdout with collected stderr wires only the requested collector", async () => {
const running = spawnSubprocess({
...spec('echo to-parent; echo err >&2'),
stdio: { stdin: 'ignore', stdout: 'inherit', stderr: { maxBytes: 1000 } },
})
const outcome = await running.done
expect(outcome.exitCode).toBe(0)
expect(running.stdout).toBeUndefined()
expect(running.collected.stdout).toBeUndefined()
expect(running.collected.stderr!.readFrom(0).text).toBe('err\n')
})
it('terminate() after settlement is a no-op', async () => {
const running = spawnSubprocess(spec('true'))
await running.done
const spy = vi.spyOn(process, 'kill')
try {
running.terminate()
expect(spy).not.toHaveBeenCalled()
} finally {
spy.mockRestore()
}
})
it('waitForExit on a failed spawn reports exited immediately', async () => {
const running = spawnSubprocess(spec('true', { cwd: '/nonexistent-dir-dsh-spawn-test' }))
await expect(running.done).rejects.toThrow()
await expect(running.waitForExit()).resolves.toBe(true)
})
it('dispose() on an already-settled handle returns without signalling', async () => {
const running = spawnSubprocess(spec('true'))
await running.done
const spy = vi.spyOn(process, 'kill')
try {
await running.dispose({ eofGraceMs: 50, graceMs: 50 })
expect(spy).not.toHaveBeenCalled()
} finally {
spy.mockRestore()
}
})
it('a batch-stdin handle exposes no stdin and dispose skips the EOF tier', async () => {
const running = spawnSubprocess(spec('cat', { stdin: 'batch\n' }))
expect(running.stdin).toBeUndefined()
await running.done
await running.dispose({ eofGraceMs: 50, graceMs: 50 })
expect(running.collected.stdout!.readFrom(0).text).toBe('batch\n')
})
})
describe('coverage seams 2', () => {
it('win32 treeAlive reports alive for a live child and gone after taskkill', async () => {
let killedPid = 0
const running = spawnSubprocess(spec('sleep 60'), {
spillDir,
platform: 'win32',
taskkill: (pid) => {
killedPid = pid
try {
process.kill(pid, 'SIGKILL')
} catch {
// Already gone.
}
},
})
const aborted = new AbortController()
aborted.abort()
await expect(running.waitForExit(aborted.signal)).resolves.toBe(false) // alive branch
running.terminate()
await running.done
expect(killedPid).toBe(running.pid)
await expect(running.waitForExit()).resolves.toBe(true)
})
it('the win32 dispose ladder skips the POSIX SIGTERM tier and force-terminates', async () => {
const kills: number[] = []
const running = spawnSubprocess({
...spec('sleep 60'),
stdio: { stdin: 'pipe', stdout: { maxBytes: 1000 }, stderr: { maxBytes: 1000 } },
}, {
spillDir,
platform: 'win32',
taskkill: (pid) => {
kills.push(pid)
try {
process.kill(pid, 'SIGKILL')
} catch {
// Already gone.
}
},
})
await running.dispose({ eofGraceMs: 50, graceMs: 5_000 })
// Exactly one forced tree termination: no POSIX SIGTERM tier ran.
expect(kills).toEqual([running.pid])
})
it('dispose throws when even SIGKILL produces no exit within the grace', async () => {
// An inert taskkill simulates a tree that never reports exit.
const running = spawnSubprocess(spec('sleep 60'), { spillDir, platform: 'win32', taskkill: () => {} })
await expect(running.dispose({ eofGraceMs: 20, graceMs: 40 }))
.rejects.toThrow(/did not exit within 40ms after forced termination/)
// Real cleanup: the injected platform spawned without detachment, so the
// child is a plain (group-less) POSIX process — kill it directly.
process.kill(running.pid, 'SIGKILL')
await running.done
})
it("stderr: 'pipe' exposes the raw stream", async () => {
const running = spawnSubprocess({
...spec('echo err >&2'),
stdio: { stdin: 'ignore', stdout: { maxBytes: 1000 }, stderr: 'pipe' },
})
expect(running.stderr).toBeDefined()
const text = new Promise<string>((resolve) => {
let out = ''
running.stderr!.on('data', (chunk: Buffer) => { out += chunk.toString('utf8') })
running.stderr!.on('end', () => { resolve(out) })
})
await running.done
expect(await text).toBe('err\n')
})
})
describe('argv validation', () => {
it('rejects an empty argv before spawning', () => {
expect(() => spawnSubprocess({ ...spec('true'), argv: [] })).toThrow(/non-empty program name/)

17
packages/subprocess/subprocess/README.md
View File

@@ -4,13 +4,14 @@ The subprocess seam (`ctx.subprocess`). The abstract `SubprocessService` exposes
## Contract
- `spawn(spec)` returns immediately with a live handle; `done` resolves at process close and rejects only for spawn-level failures.
- The spec is fully explicit — argv, cwd, per-stream byte caps, spill cap, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted here; a consumer that wants a shell passes `['bash', '-c', command]` itself.
- Output readers take whole-stream byte offsets and never consume: independent readers cannot steal one another's deltas. A read whose offset slid out of the in-memory tail is `lossy` and points at the full-stream spill file when one exists.
- `kill()` and the spec's abort signal escalate SIGTERM→grace→SIGKILL across the whole detached group; the service reacts to the abort but never classifies why (callers own deadlines and cause classification).
- Disposal kills all still-running managed processes and awaits their exit.
- `spawn(spec)` returns immediately with a live handle; `done` resolves at process close with exit facts (`SubprocessOutcome` carries no output and no cause classification) and rejects only for spawn-level failures.
- The spec is fully explicit — argv, cwd, per-stream stdio dispositions, grace — because deployment-varying defaults belong to the calling seam's config, not to a hidden subprocess-service default (the `dsh-bash` request/spec split is the owning template). `argv` is never shell-interpreted; a consumer that wants a shell passes `['bash', '-c', command]` itself.
- Stdio is Node-shaped per stream: `'pipe'` hands the caller the raw stream for its own protocol framing (LSP JSON-RPC, ACP ndjson), `'inherit'` passes the parent descriptor through for diagnostics, and collect mode (`{ maxBytes, spill? }`) buffers a bounded tail with an optional full-stream spill file. Collect readers take whole-stream byte offsets and never consume, so independent readers cannot steal one another's deltas; a read whose offset slid out of the in-memory tail is `lossy` and points at the spill file when one exists. Collected output stays readable after settlement.
- Termination is tree-scoped on every platform (POSIX detached groups with direct-child fallback; Windows `taskkill /T`): `kill(signal)` sends one signal Node-style and is a no-op after settlement, `terminate()` (and the spec's abort signal) escalates SIGTERM→grace→SIGKILL, `waitForExit()` observes the whole tree, and `dispose(graces)` runs the cooperative stdin-EOF→SIGTERM→SIGKILL ladder out-of-process children need — the manager reacts but never classifies why (callers own deadlines and cause classification).
- `scrubbedParentEnv()` / `SENSITIVE_ENV_PATTERN` are the one shared scrub definition: ambient credential-shaped and `DSH_*` names are dropped, explicit `env` merges after the scrub (a deliberately forwarded key survives), and `dshEnv` carries current harness facts on its own validated channel. Spawners that cannot route through the service (node-pty backends, SDK-managed transports) import the function.
- Disposal of the service terminates all still-running managed processes and awaits their exit.
See the [process data-structure catalog](../../../docs/core-data-structures/subprocess.md) and the [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md).
See the [subprocess data-structure catalog](../../../docs/core-data-structures/subprocess.md) and the [seam Agent Note](../../../.agents/notes/implemented/architecture/2026-07-26-subprocess-seam.md).
## Model Experience
@@ -22,5 +23,5 @@ No direct invalidation; the named consumers own any request-prefix changes.
## Known Limitations and Deferred Work
- **One consumer family so far** — the seam's shape is proven against the bash executors only; the other in-repo spawn sites (LSP servers, PTY backends, subagent transports) keep their own bespoke process handling until their stream/lifecycle needs are re-examined against this contract.
- **POSIX group semantics are assumed** — the handle vocabulary (`pid` as group leader, group kills, SIGTERM/SIGKILL escalation) has no Windows story.
- **node-pty and SDK-managed spawns share only the scrub** — the PTY backend's terminal fork and the MCP SDK's own stdio transport cannot route their spawns through this seam (the library owns the fork/spawn call); they import `scrubbedParentEnv` so the environment policy stays single-sourced.
- **The dispose ladder assumes stdin-EOF cooperation** — a child that quiesces on a different signal (SIGHUP conventions, control sockets) needs its own tier-1 before the generic ladder fits.