Merge remote-tracking branch 'origin/master' into codex/agent-session-jsonl-location

# Conflicts:
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/bash.md
#	docs/module-graph.md
#	docs/rfc/INDEX.md
#	docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md
#	docs/rfc/implemented/feature/2026-06-30-hook-bridges.md
#	examples/acp-agent/tests/snapshots/advanced-toolchain/session.1.jsonl
#	examples/acp-agent/tests/snapshots/advanced-toolchain/session.2.jsonl
#	examples/acp-agent/tests/snapshots/advanced-toolchain/session.jsonl
#	examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md
#	examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl
#	examples/acp-agent/tests/snapshots/both-mode-turn/system-prompt.golden.md
#	examples/acp-agent/tests/snapshots/code-mode-turn/system-prompt.golden.md
#	examples/acp-agent/tests/snapshots/skill-load/session.jsonl
#	examples/acp-agent/tests/snapshots/text-turn/session.jsonl
#	examples/sandbox-acp-agent/tests/snapshots/mode-switching/session.jsonl
#	packages/bash/bash-local/README.md
#	packages/bash/bash-local/src/run.ts
#	packages/bash/bash-local/tests/run.spec.ts
#	packages/bash/bash/README.md
#	packages/bash/tool-bash/README.md
#	packages/bash/tool-bash/src/index.ts
#	packages/bash/tool-bash/tests/tools.spec.ts
#	packages/cordis/tool-cordis/src/api-catalog.ts
#	packages/core/agent-core/src/index.ts
#	packages/session-persistence/session-persistence-jsonl/src/index.ts
#	packages/session-persistence/session-persistence-sqlite/src/index.ts
#	packages/session-persistence/session-persistence/README.md
#	packages/session-persistence/session-persistence/src/index.ts
#	packages/ui/acp-agent/README.md
#	packages/ui/stdio-agent/README.md
This commit is contained in:
Yichen Jiang
2026-07-14 18:04:05 +08:00
672 changed files with 10233 additions and 14251 deletions

View File

@@ -22,9 +22,20 @@ Design surveyed against the bash tools of Claude Code, OpenCode, Codex, and pi;
- **Spawn per call, no shell state** — every call is a fresh non-login `bash -c` (deterministic; no rc files). All four surveyed tools spawn per call. `XXX(stateful-shell)` in `src/run.ts` records the two proven stateful designs (Claude Code's cwd-only persistence; Codex's PTY exec sessions) for when real workflows demand them.
- **Process-group kills with escalation** — children are spawned `detached` (own process group); kills send SIGTERM to the group, then SIGKILL after the `graceMs` grace (default 3s — OpenCode's escalation; pipelines and subshells die with the parent). ESRCH is tolerated; daemons that re-parent away from the group can still survive — same caveat as the surveyed tools.
- **Tail-keep truncation + spill files** — output beyond `maxOutputBytes` keeps the in-memory TAIL (errors/results cluster at the end — pi/OpenCode rationale) while the FULL stream is appended to a temp file whose path is reported when available. If the final spill close reports a delayed writeback failure, the executor still returns the tail but withholds the path rather than advertising a possibly incomplete file.
- **Model-friendly env + credential/namespace scrub** — start with `process.env` minus credential-shaped vars (`*KEY*`/`*SECRET*`/`*TOKEN*`) and every `DSH_*`, then apply `NO_COLOR=1 TERM=dumb PAGER=cat GIT_PAGER=cat`. Ordinary spec `env` is merged next and may restore caller-held credential-shaped values, but is rejected if it tries to set reserved `DSH_*`; the managed spec `dshEnv` snapshot is rejected if it contains ordinary names and otherwise merges last. This keeps ambient secrets out, catches wrong-channel plugin configuration before spawn, and prevents a nested/previous harness identity from surviving when the current registry omits it. The spec's `stdin`, when supplied, is written to the child and closed; with none supplied, fd 0 is `/dev/null`. See [the bash-stdin-env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [the session environment RFC](../../../docs/rfc/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md).
- **Model-friendly environment** — ambient credentials and all `DSH_*` names are removed before noninteractive terminal defaults and ordinary caller `env` are applied. Ordinary `env` rejects `DSH_*`; managed `dshEnv` rejects other names and merges last, preventing stale nested-harness identity. Supplied stdin is written and closed; otherwise fd 0 is `/dev/null`. See the [stdin/env RFC](../../../docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md) and [managed environment RFC](../../../docs/rfc/implemented/feature/2026-07-10-agent-session-identity-and-log-location.md).
- **Background tasks** — `start()` returns immediately, no timeout applies (Claude Code detaches timeouts when backgrounding), `readOutput()` is incremental with whole-stream byte offsets, and disposal kills everything. The spec's opaque `owner` token is stored on the tracked task and returned by `ownerOf(id)` — the executor never interprets it (the consumer's access policy does), and because it lives with the task here it survives a `tool-bash` HMR reload.
## Sandboxing
## Model Experience
Execution policy does NOT belong in this package: this executor always runs commands unconfined. Confinement is [`dsh-bash-sandbox`](../bash-sandbox/README.md), which extends this executor verbatim and confines commands under the `ctx.sandbox` seam's bwrap/Landlock/Seatbelt backends ([sandbox RFC](../../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)); per-call allow/deny/ask policy belongs on the `tools/pre-execute` gate.
Indirectly, through `dsh-tool-bash`, which renders this executor's bounded stdout/stderr tails, background-task deltas and state, spill-file path, exact `Error: unknown bash task "<taskId>"` and `Error: aborted before spawn: <reason>` failures, and retains each resulting tool message until compaction.
## Known Limitations and Deferred Work
- **Unconfined by itself** — this executor always runs commands with the harness process's authority; deployments needing confinement compose [`dsh-bash-sandbox`](../bash-sandbox/README.md), while per-call allow/deny/ask policy belongs on `tools/pre-execute`.
- **No persistent shell or PTY** — every call starts a fresh non-login `bash -c`; cwd-only persistence and interactive terminal sessions remain deferred until a real workflow requires them.
- **POSIX-only** — the `bash` binary, detached process groups, group kills, and SIGTERM→SIGKILL escalation are hardcoded; Windows is unsupported.
- **The credential scrub is a name heuristic** — `*KEY*`/`*SECRET*`/`*TOKEN*` only; differently-named secrets (e.g. `*PASSWORD*`) pass through, and a whitelist for over-scrubbed vars is noted future work.
- **Spill files are never deleted** — full-output recovery files (and the private per-process spill dir) accumulate under the OS tmpdir until something external cleans them.
- **Finished background tasks are never evicted** — they stay in the task map, retaining their in-memory output tails, until executor disposal.
The raw process handling lives in `src/run.ts`; `src/index.ts` is the service wiring.

View File

@@ -1,16 +1,8 @@
/**
* `LocalBashExecutor`: the local-subprocess implementation of the
* `@deepseek-ai/dsh-bash` executor seam. Spawns `bash -c` per call in its
* own process group (see `./run.ts` for the plumbing and the agent-tool
* survey notes), tracks background tasks, and kills everything on dispose.
*
* TODO(permissions/sandbox): execution policy does NOT belong here — use
* the `tools/pre-execute` deny/ask gate (see docs/architecture.md
* § Extending The Harness) or implement a sandboxing `BashExecutor`.
* Reference points:
* Claude Code wraps commands in sandbox-exec/bubblewrap; Codex applies
* seatbelt/landlock plus an execpolicy prefix-rule engine.
*
* Local-subprocess implementation of the bash seam. Each call runs in its own
* process group, background tasks are tracked, and disposal kills and awaits
* them. Execution policy belongs in `tools/pre-execute` or a sandboxing
* executor, not this local process layer.
* @module @deepseek-ai/dsh-bash-local
*/
@@ -90,10 +82,9 @@ export class LocalBashExecutor extends BashExecutor {
assertPositiveFinite('maxOutputBytes', this.config.maxOutputBytes)
assertPositiveFinite('graceMs', this.config.graceMs)
ctx.effect(() => async () => {
// Kill every live process group and WAIT for the processes to close so
// nothing outlives the fiber (HMR safety) — a TERM-trapping child is
// held until the SIGKILL escalation lands. The base class already
// silenced listeners, so these kills complete without notices.
// Kill every live process group and WAIT for the processes to close so nothing outlives
// the fiber (HMR safety) — a TERM-trapping child is held until the SIGKILL escalation
// lands.
const pending: Promise<void>[] = []
for (const task of this.tasks.values()) {
if (task.status === 'running') {
@@ -156,23 +147,18 @@ export class LocalBashExecutor extends BashExecutor {
env: spec.env,
dshEnv: spec.dshEnv,
}, this.internals).done
// Classify the FIRST abort reason: a BASH_TIMEOUT TimeoutReason means our
// timeout cut the command short; any other abort — an upstream cancel, or a
// foreign (outer) deadline's timeout under nesting — is aborted. Scoping to
// our own code keeps a nested outer deadline from reading as our timeout.
// Mutually exclusive by construction — the fused signal reports one cause.
// Classify the FIRST abort reason: a BASH_TIMEOUT TimeoutReason means our timeout cut the
// command short; any other abort — an upstream cancel, or a foreign (outer) deadline's
// timeout under nesting — is aborted.
const timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined
const aborted = d.signal.aborted && !timedOut
return { ...outcome, timedOut, aborted, timeoutMs: spec.timeoutMs }
}
start(spec: BashExecSpec): BashTask {
// No timeout for background tasks (matches Claude Code, which detaches
// the timeout when backgrounding); callers stop tasks via kill() — or
// via spec.signal, which the seam contract honors for background runs
// too (runBash wires it to the group kill). No deadline is created here,
// so spec.timeoutMs is ignored by design — background tasks stay
// timeout-free (see the timeout-library RFC).
// No timeout for background tasks (matches Claude Code, which detaches the timeout when
// backgrounding); callers stop tasks via kill() — or via spec.signal, which the seam
// contract honors for background runs too (runBash wires it to the group kill).
const running = runBash({
command: spec.command,
cwd: spec.workdir,

View File

@@ -1,23 +1,7 @@
/**
* Process plumbing for the local bash executor: spawn, output collection
* with tail-keep + spill-to-disk truncation, and process-group kill with
* SIGTERM→SIGKILL escalation.
*
* Everything here is deliberately free of Cordis concepts so it can be unit
* tested in isolation; `LocalBashExecutor` owns lifecycle and configuration.
*
* runBash owns NO timing: it kills the process group when its `spec.signal`
* fires and does not distinguish a timeout from a cancel. The executor fuses
* timeout + upstream cancellation into that one signal via
* `@deepseek-ai/dsh-timeout`'s `deadline`, and classifies the outcome from the
* signal afterward — the timing/classification half is shared, the kill is not.
*
* Design notes (surveyed against Claude Code, OpenCode, Codex, and pi — see
* the package README): spawn-per-call with `detached: true` so the child
* leads its own process group; kills target the group (`kill(-pid)`) so
* pipelines and subshells die with the parent. SIGTERM first, SIGKILL after a
* grace period (OpenCode's escalation; Codex/pi jump straight to SIGKILL).
*
* Process plumbing for the local bash executor: detached process-group spawn,
* tail-keep output with spill files, and SIGTERM→SIGKILL escalation. This layer
* reacts to an abort signal; the executor owns deadlines and classifies causes.
* @module dsh-bash-local/run
*/
@@ -52,17 +36,11 @@ export const SENSITIVE_ENV_PATTERN = /KEY|SECRET|TOKEN/i
/**
* Build a child environment from scrubbed ambient values, terminal overrides,
* ordinary caller entries, and a trusted managed `DSH_*` snapshot.
*
* Ambient credentials and all ambient `DSH_*` are removed first;
* `ENV_OVERRIDES` then forces model-friendly terminal values, ordinary `extra`
* follows, and `dshEnv` merges last. Ordinary `extra` may restore a
* credential-shaped name whose value the caller already holds, but cannot set
* the managed namespace; `dshEnv` rejects ordinary names symmetrically.
* `dsh-tool-bash` builds both channels from trusted named fields and never
* forwards model-provided environment objects.
* @param extra - ordinary caller-supplied entries; `DSH_*` names are rejected.
* @param dshEnv - managed entries; names outside `DSH_*` are rejected.
* ordinary caller entries, and a managed `DSH_*` snapshot. Ambient managed
* names are removed; ordinary and managed entries reject the other channel's
* namespace before `dshEnv` merges last.
* @param extra - caller entries; `DSH_*` names are rejected.
* @param dshEnv - managed entries; non-`DSH_*` names are rejected.
* @returns the environment to hand to `spawn` for the child process.
*/
export function childEnv(
@@ -275,10 +253,8 @@ export class OutputCollector {
try {
closeSync(this.spillFd)
} catch {
// close can surface delayed writeback failures (for example EIO/ENOSPC)
// after writeSync appeared to succeed. Keep finalize total so runBash's
// close handler still resolves, but stop advertising a spill file that
// may be missing its tail.
// A delayed writeback failure makes the spill unreliable; keep finalize
// total but stop advertising that file.
this.spillFile = undefined
}
this.spillFd = undefined
@@ -288,13 +264,9 @@ export class OutputCollector {
}
/**
* Send `sig` to the process GROUP led by `pid` (requires the child to have
* been spawned with `detached: true`). NEVER throws: kills race process exit
* by design (ESRCH), and the other failure modes (EPERM from setuid
* children, …) fire inside timer callbacks where a throw would crash the
* host process — a kill that cannot be delivered is reported by the process
* NOT dying, which callers already handle via escalation/timeouts. No-op for
* non-positive pids (spawn never started a process).
* Send `sig` to a detached process group. Never throws: delivery races process
* exit and may run in a timer callback, so failures are contained and a
* non-positive pid is a no-op.
* @param pid - the group leader's pid; non-positive means the spawn failed and the call is a no-op.
* @param sig - the signal to deliver to the whole group.
*/
@@ -324,24 +296,13 @@ export interface RunningBash {
}
/**
* Spawn `bash -c <command>` in its own process group and collect output.
*
* Outcome semantics: the returned promise REJECTS only for spawn-level
* failures (bad cwd → ENOENT, missing binary, pre-aborted signal); every
* runtime outcome — nonzero exit, timeout kill, abort kill, signal death —
* RESOLVES with a {@link SpawnOutcome} describing what happened, so callers
* shape one consistent report for the model.
*
* XXX(stateful-shell): per the agent-tool survey there are two proven
* stateful designs worth revisiting — Claude Code persists ONLY cwd between
* calls (captures `pwd -P` after each command), and Codex keeps whole PTY
* exec sessions addressable via session ids + stdin writes. We deliberately
* spawn a fresh non-login `bash -c` per call for determinism (no rc files,
* no inherited shell state); revisit when real workflows demand it.
* @param spec - the fully-resolved run (command, cwd, limits); no defaulting happens here.
* @param internals - test-only knobs; omitted fields fall back to the private per-process spill dir.
* @returns the live handle: pid, the two live collectors, the outcome promise, and `kill()`.
* Spawn one isolated `bash -c` process group and collect its output.
* Runtime exits resolve as {@link SpawnOutcome}; only spawn failures reject.
* @param spec - fully resolved command, cwd, limits, and cancellation.
* @param internals - test-only process and spill-directory overrides.
* @returns live process handle and outcome promise.
*/
// XXX(stateful-shell): evaluate persistent cwd or PTY sessions when workflows require shell state.
export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningBash {
const spillDir = internals.spillDir ?? privateSpillDir()
@@ -349,16 +310,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
throw new Error(`aborted before spawn: ${String(spec.signal.reason ?? 'aborted')}`)
}
// stdin is a pipe ONLY when the caller supplied bytes; with none it is `ignore`
// (fd 0 → /dev/null) — the exact pre-seam default. This matters: a spawn pipe
// and /dev/null are NOT observationally identical (node's pipe is an AF_UNIX
// socket, so a command that probes stdin's type — `test -c /dev/stdin`, `stat
// /proc/self/fd/0` — sees a char device vs a socket), so the no-stdin path
// (every model-driven call) must keep /dev/null rather than regress to a socket.
// Two LITERAL `stdio` tuples (not one variable tuple): only a literal lets the
// typed `spawn` overload infer non-null stdout/stderr, which the
// `ChildProcessByStdio` annotation captures (stdin `Writable | null`; stdout/
// stderr the non-null `Readable` the collectors attach to without a cast).
// Keep absent stdin as /dev/null; literal tuples preserve non-null output types.
const env = childEnv(spec.env, spec.dshEnv)
const child: ChildProcessByStdio<Writable | null, Readable, Readable> = spec.stdin !== undefined
? spawn('bash', ['-c', spec.command], { cwd: spec.cwd, env, stdio: ['pipe', 'pipe', 'pipe'], detached: true })
@@ -371,8 +323,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
let graceTimer: NodeJS.Timeout | undefined
// pid is undefined when the spawn itself fails (bad cwd, missing binary);
// the 'error' handler rejects `done` and kills become no-ops via pid -1.
// Failed spawns use pid -1 so kill remains a no-op.
const pid = child.pid ?? -1
const kill = (): void => {
@@ -381,27 +332,11 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
graceTimer = setTimeout(() => { killGroup(pid, 'SIGKILL') }, spec.graceMs)
}
// runBash owns no timer: the executor's `run()` fuses timeout+cancel into one
// deadline signal (`@deepseek-ai/dsh-timeout`) and passes it here; we only
// listen and run the SIGTERM→grace→SIGKILL kill. Whether the abort was a
// timeout or an upstream cancel is classified by the executor from that
// signal, not tracked here.
// The executor owns timeout classification; this layer only reacts to abort.
const onAbort = (): void => { kill() }
spec.signal?.addEventListener('abort', onAbort, { once: true })
// Write stdin and close it, but ONLY when the caller supplied bytes — with no
// stdin, fd 0 is `ignore` (/dev/null) and `child.stdin` is null. The error
// handler must exist whenever we write: an unhandled 'error' on the stream
// would throw and crash the host. We swallow the error rather than reject
// `done`, and that is correct for ANY stdin-write error, not just the common
// one — the stdin write is BEST-EFFORT, while the command's authoritative
// outcome is its exit code + captured output, which the `close` handler reports
// regardless of whether the write landed. The expected case is EPIPE (the child
// exited without reading, so closing our end of a still-full pipe fails); a rare
// non-EPIPE pipe fault means the command ran with incomplete stdin, and it
// surfaces that itself through its own exit/output (e.g. a hook that gets
// truncated JSON errors out) — rejecting here would instead discard that real
// output and turn it into an opaque infrastructure error, which is worse.
// Stdin writes are best-effort; process exit and captured output remain authoritative.
if (child.stdin !== null) {
child.stdin.on('error', () => { /* stdin write is best-effort; outcome rides on exit/output. */ })
child.stdin.end(spec.stdin)
@@ -409,8 +344,7 @@ export function runBash(spec: SpawnSpec, internals: RunInternals = {}): RunningB
const done = new Promise<SpawnOutcome>((resolve, reject) => {
child.on('error', (error) => {
// Spawn-level failure (ENOENT cwd, EACCES, …): no close event with
// meaningful output follows; clean up and reject.
// No meaningful close outcome follows a spawn failure.
cleanup()
reject(error)
})

View File

@@ -345,7 +345,7 @@ describe('LocalBashExecutor background tasks', () => {
})
})
describe('review fixes: lifecycle hardening', () => {
describe('executor cancellation, callback, and disposal contracts', () => {
it('start honors a pre-aborted or later-aborted AbortSignal', async () => {
const { bash } = await setup()
const controller = new AbortController()

View File

@@ -190,12 +190,8 @@ describe('stdin and extra env (set by in-process plugins)', () => {
})
it('gives fd 0 the exact pre-seam type: /dev/null when no stdin, a pipe when supplied', async () => {
// The no-stdin path must stay observationally identical to the pre-seam
// `ignore` default: a command that probes stdin's file type sees a char
// device (/dev/null). Regressing to an always-open pipe would make fd 0 a
// socket (node's spawn pipe is an AF_UNIX socket, not a FIFO), flipping
// `test -c /dev/stdin` for every model-driven call. When bytes ARE supplied,
// fd 0 is that pipe (a socket), as it must be to carry them.
// With no bytes, fd 0 remains the pre-seam `ignore` default (/dev/null, a character device).
// Supplied bytes use Node's spawn pipe, which is an AF_UNIX socket rather than a FIFO.
const none = await runBash(spec('test -c /dev/stdin && echo char || echo other')).done
expect(none.stdout.text).toBe('char\n')
const piped = await runBash(spec('test -S /dev/stdin && echo socket || echo other', { stdin: 'x' })).done
@@ -220,9 +216,8 @@ describe('stdin and extra env (set by in-process plugins)', () => {
})
it('does not crash or reject when the child ignores a large stdin (EPIPE)', async () => {
// The child exits immediately without reading; closing our end of a stdin
// pipe still holding ~1MiB triggers EPIPE on the write. The handler must
// swallow it: `done` resolves normally with the child's real exit.
// The child exits without reading, so closing a stdin pipe holding ~1 MiB triggers EPIPE.
// The handler swallows that write error and `done` reports the child's real exit.
const big = 'x'.repeat(1024 * 1024)
const result = await runBash(spec('exit 7', { stdin: big })).done
expect(result.exitCode).toBe(7)
@@ -361,7 +356,7 @@ describe('abort edge cases', () => {
})
})
describe('review fixes: env scrubbing and spill hardening', () => {
describe('environment and spill-file hardening', () => {
it('scrubs credential-shaped and ambient DSH env vars from child processes', async () => {
process.env.DSH_TEST_API_KEY = 'super-secret'
process.env.DSH_TEST_TOKEN = 'also-secret'