refactor(examples): fold the sandbox composition into acp-agent as its variant
sandbox-acp-agent's whole surface moves into examples/acp-agent following the example's existing multi-variant shape (the code-mode/both-mode overlays): sandbox.cordis.yml + its replay overlay, the four snapshot scenarios (config-options / mode-switching / escalation-approved / escalation-rejected) as a 'sandbox' header class over per-scenario configPath — goldens byte-identical, zero re-recording — and escalation.e2e.ts unchanged apart from the config path. One ACP example remains; demo:sandbox-acp keeps its name and boots the variant. References (both RFCs, group/package READMEs, the examples table and test map, the e2e workflow comment) now point at the variant.
This commit is contained in:
@@ -22,7 +22,6 @@ A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_P
|
||||
| `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) |
|
||||
| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit; `tests/code-mode-keyless-smoke.e2e.ts` — the same boot guard for the Code Mode overlay | `tests/{full-loop,coding-task,resume,compaction,todo-write}.e2e.ts` — real model + real bash + real todo_write, world-verified; `tests/code-mode.e2e.ts` — a real model composes two bash calls in one `run_code` program; collapsed header, dispatch events, written file all verified |
|
||||
| `cordis-agent` | `tests/keyless-smoke.e2e.ts` — boots the real tree incl. `@deepseek-ai/dsh-tool-cordis` by package name; the tool logic is unit-tested in `packages/cordis/tool-cordis` | `tests/cordis-tools.e2e.ts` — real model mounts a listener (tagged line fires), builds+calls its own tool, composes two mounts via provide/inject |
|
||||
| `sandbox-acp-agent` | `escalation.e2e.ts` — boots the real tree (sandbox + approval + bridge) keyless: initialize + `session/new` | same file — denied → escalates → a scripted client grants (the write must land) or rejects (it must not); skips without key/runner |
|
||||
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. the hook matrix: a scenario per hook point × outcome for BOTH the Claude and Codex bridges — block, deny, ask, context-fold, force-continue); `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written |
|
||||
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. the hook matrix: a scenario per hook point × outcome for BOTH the Claude and Codex bridges — block, deny, ask, context-fold, force-continue); `tests/acp.e2e.ts` also asserts stdout purity without a key; `tests/escalation.e2e.ts` boots the sandbox variant's real tree (sandbox + approval + bridge) keyless: initialize + `session/new` | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written; `tests/escalation.e2e.ts` — denied → escalates → a scripted client grants (the write must land) or rejects (it must not); skips without key/runner |
|
||||
|
||||
See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design.
|
||||
|
||||
@@ -33,8 +33,4 @@ An agent demo exposed as an **Agent Client Protocol (ACP)** server over JSON-RPC
|
||||
|
||||
Run with: `pnpm run demo:acp` (needs `DEEPSEEK_API_KEY`); `pnpm run demo:code-mode acp` boots the same server in Code Mode via the `code-mode.cordis.yml` overlay. See [acp-agent/README.md](acp-agent/README.md) for the Zed setup and the snapshot-test design.
|
||||
|
||||
## sandbox-acp-agent
|
||||
|
||||
The coding agent with its bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../packages/sandbox/sandbox-local) + [`@deepseek-ai/dsh-bash-sandbox`](../packages/bash/bash-sandbox) — the one-entry executor swap the `ctx.bash` capability seam exists for), served over ACP with [`@deepseek-ai/dsh-user-approval`](../packages/ui/user-approval) mounted — the first composition where the approval loop is LIVE: a sandbox denial escalated by the model becomes a `session/request_permission` prompt in the editor, and "Allow once" runs exactly that command under the wider mode.
|
||||
|
||||
Run with: `pnpm run demo:sandbox-acp` (needs `DEEPSEEK_API_KEY`; bwrap, a Landlock-enforcing kernel, or macOS for confined runs). See [sandbox-acp-agent/README.md](sandbox-acp-agent/README.md).
|
||||
The **sandbox variant** (`sandbox.cordis.yml`) swaps the bash executor for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../packages/sandbox/sandbox-local) + [`@deepseek-ai/dsh-bash-sandbox`](../packages/bash/bash-sandbox) — the one-entry executor swap the `ctx.bash` capability seam exists for) and mounts [`@deepseek-ai/dsh-user-approval`](../packages/ui/user-approval) — the composition where the approval loop is LIVE: a sandbox denial escalated by the model becomes a `session/request_permission` prompt in the editor, and "Allow once" runs exactly that command under the wider mode. Run with: `pnpm run demo:sandbox-acp` (needs `DEEPSEEK_API_KEY`; bwrap, a Landlock-enforcing kernel, or macOS for confined runs).
|
||||
|
||||
@@ -35,6 +35,17 @@ The editor sets each session's `cwd` to the project it opens; both the agent's b
|
||||
|
||||
This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`<scenario>/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`"; use `pnpm run test:snapshot:record` when the model transcript itself should change, and `pnpm run test:snapshot:refresh` when the committed model transcript is still the right mock input and only the current replay output/goldens need to be rewritten. The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `<scenario>/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `<scenario>/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [the ACP snapshot tests RFC](../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md) for the full design.
|
||||
|
||||
## The sandbox variant (`sandbox.cordis.yml`)
|
||||
|
||||
The same server with the bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../../packages/sandbox/sandbox-local/) + [`@deepseek-ai/dsh-bash-sandbox`](../../packages/bash/bash-sandbox/) — the one-entry executor swap the `ctx.bash` capability seam exists for) plus [`@deepseek-ai/dsh-user-approval`](../../packages/ui/user-approval/) — the composition where the approval loop is LIVE end to end: bash runs under `read-only`, a denial comes back as the structured marker, the model retries once with `sandbox_permissions` + `justification`, the ACP bridge's answerer turns that ask into a `session/request_permission` prompt in your editor, and "Allow once" runs exactly that command under the wider mode ([sandbox RFC § Escalation](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)). Run it with `pnpm run demo:sandbox-acp`; Zed setup is the same as above with this command.
|
||||
|
||||
- **Every approval is one-shot** (`Allow once` / `Reject` — no `allow_always`: the harness has no grant storage yet), and a dismissed prompt or a rejected ask fails closed with its own error text; so does every ask when no editor is attached to answer.
|
||||
- **Two session config options are live** ([sandbox RFC § Per-session mode switching](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)): a capable client shows `Sandbox` (`read-only`/`workspace-write`/`danger-full-access`) and `Approvals` (`ask`/`never`) selectors per session — a switch is one log-only event on that session's log and execution follows it; the sandbox mode is deliberately NOT stated in the prompt or narrated (the model learns the boundary from the denial marker — behavior, not belief), while an approval switch to `never` is stated and narrated; a resumed session reports its overrides back on `session/load`.
|
||||
- **The write boundary is config-fixed**: an escalated `workspace-write` run may write under the launch directory (`workspaceRoot: process.cwd()`) plus the platform temp area — a per-session root is config-phase future work in the [sandbox RFC](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md).
|
||||
- **No usable runner fails closed per command** (structured `SANDBOX_UNAVAILABLE`), and the variant loads no filesystem tools: they would bypass the bash sandbox.
|
||||
|
||||
Variant tests, in this example's suites: `tests/escalation.e2e.ts` — keyless, it boots the real `sandbox.cordis.yml` through the Loader as an ACP subprocess, proves the whole tree (sandbox executor + approval service + bridge) initializes and opens a session, and drives the config options end to end; with a key and a usable runner, a scripted ACP client plays the human — the real model gets denied, escalates, the client answers `allow-once`, and the retried write must land on disk. Four scenarios in `tests/acp.snapshot.ts` run against the variant's `sandbox.cordis.snapshot.yml` replay overlay under the `sandbox` header class: the keyless `config-options` exchange, the recorded `mode-switching` arc (that class's pinned header — the approval prompt-section delta, its "changed by the user" notice, and a confined write landing under the switched mode), and both recorded escalation branches (`session/request_permission` answered allow-once / reject-once). Replay re-executes every recorded bash call under the host's real runner — Seatbelt works out of the box on macOS; on Linux install bubblewrap first, exactly what ci.yml's snapshot lane does. No fixture carries a real denial: denial stderr is backend dialect and would pin a fixture to its recording platform (the rationale comment atop the suite file).
|
||||
|
||||
## MVP limitations
|
||||
|
||||
The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, and `additionalDirectories` and `mcpServers` are rejected. Permission prompts (`session/request_permission`) are wired through the approval seam, but this example composes no ask-producing policy, so tools run with the executor's full authority. See `packages/ui/acp/README.md` for the full contract.
|
||||
The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, and `additionalDirectories` and `mcpServers` are rejected. Permission prompts (`session/request_permission`) are wired through the approval seam; the MAIN tree composes no ask-producing policy, so its tools run with the executor's full authority — the sandbox variant above is the composition that exercises the live prompt. See `packages/ui/acp/README.md` for the full contract.
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
path: ./sandbox.cordis.yml
|
||||
patches:
|
||||
# The name is an assertion, not an override: the include skips the patch
|
||||
# (warning) when the id points at a different plugin, so this can never
|
||||
@@ -1,13 +1,13 @@
|
||||
# The sandbox-acp-agent plugin tree: the sandboxed coding agent served over the
|
||||
# Agent Client Protocol, with the approval seam composed — the first LIVE
|
||||
# approval composition. A sandbox denial escalated by the model
|
||||
# The SANDBOX VARIANT of the acp-agent example: the sandboxed coding agent
|
||||
# served over the Agent Client Protocol, with the approval seam composed —
|
||||
# the live approval composition (`pnpm run demo:sandbox-acp`). A sandbox denial escalated by the model
|
||||
# (sandbox_permissions + justification) reaches the EDITOR as a
|
||||
# session/request_permission prompt through the ACP bridge's answerer, and an
|
||||
# "Allow once" runs exactly that command under the wider mode.
|
||||
#
|
||||
# CRITICAL: this tree loads NO stdout logger and NO hmr — stdout is reserved
|
||||
# for the ACP JSON-RPC protocol (a property of @deepseek-ai/dsh-acp-agent,
|
||||
# same as examples/acp-agent).
|
||||
# same as the sibling cordis.yml).
|
||||
#
|
||||
# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) — the
|
||||
# dsh-acp-agent bin loads the gitignored repo-root .env first (on STDERR only).
|
||||
@@ -27,6 +27,10 @@ const AGENT = {
|
||||
const CODE_MODE_CONFIG = fileURLToPath(new URL('../code-mode.cordis.yml', import.meta.url))
|
||||
const BOTH_MODE_CONFIG = fileURLToPath(new URL('../both-mode.cordis.yml', import.meta.url))
|
||||
|
||||
// The sandbox variant (its own composition, not an include patch: sandboxed
|
||||
// bash executor + the approval seam over the same app spine).
|
||||
const SANDBOX_CONFIG = fileURLToPath(new URL('../sandbox.cordis.yml', import.meta.url))
|
||||
|
||||
function snapshotModeFromEnv(value: string | undefined): SnapshotSuiteOptions['mode'] {
|
||||
switch (value) {
|
||||
case undefined:
|
||||
@@ -121,6 +125,37 @@ const SCENARIOS: Scenario[] = [
|
||||
// therefore pins its own class.
|
||||
{ name: 'code-mode-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'code', configPath: CODE_MODE_CONFIG },
|
||||
{ name: 'both-mode-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'both', configPath: BOTH_MODE_CONFIG },
|
||||
// The SANDBOX variant (sandbox.cordis.yml: sandboxed bash + approval seam).
|
||||
// Replay swaps only the MODEL for the recorded transcript — every bash call
|
||||
// re-executes for real under the host's actual runner (Seatbelt on macOS,
|
||||
// bwrap on Linux CI), so these recordings double as cross-backend
|
||||
// confinement regression; their commands are limited to `cat`/`printf`
|
||||
// shapes whose bytes are identical across those backends and across
|
||||
// GNU/BSD userlands. Deliberately ABSENT: a scenario whose transcript
|
||||
// carries a real sandbox DENIAL — the denied command's own stderr is the
|
||||
// backend's dialect (EROFS/EACCES/EPERM phrasing), so such a fixture
|
||||
// replays only on the platform that recorded it; the denial→marker path
|
||||
// stays on dsh-tool-bash's unit tests and the real-kernel sandbox e2e legs,
|
||||
// and the escalation scenarios sidestep it by having the USER assert the
|
||||
// prior denial. config-options: the session config-option surface this
|
||||
// composition adds (both advertised selects, the refreshed state every
|
||||
// set_config_option answers with, both rejection shapes) — protocol-only,
|
||||
// replays on runner-less hosts. mode-switching: the runtime switching arc
|
||||
// and NECESSARILY this class's pinned-header scenario (an approval-policy
|
||||
// switch rewrites its prompt section; the resulting request/header-delta is
|
||||
// legal only in the pinning scenario) — the pin commits the full sandbox
|
||||
// header (persona, tool schemas WITH the escalation fields) plus the
|
||||
// approval delta and its "changed by the user" notice; the SANDBOX switch
|
||||
// stays deliberately silent (the visibility asymmetry), proven by BEHAVIOR.
|
||||
// escalation-approved/rejected: the approval wire end-to-end under the
|
||||
// default read-only/ask — the escalating call streams,
|
||||
// session/request_permission attaches to it, and the scripted answer drives
|
||||
// each branch (approved runs CONFINED under the granted workspace-write;
|
||||
// rejected executes nothing, failing with the deterministic text).
|
||||
{ name: 'config-options', hasModelTurn: false, recorded: false, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'mode-switching', hasModelTurn: true, recorded: true, pinsHeader: true, expectedHeaderDeltas: 1, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'escalation-approved', hasModelTurn: true, recorded: true, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'escalation-rejected', hasModelTurn: true, recorded: true, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
]
|
||||
|
||||
defineAcpSnapshotSuite({
|
||||
|
||||
@@ -17,9 +17,9 @@ import {
|
||||
} from '@agentclientprotocol/sdk'
|
||||
|
||||
/**
|
||||
* examples/sandbox-acp-agent end to end.
|
||||
* The sandbox variant (`sandbox.cordis.yml`) end to end.
|
||||
*
|
||||
* Keyless smoke: boot the REAL `cordis.yml` through the `dsh-acp-agent` bin as
|
||||
* Keyless smoke: boot the REAL `sandbox.cordis.yml` through the `dsh-acp-agent` bin as
|
||||
* an ACP subprocess and drive initialize + session/new — the real-Loader-path
|
||||
* guard (postmortem 0001) for THIS tree's export shapes, which now include the
|
||||
* sandbox executor AND the approval service. No prompt is sent, so neither the
|
||||
@@ -36,7 +36,7 @@ import {
|
||||
*/
|
||||
|
||||
const binScript = fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../sandbox.cordis.yml', import.meta.url))
|
||||
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
|
||||
// The subprocess runs from a temp cwd OUTSIDE the repo; point tsx at the repo
|
||||
// tsconfig so the unbuilt `paths` map resolves (see examples/AGENTS.md).
|
||||
@@ -116,7 +116,7 @@ afterEach(async () => {
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe('sandbox-acp-agent keyless smoke (real cordis.yml via the Loader)', () => {
|
||||
describe('sandbox variant keyless smoke (real sandbox.cordis.yml via the Loader)', () => {
|
||||
it('boots the tree — sandbox executor + approval service + bridge — and opens a session', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'sandbox-acp-smoke-'))
|
||||
spawned = spawnSandboxAcpAgent(workdir, 'reject-once')
|
||||
@@ -161,7 +161,7 @@ describe('sandbox-acp-agent keyless smoke (real cordis.yml via the Loader)', ()
|
||||
}, 30_000)
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY || !hasRunner)('sandbox-acp-agent e2e: the live approval loop', () => {
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY || !hasRunner)('sandbox variant e2e: the live approval loop', () => {
|
||||
it('denial → model escalation → editor prompt → allow-once → the retried write lands on disk', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'sandbox-acp-e2e-'))
|
||||
spawned = spawnSandboxAcpAgent(workdir, 'allow-once')
|
||||
@@ -1,16 +0,0 @@
|
||||
# sandbox-acp-agent
|
||||
|
||||
The coding agent with its bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../../packages/sandbox/sandbox-local/) + [`@deepseek-ai/dsh-bash-sandbox`](../../packages/bash/bash-sandbox/) — the one-entry executor swap the `ctx.bash` capability seam exists for), served over the **Agent Client Protocol**, plus [`@deepseek-ai/dsh-user-approval`](../../packages/ui/user-approval/) — which makes this the first composition where the approval loop is LIVE end to end: bash runs under `read-only`, a denial comes back as the structured marker, the model retries once with `sandbox_permissions` + `justification`, the ACP bridge's answerer turns that ask into a `session/request_permission` prompt in your editor, and "Allow once" runs exactly that command under the wider mode ([sandbox RFC § Escalation](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)).
|
||||
|
||||
```sh
|
||||
pnpm run demo:sandbox-acp # needs DEEPSEEK_API_KEY; drive it from Zed or any ACP client
|
||||
```
|
||||
|
||||
Zed setup is the same as [acp-agent](../acp-agent/README.md) with this example's command; only the leaf `cordis.yml` differs (the sandbox stack + the approval entry in place of the local bash executor and the extra tool stacks).
|
||||
|
||||
- **Every approval is one-shot** (`Allow once` / `Reject` — no `allow_always`: the harness has no grant storage yet), and a dismissed prompt or a rejected ask fails closed with its own error text; so does every ask when no editor is attached to answer.
|
||||
- **Two session config options are live** ([sandbox RFC § Per-session mode switching](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)): a capable client shows `Sandbox` (`read-only`/`workspace-write`/`danger-full-access`) and `Approvals` (`ask`/`never`) selectors per session — a switch is one log-only event on that session's log and execution follows it; the sandbox mode is deliberately NOT stated in the prompt or narrated (the model learns the boundary from the denial marker — behavior, not belief), while an approval switch to `never` is stated and narrated; a resumed session reports its overrides back on `session/load`.
|
||||
- **The write boundary is config-fixed**: an escalated `workspace-write` run may write under the launch directory (`workspaceRoot: process.cwd()`) plus the platform temp area — a per-session root is config-phase future work in the [sandbox RFC](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md).
|
||||
- **No usable runner fails closed per command** (structured `SANDBOX_UNAVAILABLE`), and the filesystem tools stay unloaded for the same reason as `sandbox-agent`: they would bypass the bash sandbox.
|
||||
|
||||
Tests: `tests/escalation.e2e.ts` — keyless, it boots the real `cordis.yml` through the Loader as an ACP subprocess, proves the whole tree (sandbox executor + approval service + bridge) initializes and opens a session, and drives the config options end to end (both advertised with composition currents, switches honored and echoed as complete state, out-of-vocabulary values rejected); with a key and a usable runner, a scripted ACP client plays the human — the real model gets denied, escalates, the client answers `allow-once`, and the retried write must land on disk. `tests/acp.snapshot.ts` (the [shared snapshot kit](../../packages/support/acp-snapshot/) over this composition's `cordis.snapshot.yml` replay overlay) pins four scenarios as committed wire bytes: the keyless config-option exchange, the recorded `mode-switching` arc (the suite's pinned header — both switches, their prompt-section deltas, one "changed by the user" notice per knob, and a confined write landing under the switched mode), and both recorded escalation branches (`session/request_permission` answered allow-once / reject-once). Replay re-executes every recorded bash call under the host's real runner — Seatbelt works out of the box on macOS; on Linux install bubblewrap (or build the Landlock launcher) first, exactly what ci.yml's snapshot lane does. No fixture carries a real denial: denial stderr is backend dialect and would pin a fixture to its recording platform (the rationale comment atop the suite file).
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"name": "sandbox-acp-agent-example",
|
||||
"description": "Runnable demo: the sandboxed coding agent as an ACP server, with sandbox-escalation approval prompts answered by the editor",
|
||||
"private": true,
|
||||
"version": "0.0.1",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -1,82 +0,0 @@
|
||||
import { dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { defineAcpSnapshotSuite, type Scenario, type SnapshotSuiteOptions } from '@deepseek-ai/dsh-acp-snapshot'
|
||||
|
||||
/**
|
||||
* Snapshot suite for the SANDBOXED composition (`../cordis.yml`, swapped to
|
||||
* the sibling `cordis.snapshot.yml` replay overlay by the bin under
|
||||
* `DSH_SNAPSHOT=replay`). Replay swaps only the MODEL for the recorded
|
||||
* transcript — every bash call re-executes for real under the host's actual
|
||||
* runner (Seatbelt on macOS, bwrap on Linux CI: ci.yml's snapshot lane
|
||||
* installs bubblewrap for exactly this), so the recorded scenarios double as
|
||||
* cross-backend confinement regression: an allowed command a runner change
|
||||
* starts denying fails replay outright. Their commands are limited to
|
||||
* `cat`/`printf` shapes whose bytes are identical across those backends and
|
||||
* across GNU/BSD userlands.
|
||||
*
|
||||
* Deliberately ABSENT: a scenario whose transcript carries a real sandbox
|
||||
* DENIAL. The harness-authored `[sandbox: file access denied …]` marker is
|
||||
* byte-stable, but the denied command's own stderr is the backend's dialect
|
||||
* (bwrap EROFS "Read-only file system", Landlock EACCES "Permission
|
||||
* denied", Seatbelt EPERM "Operation not permitted", GNU vs BSD phrasing on
|
||||
* top), and stderr reaches both compared surfaces — such a fixture replays
|
||||
* only on the platform that recorded it. The denial→marker path stays on
|
||||
* dsh-tool-bash's unit tests and the real-kernel sandbox e2e legs
|
||||
* (.github/workflows/sandbox.yml); the escalation scenarios below sidestep
|
||||
* it by having the USER assert the prior denial, so the recorded model
|
||||
* escalates without a platform-variant denial in the log.
|
||||
*/
|
||||
const SCENARIOS: Scenario[] = [
|
||||
// Protocol-only (keyless, authored): the session config-option surface
|
||||
// this composition adds — both advertised selects on session/new, the
|
||||
// complete refreshed state every session/set_config_option answers with,
|
||||
// and both rejection shapes — as committed wire bytes. No bash runs, so
|
||||
// this one still replays on runner-less hosts.
|
||||
{ name: 'config-options', hasModelTurn: false, recorded: false },
|
||||
// The runtime mode-switching arc, and NECESSARILY the pinned-header
|
||||
// scenario: an approval-policy switch rewrites its prompt section, and the
|
||||
// resulting request/header-delta is legal only in the pinning scenario
|
||||
// (the factory's uniformity guard). The pin commits this composition's
|
||||
// full header — persona, tool schemas WITH the escalation fields — plus
|
||||
// the approval delta and its "changed by the user" notice verbatim. The
|
||||
// SANDBOX switch is deliberately silent (no section, no notice — the
|
||||
// sandbox RFC's visibility asymmetry): the recorded arc proves it by
|
||||
// BEHAVIOR, a confined write landing under the switched mode with no
|
||||
// header change.
|
||||
{ name: 'mode-switching', hasModelTurn: true, recorded: true, pinsHeader: true, expectedHeaderDeltas: 1 },
|
||||
// The approval wire end-to-end, under the DEFAULT read-only/ask (a switch
|
||||
// would emit a header-delta the uniformity guard forbids here): the
|
||||
// escalating bash call streams, session/request_permission attaches to it
|
||||
// (allow-once / reject-once), and the scripted answer drives each branch —
|
||||
// an approved run executes CONFINED under the granted workspace-write; a
|
||||
// rejected one executes nothing and fails with the deterministic
|
||||
// rejection text.
|
||||
{ name: 'escalation-approved', hasModelTurn: true, recorded: true },
|
||||
{ name: 'escalation-rejected', hasModelTurn: true, recorded: true },
|
||||
]
|
||||
|
||||
function snapshotModeFromEnv(value: string | undefined): SnapshotSuiteOptions['mode'] {
|
||||
switch (value) {
|
||||
case undefined:
|
||||
case '':
|
||||
case 'replay':
|
||||
return 'replay'
|
||||
case 'record':
|
||||
return 'record'
|
||||
case 'refresh':
|
||||
return 'refresh'
|
||||
default:
|
||||
throw new Error(`unknown DSH_SNAPSHOT mode: ${value}`)
|
||||
}
|
||||
}
|
||||
|
||||
defineAcpSnapshotSuite({
|
||||
agent: {
|
||||
binScript: fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)),
|
||||
configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)),
|
||||
tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)),
|
||||
},
|
||||
snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'),
|
||||
scenarios: SCENARIOS,
|
||||
mode: snapshotModeFromEnv(process.env.DSH_SNAPSHOT),
|
||||
})
|
||||
Reference in New Issue
Block a user