Merge remote-tracking branch 'origin/master' into codex/pr48-repo-hardening-rfcs
# Conflicts: # docs/adr/README.md # docs/rfc/009-session-persistence-and-resumability.md # docs/rfc/README.md # docs/rfc/implemented/2026-06-11-doc-sync-enforcement.md # docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md # examples/acp-agent/tests/acp.e2e.ts # packages/acp/README.md # packages/acp/src/index.ts # packages/acp/tests/stream-update.spec.ts # packages/agent-loop/src/loop.ts # packages/tools/src/index.ts
This commit is contained in:
Tianyi Cui
19
docs/rfc/proposed/2026-06-11-api-extractor-reports.md
Normal file
19
docs/rfc/proposed/2026-06-11-api-extractor-reports.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# RFC: API extractor reports
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
> Split out from the original "Doc-sync and API reports" RFC (2026-06-11). Parts 1-2 (doc-block typechecking, event-taxonomy verification) shipped — see [doc-sync enforcement](../implemented/2026-06-11-doc-sync-enforcement.md). This is the deferred part 3, kept as a standalone proposal.
|
||||
|
||||
## Problem
|
||||
|
||||
Public API changes are invisible — nothing makes "this commit changed the public surface" an explicit, reviewable fact. A reviewer reading a diff can miss that an exported type gained a field or a method signature shifted.
|
||||
|
||||
## Proposal
|
||||
|
||||
api-extractor (or `tsc --emitDeclarationOnly` + a normalized public-surface dump) producing a checked-in `etc/<pkg>.api.md` per package; CI fails if regeneration differs. Every public-API change becomes a diff line a reviewer (or review agent) must see.
|
||||
|
||||
## Status / why deferred
|
||||
|
||||
Deferred when doc-sync landed: low value for an internal monorepo where reviewers already see the source diff, and a heavy, finicky dependency. Revisit if the packages are ever published externally — at that point a stable, diffable public surface earns its keep.
|
||||
29
docs/rfc/proposed/2026-06-11-architectural-conformance.md
Normal file
29
docs/rfc/proposed/2026-06-11-architectural-conformance.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# RFC: Architectural conformance — dependency rules and the adapter kit
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
Two architectural guarantees currently live only in prose: (1) nothing depends on the concrete loop package ([the microkernel promise](../implemented/2026-06-11-microkernel-event-taxonomy.md)), and (2) every LlmAdapter speaks the chunk protocol correctly. Both should be mechanical ([the quality-gates principle](../implemented/2026-06-11-quality-gates.md)).
|
||||
|
||||
## Proposal
|
||||
|
||||
**dependency-cruiser** with rules:
|
||||
|
||||
- `packages/*` (except agent-loop's own tests and examples/) must not import `@deepseek-ai/dsh-agent-loop`.
|
||||
- No cross-package deep imports (`@deepseek-ai/dsh-*/src/...` paths) — public entry points only.
|
||||
- No import cycles anywhere in packages/.
|
||||
- `vendor/*` must not import from `packages/*`.
|
||||
- Layering: dsh-llm imports nothing from other dsh packages; dsh-session only dsh-llm; etc. (the dependency table in packages/README.md, enforced).
|
||||
|
||||
**Adapter conformance kit** in dsh-llm (`@deepseek-ai/dsh-llm/conformance`): a reusable vitest suite parameterized by an adapter factory, asserting the chunk-protocol contract — index monotonicity per block, no deltas after `block-end` for an index, exactly one `finish`, usage at most once, every `tool-call-delta` carries the call id, abort honored promptly. Run it against the mocks now; the DeepSeek V4 adapter inherits it on day one. Optionally a dev-mode `strictAdapter()` wrapper enforcing the same at runtime behind a debug flag (pairs with [the dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md)).
|
||||
|
||||
## Plan
|
||||
|
||||
dependency-cruiser config + CI step first (an hour of work, permanent guarantee); the conformance kit lands with its first consumer test against MockAdapter, and is a prerequisite for the V4 adapter phase.
|
||||
|
||||
## Risks
|
||||
|
||||
Dep-cruiser rule maintenance as packages are added — keep rules pattern-based (`dsh-*`) rather than enumerated.
|
||||
@@ -0,0 +1,25 @@
|
||||
# RFC: Deterministic tests, the replay invariant fixture, and race stress
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
Several loop tests synchronize with `setTimeout(30)` sleeps — flakiness debt that wastes agent cycles on retries and can mask ordering bugs. Separately, our core architectural promise (any session log replays to identical derived history) is asserted in two tests but is cheap to assert *everywhere*. And the inbox wakeup race was verified by hand exactly once; nothing re-verifies it continuously.
|
||||
|
||||
## Proposal
|
||||
|
||||
Three measures:
|
||||
|
||||
1. **No wall-clock sleeps in tests.** Replace `setTimeout(N)` waits with event-driven waits (the existing `waitForIdle` pattern, extended to `waitForStatus`, `waitForEvent(n)`) or vitest fake timers where time itself is under test. Enforce with a lint rule banning `setTimeout` in `packages/*/tests` outside an allowlisted helper module.
|
||||
2. **Universal replay fixture.** A shared test helper wraps the loop harness so that after every test, the agent's session log is replayed into a fresh Session and `deriveMessages()` equality is asserted automatically. The invariant then gets checked hundreds of times per CI run across every scenario the suite produces, not twice.
|
||||
3. **Nightly race stress.** A CI job running the agent-loop and inbox suites with `vitest --repeat=200` (and `--shuffle`) to flush scheduling-dependent failures; any flake found is a bug to fix, never a retry.
|
||||
|
||||
## Plan
|
||||
|
||||
Land 1 and 2 together (they touch the same helpers); add the nightly job after the suite is sleep-free so repeats are fast.
|
||||
|
||||
## Risks
|
||||
|
||||
Fake timers interact subtly with Promise scheduling in the loop — prefer event-driven waits; reserve fake timers for timer-service behavior itself.
|
||||
28
docs/rfc/proposed/2026-06-11-mutation-testing.md
Normal file
28
docs/rfc/proposed/2026-06-11-mutation-testing.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# RFC: Mutation testing as the coverage counterweight
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
The per-file 100% coverage gate ([the quality-gates decision](../implemented/2026-06-11-quality-gates.md)) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs.
|
||||
|
||||
## Proposal
|
||||
|
||||
Stryker (`@stryker-mutator/vitest-runner`) over `packages/*/src`:
|
||||
|
||||
- **PR-scoped incremental runs** (changed files only) as a CI job — fast enough to gate merges once tuned.
|
||||
- **Nightly full runs** with a tracked mutation score; start by recording, then set the threshold at the observed baseline and ratchet upward (same policy as coverage: thresholds only ever tighten).
|
||||
- Surviving mutants are work items: an agent picks a survivor, writes the killing test, repeats — a well-shaped autonomous loop.
|
||||
- Equivalent mutants (provably behavior-preserving) get annotated exclusions with reasons, mirroring the `/* v8 ignore */` policy.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Add Stryker config scoped to one package (llm — smallest, most algorithmic) and measure runtime.
|
||||
2. Expand to all packages; record baseline scores in the config.
|
||||
3. Wire the nightly job; add the incremental PR job once runtime is acceptable.
|
||||
|
||||
## Risks
|
||||
|
||||
Runtime: mutation testing is expensive; per-file 100% coverage helps (every mutant is at least reached). If PR-scoped runs stay too slow, keep them nightly-only and rely on the score ratchet.
|
||||
@@ -0,0 +1,24 @@
|
||||
# RFC: Supply chain checks and vendor drift verification
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
The vendor manifest ([the vendoring decision](../implemented/2026-06-11-vendor-cordis-as-source.md)) is enforced at commit time in the *forward* direction (vendored change ⇒ manifest update) but nothing verifies the manifest's *claims*: that vendor/ actually equals upstream-at-SHA plus exactly the logged modifications. And the handful of true npm dependencies have no advisory monitoring or update cadence.
|
||||
|
||||
## Proposal
|
||||
|
||||
1. **Vendor drift check** (nightly CI): clone the upstream repos at the manifest SHAs (shallow), copy the corresponding package sources, and diff against `vendor/*/src`. The job fails unless the diff matches the logged local modifications (kept as a checked-in patch file per modification — the log entries become verifiable artifacts rather than prose).
|
||||
2. **Dependency advisories**: osv-scanner (or `pnpm audit`) job on the lockfile, scheduled + on lockfile-touching PRs.
|
||||
3. **License inventory**: a script asserting every vendored package carries its LICENSE and that package.json `license` fields match the inventory in vendor/README.md (we mix vendored MIT with our BSD-3) — CI step.
|
||||
4. **Renovate** (or a scheduled agent task) proposing npm dependency updates in small PRs that ride the full gate suite; vendored packages are excluded (their updates follow the manifest sync procedure, ideally as a semi-automated agent workflow: fetch upstream, re-apply patches, run gates, open PR with the manifest table updated).
|
||||
|
||||
## Plan
|
||||
|
||||
3 is trivial — do first. 1 requires network access from CI to the upstream repos (private — needs a token) and converting the two existing logged modifications into patch files. 2 and 4 are config.
|
||||
|
||||
## Risks
|
||||
|
||||
Upstream repos are private mirrors; CI credentials and availability are the main friction for the drift check. If blocked, run it as a local scheduled agent task instead of CI.
|
||||
74
docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md
Normal file
74
docs/rfc/proposed/2026-06-14-acp-agent-client-protocol.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# RFC: Agent Client Protocol (ACP) support — drive the coding agent from external editors
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
> **Implementation status (MVP landed):** steps 1, 2, 3, 4, 6, 7, 8 are implemented in `packages/acp` + `examples/acp-agent`. **Step 5 (the `session/request_permission` permission gate) is deferred** — the bridge ships a pass-through (tools run with the executor's full authority) marked `TODO(rfc010-permission-gate)`, and lays down only the `WeakMap<Agent, sessionId>` ownership seam the gate will build on. Status stays `proposed` until the gate lands. One further best-effort limitation is tracked as `TODO(rfc010-cancel-prestep)`: `session/cancel` aborts a running step and settles the RPC as `cancelled`, but a turn still queued (not yet started) when the cancel arrives may execute before the abort takes effect, pending a loop-level pre-step cancel. **Per-session `cwd` is now honored** (lifting the original "launch the server in the workspace root" restriction — see § Deferred): `session/new` accepts any absolute `cwd`, and `session/load` requires the request `cwd` to match the persisted session `cwd` so the editor and bash executor agree on the workspace.
|
||||
|
||||
## Problem
|
||||
|
||||
The coding agent is reachable only through the readline `stdio-chat` plugin: it reads lines from stdin, calls `agent.send()`, and prints `agent/stream-chunk` to stdout. There is no structured protocol, so the agent cannot be embedded in an editor — no streaming render, no tool-call display, no permission UI, no resumable sessions.
|
||||
|
||||
Editors are converging on the Agent Client Protocol (ACP), which Zed and others speak: JSON-RPC 2.0 over newline-delimited stdio, modeled on the Language Server Protocol. An editor boots the agent as a subprocess and exchanges `initialize` / `session/new` / `session/prompt`, rendering streamed `session/update` notifications and `session/request_permission` prompts. The goal is for the agent to be a drop-in ACP server — implement the protocol once and run in any ACP client, with no per-editor glue.
|
||||
|
||||
This RFC has a hard prerequisite on [session persistence](../implemented/2026-06-14-session-persistence.md): it assumes durable session persistence (the `SessionPersistence` service and the async `AgentLoop.resume` seam) is implemented, so resuming a session via `session/load` is in scope. None of those APIs exist yet — `AgentLoop` currently exposes only the synchronous `create` — so ACP must land after, or in the same change as, [session persistence](../implemented/2026-06-14-session-persistence.md), and pins to its `resume(agentId, resumeSessionId)` contract. Session persistence persists every `SessionEvent` verbatim (including `assistant/chunk`), so a loaded session has the stream chunks needed to replay turns to the client.
|
||||
|
||||
## Proposal
|
||||
|
||||
A new plugin package `@deepseek-ai/dsh-acp` — a client-driver / UI plugin, the structured analogue of `stdio-chat`. It is NOT a change to the loop and NOT an [capability seams](../implemented/2026-06-13-capability-seams.md) interface/implementation/consumer capability split; it consumes the existing `agent/*` event taxonomy and the `tools/execute` waterfall.
|
||||
|
||||
It depends on the official `@agentclientprotocol/sdk` (the `AgentSideConnection` class) — Apache-2.0, actively versioned. The SDK declares a `zod` peer dependency and imports `zod/v4` at runtime, so `packages/acp` must declare `zod` itself (per the workspace dependency constraints). This is the renamed successor to `@zed-industries/agent-client-protocol`, which is now deprecated on npm.
|
||||
|
||||
The mapping between ACP and existing harness seams — each row names the seam and any required extension:
|
||||
|
||||
| ACP (client ⇄ agent) | Harness seam | Notes |
|
||||
|---|---|---|
|
||||
| `initialize` | static handler | negotiate `protocolVersion` (echo the supported version, else error); advertise text-only `promptCapabilities` and `loadSession: true`; report agent name/version |
|
||||
| `session/new {cwd, mcpServers, additionalDirectories}` → `{sessionId}` | the `dsh-agent` create factory (see Dependency note + Plan) | the seam must accept `{ sessionId, meta }` so the ACP-generated `sessionId` becomes the live/persisted session id and the validated `cwd` is attached as the `SessionHeader` (today `AgentLoop.create(id)` hardcodes `${id}-session` and takes no metadata); reject a 2nd session (single-session MVP, see [ACP multi-session](2026-06-14-acp-multi-session.md)); `cwd` validated (require absolute) — any absolute cwd is honored: it becomes the session's `SessionHeader.cwd` and the default bash workdir (per-session cwd, see § Deferred → RESOLVED), so the server need not launch in the workspace; non-empty `mcpServers` and `additionalDirectories` are rejected for the MVP because silently ignoring requested servers/roots would desync the client's tool and filesystem-scope UI |
|
||||
| `session/load {sessionId, cwd, mcpServers, additionalDirectories}` | the `dsh-agent` resume factory ([session persistence](../implemented/2026-06-14-session-persistence.md) + Dependency note) | load `{ meta, events }`, seed the session, re-derive history via `deriveMessages()`, replay prior turns to the client as `session/update` per the ACP load contract; `mcpServers` and `additionalDirectories` rejected as in `session/new` |
|
||||
| `session/prompt {prompt}` | `agent.send()` (idle) | text blocks → `TextBlock`; reject image/audio per advertised capabilities; one in-flight prompt per session |
|
||||
| resolve `session/prompt` → `{stopReason}` | `agent/turn-end` (extended, see Plan) | map the harness kebab `TurnEndReason` to the ACP snake_case `StopReason` wire enum: `completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`(cancel)→`cancelled`, plus `refusal`/`max_turn_requests` when applicable; honor the batch-into-one-turn and send-not-synchronously-running settle semantics |
|
||||
| `session/update: agent_message_chunk` | `agent/stream-chunk` `text-delta` only | do NOT also emit on `block-end(TextBlock)` — it carries the fully-assembled block and would duplicate the streamed text |
|
||||
| `session/update: agent_thought_chunk` | `agent/stream-chunk` `reasoning-delta` | |
|
||||
| `session/update: tool_call` (pending→in_progress) | `session/event` `tool/call` | demux via a Session→sessionId map; `kind` inferred from the tool name |
|
||||
| `session/update: tool_call_update` (completed/failed) | `session/event` `tool/result` | a throwing `tools/execute` yields NO `tool/result` → fail the pending tool UI from `agent/error`/turn-end |
|
||||
| `session/request_permission {sessionId, toolCall, options}` | prepended `tools/execute` listener | no-op unless `exec.agent` is ACP-owned; await the outcome; `selected/allow_*` → `next()`; `reject_*`/`cancelled` → veto `ToolExecutionResult{isError}` |
|
||||
| `session/cancel` (notification) | `agent.abort(reason)` | settle the in-flight prompt as `cancelled`; resolve any pending permission as `cancelled` exactly once |
|
||||
|
||||
The permission gate is the first real consumer of the `tools/execute` veto seam (the documented "single veto/sandbox/permission seam" plus the deferred "Permission system" TODO in [docs/architecture.md](../../architecture.md)). It is a single global listener registered with `prepend: true` so it runs before any other tool wrapper. `ToolExecution.agent` is optional and the `Agent` interface carries no origin marker, so the bridge tracks ownership itself: it records each agent it creates in a `WeakMap<Agent, sessionId>` and the gate no-ops (calls `next()` immediately) for any `exec.agent` it does not own — non-ACP agents and the no-agent case pass straight through. For an owned agent it resolves the session, issues `session/request_permission`, and stores the pending resolver on that session's record so the outcome — or a `session/cancel`/connection-close — settles it exactly once.
|
||||
|
||||
Lifecycle and disposal: the connection, listeners, and in-flight permission promises register via `ctx.effect`/`ctx.on`; teardown is async and awaits quiescence — close the connection, settle/reject pending permissions, `agent.abort()`, and wait for the agent to settle. The disposal-settle signal must come from the `dsh-agent` interface, not the loop: `agent.done` exists only on the concrete `LoopAgent`, so the bridge instead observes `agent/status` reaching `idle`/`disposed` (or the RFC lifts a quiescence promise onto the `Agent` interface). Every listener contains its `send()` exceptions (log, never reject the turn) because stream chunks are emitted inside the model step, so a throwing listener would corrupt the turn.
|
||||
|
||||
**Dependency note (architecture rule).** [docs/architecture.md](../../architecture.md) states "plugins depend on interface packages, never on `dsh-agent-loop`." Creating and resuming agents is currently only on the concrete `AgentLoop` (`ctx.agentLoop`), so this RFC proposes adding an **abstract create/resume factory** to the `dsh-agent` interface (registry-level `create({ sessionId, meta })` / `resume(...)`), implemented by the loop, so `dsh-acp` injects only `agents` (the interface) and the dependency rule holds. The alternative — injecting the concrete `agentLoop` and recording a documented exception in the architecture doc — is explicitly the non-preferred fallback.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Package scaffold `packages/acp/` per [the cookbook](../../cookbook/adding-a-package.md); add `@agentclientprotocol/sdk` and `zod`. Add the abstract create/resume factory to `dsh-agent` (the interface) so the bridge can `inject: ['agents', 'sessions', 'tools', 'sessionPersistence']` without depending on the concrete loop; `sessionPersistence` is required because `session/load` advertises `loadSession: true`. (Fallback only if the factory is judged not worth it: inject `agentLoop` directly and record the architecture-rule exception in `docs/architecture.md`.)
|
||||
2. Connection plus `initialize`/`session/new`: wire `AgentSideConnection` to stdin/stdout; protocolVersion negotiation; the single-session guard; create the live session through the new `{ sessionId, meta }` factory seam (so the ACP `sessionId` and validated `cwd` become the session's id and header); the `sessionId↔agent` and `Session↔sessionId` maps.
|
||||
3. Internal edit — turn-end reason fidelity (sanctioned: edit internals to fit ACP). Extend `TurnEndReasonMap` in the proper places: (a) declaration-merge a `max-tokens` variant in the owning package (`packages/session/src/types.ts`, alongside `completed|aborted|error|disposed`) — add `max-tokens` because `FinishReasonMap` produces it (DeepSeek maps `length` → `max-tokens`); do not add `refusal`, since no current adapter produces it (unknown DeepSeek finish reasons collapse to `error`), but leave a comment in `TurnEndReasonMap` noting `refusal` should be added when an adapter first emits it (`FinishReasonMap` is merge-extensible); (b) make `agent-loop`'s `loop.ts` populate the reason from the model `finish` chunk — `assembler.finish` lives inside `runStep`, so `runStep` must return it up to `runTurn`, and the rule is "the last step's finish reason wins, but any `max-tokens` in the turn surfaces as `max-tokens`"; (c) no consumer exhaustively switches over `TurnEndReason` today (the invariants plugin switches on `SessionEventType`, and `deriveMessages` ignores `turn/end`), so adding `max-tokens` is a non-breaking extension — but recheck before landing; (d) update [docs/architecture.md](../../architecture.md) (the CI-verified loop-lifecycle/event-taxonomy doc) and the affected package READMEs/JSDoc (`dsh-session`, `dsh-agent`, `dsh-agent-loop`) per the repo doc-sync policy. This replaces a fragile "observe the finish chunk in the bridge" hack with a real, documented contract.
|
||||
4. Prompt-turn streaming plus load: translate `agent/stream-chunk` and `session/event` into `session/update`; resolve `session/prompt` on settle, mapping the harness `TurnEndReason` to the ACP `StopReason` wire enum (`completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`→`cancelled`) — a small total function with a test asserting the exact wire strings, since the SDK rejects an unknown `stopReason`. Concrete correlation, since the loop batches queued messages into one turn and `send()` does not synchronously flip to running: install listeners before `send()`; gate on an observed `agent/turn-start` (confirms work was accepted) then resolve on the next `agent/turn-end`; reject an empty/whitespace prompt up front rather than calling `send()` (no turn would ever start, so the RPC would hang). Implement `session/load` on the session-persistence resume seam.
|
||||
5. Permission gate: a single `tools/execute` listener registered with `prepend: true`, owning a `WeakMap<Agent, sessionId>` of bridge-created agents; no-op (`next()`) for unowned/no-agent calls; for owned calls → `session/request_permission` → allow (`next()`) / veto; settle the stored resolver exactly once on outcome, cancel, or connection close.
|
||||
6. Example wiring (extract a shared base). `@cordisjs/plugin-include` is itself a plugin entry that resets `ctx.baseUrl` and loads a path, so a child `cordis.yml` can nest-include a shared base; the extraction is safe because every dependent plugin declares `inject` (loader groups initialize via `Promise.all`, so YAML order is NOT the dependency mechanism — never rely on it). Extract the provider/tool core (`llm, sessions, system-prompt, tools, agents, invariants, llm-deepseek, bash-local, tool-bash`) into `examples/base.yml`; have both `coding-agent` and a new `examples/acp-agent/` include it and add their own UI plugin plus logger. Keep `agent-loop` per-example (NOT in the base): `AgentLoop` creates its configured agents in its constructor, and the two examples disagree — `coding-agent` needs a pre-created `main` (its `stdio-chat` calls `ctx.agents.get('main')`), while `acp-agent` must pre-create none (ACP `session/new` creates agents). So `coding-agent` declares `agent-loop` with `agents: [{ id: main, … }]` and `acp-agent` with `agents: []`. `acp-agent` loads `dsh-session-persistence-jsonl` (from [session persistence](../implemented/2026-06-14-session-persistence.md) — required for `session/load`), omits the stdout logger (see Risks), and adds `pnpm run demo:acp` plus the Zed `agent_servers` snippet.
|
||||
7. Tests (the repo cares a lot here): a property-based test for the protocol shape (precedent: [property-based testing](../implemented/2026-06-11-property-based-testing.md)) — fuzz arbitrary harness event sequences and assert ACP-stream invariants (never a `tool_call_update` before its `tool_call`; exactly one `session/prompt` resolution per prompt; monotonic, well-formed ordering; `stopReason` in the legal set); codec unit tests over an in-memory `Duplex` pair (drive `AgentSideConnection` without a subprocess; assert exact frames for `initialize`, `session/new`, a full prompt turn); the mandatory HMR-safety test (dispose the fiber; assert the connection closed, all `ctx.on` listeners gone, any in-flight `request_permission` settled); failure-path tests (connection closes mid-stream; closes with a permission pending; a notification `send()` rejects but the turn survives; `finish{kind:'error'|'aborted'}`; a `tools/execute` throw with no `tool/result`; a second `session/new` rejected; a `session/prompt` while one is in flight; an empty prompt rejected without hanging; a `session/load` re-derives identical history and replays it); and an e2e (`*.e2e.ts`, self-skips without `DEEPSEEK_API_KEY`) that boots `examples/acp-agent`, connects a `ClientSideConnection`, sends a real prompt, owns and disposes the harness in `afterEach`, and verifies the world (files on disk), not the agent's self-report.
|
||||
8. Docs: module/JSDoc plus a package README; extend [the extension cookbook](../../cookbook/extension-cookbook.md) with the client-driver pattern. Flip Status to `implemented` on landing; record a decision in this RFC only if it proves durable, contested, and surprising (candidates: the `tools/execute` permission-ownership rule, the npm-dependency choice) — not auto-required.
|
||||
|
||||
Deferred (each names its owning future work):
|
||||
|
||||
- Multiplexing concurrent sessions → [ACP multi-session](2026-06-14-acp-multi-session.md).
|
||||
- ~~`cwd` honoring.~~ **RESOLVED.** Originally there was no path from `session/new.cwd` to the bash workdir (`tool-bash` forwarded only an explicit `args.workdir`; `LocalBashExecutor.resolve` defaulted to its own config or `process.cwd()`), so the MVP validated `cwd` (require absolute) AND required the server to launch in the workspace root, erroring on a mismatch. This is now lifted: the validated `cwd` is stored as `SessionHeader.cwd`, and `dsh-tool-bash` defaults the bash workdir to the calling agent's `session.header.cwd` (an explicit model `workdir` still wins; a relative one resolves against it). Any absolute `cwd` is honored — the server need not launch in the workspace, and N sessions can each target a different directory. Widening scope beyond the single cwd (`additionalDirectories`) remains deferred.
|
||||
- Client `terminal/*` proxying (a live editor terminal) and `fs/*` (editor-rendered diffs) — a future `BashExecutor` over the [capability seams](../implemented/2026-06-13-capability-seams.md) bash seam, gated on `clientCapabilities.terminal`.
|
||||
- Image/audio prompts (blocked on the DeepSeek adapter, which skips `image` blocks today), modes, auth, `available_commands`/slash-commands, `plan`, and `usage_update`.
|
||||
|
||||
## Risks
|
||||
|
||||
stdout is the protocol — guaranteed by config, not by monkey-patching. The console logger writes through `console.log` to stdout, so any stdout UI/logger plugin corrupts JSON-RPC. The guarantee is config-only: the `acp-agent` example loads no stdout plugin (no console logger, no `stdio-chat`) and, if logging is wanted, uses a stderr exporter. A defensive process-wide `process.stdout.write`/`console.log` hijack inside `dsh-acp` is explicitly rejected — it lives outside Cordis' effect-scoped, HMR-friendly plugin model, races the connection's own stdout handoff, and fights the logger. A test asserts the example emits only framed JSON-RPC on stdout.
|
||||
|
||||
New third-party runtime dependency plus protocol drift: `@agentclientprotocol/sdk` is young (0.25.x, recently renamed) and evolving. Pin the version and isolate churn to the one bridge package. This is not a vendoring-policy violation — [vendoring Cordis as source](../implemented/2026-06-11-vendor-cordis-as-source.md) vendors the framework; genuine third-party deps already live on npm (`@earendil-works/pi-ai`).
|
||||
|
||||
Turn-settle and prompt-correlation hazards: honor "queued messages batch into one turn" and "`send()` does not synchronously flip to running" (see `stdio-chat.ts` and the defensive-patterns section of [docs/architecture.md](../../architecture.md)); gate resolution on an observed running→idle transition and handle the empty-prompt / no-work branch so an RPC can't hang.
|
||||
|
||||
Permission-await and disposal hangs: a pending `request_permission` whose connection closes or whose turn aborts must settle exactly once; disposal must reach quiescence (observe the interface-level settle signal — `agent/status` reaching `idle`/`disposed`, since `agent.done` is `LoopAgent`-only), not orphan awaits on a closed pipe.
|
||||
|
||||
The 100% per-file coverage gate (repo policy) makes a branch-heavy protocol bridge real work. Accepted deliberately, surfaced so it isn't a surprise at PR time.
|
||||
|
||||
ACP protocol-shape details (exact method names, `session/update` variants, permission option kinds, stop reasons) are taken from the ACP spec and the `@agentclientprotocol/sdk` types; they are not independently verifiable until the dependency is added, so the implementation pins the SDK version and conforms to its types rather than to this RFC's prose where they differ.
|
||||
37
docs/rfc/proposed/2026-06-14-acp-multi-session.md
Normal file
37
docs/rfc/proposed/2026-06-14-acp-multi-session.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# RFC: Multiplex concurrent ACP sessions over one connection
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
> **Implementation status:** the multi-session bridge (steps 1, 3, 4) and the bash task-ownership isolation are implemented in `packages/acp` + `packages/tool-bash`. **Per-session *permission* ownership is deferred** — it depends on [the ACP support permission gate](2026-06-14-acp-agent-client-protocol.md) (`TODO(rfc010-permission-gate)`), which is itself deferred; the `agent→sessionId` reverse map the gate will route through is in place. Step 2's "real per-session disposer scope" is also deferred (`TODO(rfc010-agent-disposal)`): the bridge demuxes via id-keyed maps and global `ctx.on` listeners (correct and leak-free — disposal drains every session in parallel to quiescence), and a per-agent disposer seam is the follow-up. Status stays `proposed` until per-session permission ownership lands.
|
||||
|
||||
## Problem
|
||||
|
||||
[ACP support](2026-06-14-acp-agent-client-protocol.md) ships with a single active session per connection: a second `session/new` is rejected. Editors expect to run several conversations over one agent subprocess — a user opens multiple threads, or a client pre-warms sessions. The single-session guard is a deliberate MVP scope cut, not an architectural limit; this RFC lifts it.
|
||||
|
||||
This paragraph is historical: the multi-session bridge has landed. The remaining proposed work is per-session permission ownership plus the lifecycle seams now tracked in [agent lifecycle and ownership seams](2026-06-18-agent-lifecycle-and-ownership-seams.md).
|
||||
|
||||
## Proposal
|
||||
|
||||
The harness core already supports many agents (`AgentRegistry.list()` and `AgentLoop.create` impose no count limit), so multiplexing is a bridge-layer change in `@deepseek-ai/dsh-acp`, not a loop or core change.
|
||||
|
||||
- Lift the single-session guard in `session/new`; allow N live sessions, each mapped to its own `LoopAgent`.
|
||||
- The bridge's `sessionId→agent` and `Session→sessionId` maps (introduced single-entry by [the ACP support RFC](2026-06-14-acp-agent-client-protocol.md)) become true multi-entry, plus a third `agent→sessionId` reverse map: the `tools/execute` permission gate receives only `exec.agent` (no sessionId), so it needs an O(1) reverse lookup to find the owning session. Every `agent/*` event and every `session/event` is demuxed strictly by id, so two sessions streaming at once never interleave their `session/update` notifications.
|
||||
- Per-session prompt queues: [the ACP support RFC](2026-06-14-acp-agent-client-protocol.md)'s single-entry in-flight-prompt state becomes multi-entry — one in-flight prompt *per session*, tracked per `sessionId`.
|
||||
- Per-session cancel routing: `session/cancel` aborts only its own session's agent and settles only that session's in-flight prompt. `agent.abort()` drives a per-agent `AbortController`, so the per-session `exec.signal` is the natural isolation fence.
|
||||
- Per-session permission ownership: a `session/request_permission` and its outcome are bound to the originating session via the reverse map, so a permission prompt or a cancel in one session can never resolve another session's pending permission.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Generalize the two id maps to multi-entry and add the `agent→sessionId` reverse map; add a per-session record holding the agent, the in-flight-prompt state, the pending-permission registry, and the session's disposer scope (see step 2).
|
||||
2. Give each session a real per-session disposer scope, NOT `ctx.extend()` — in Cordis `ctx.extend()` only creates a child context/prototype, but `ctx.on()` registered on it is still owned by the current plugin fiber, so disposing it would not remove that session's listeners. Use a genuine child fiber (load a per-session sub-plugin, e.g. `ctx.plugin(...)` returning a fork, or collect each session's `ctx.on` disposers in its session record and call them on teardown). Demux every `agent/*` and `session/event` by id into the right session record. Note the single global `tools/execute` listener stays on the bridge root (it must see all agents) and routes via the reverse map.
|
||||
3. Lift the `session/new` guard; keep `session/load` ([from ACP support](2026-06-14-acp-agent-client-protocol.md)) working per session.
|
||||
4. Tests for cross-session isolation: two sessions streaming and permission-prompting concurrently never interleave; a cancel/abort in one session leaves the other's stream and pending permission untouched; per-session in-flight-prompt enforcement holds independently; disposing one session leaves the others running.
|
||||
|
||||
## Risks
|
||||
|
||||
Listener fan-out cost: each session adds listeners; ensure disposal of one session removes exactly its own and the connection teardown ([from ACP support](2026-06-14-acp-agent-client-protocol.md)) still reaches quiescence across all sessions.
|
||||
|
||||
The subtle correctness trap is cross-session leakage — a cancel or abort on one session settling another session's pending permission. The per-session permission ownership rule (routed via the `agent→sessionId` reverse map) and its isolation test are the guard.
|
||||
|
||||
Shared background-task state: the bash executor's task ids are global and predictable (`bash-1`, `bash-2`, …), and `bash_output`/`bash_kill` look up by id without checking the caller. Under one session this is benign; under N sessions one session's agent could read or kill another's background task. This is a pre-existing `tool-bash` gap that multi-session turns into a real isolation hole — fixing it (validate the caller against the task owner) belongs with this RFC or a companion `tool-bash` change.
|
||||
111
docs/rfc/proposed/2026-06-15-optional-code-mode.md
Normal file
111
docs/rfc/proposed/2026-06-15-optional-code-mode.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# RFC: Optional Code Mode — model writes TypeScript against an SDK of all tools
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
Today the agent loop advertises every registered tool to the model as a native JSON-schema function definition. `ToolRegistry` feeds its schemas into `ctx.systemPrompt`, the loop puts them on `GenerateOptions.tools`, and the adapter serializes them to the provider's function-calling wire format. The model then invokes one `tool-call` block per step, the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../architecture.md)), and **every** intermediate `tool-result` re-enters the model's context on the next request.
|
||||
|
||||
For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each of those round-trips drags the entire intermediate result back into context whether the model needs it or not.
|
||||
|
||||
Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) (shipped as the `@cloudflare/codemode` npm package) proposes an alternative grounded in a simple observation: LLMs are better at writing code than at emitting tool calls, because they have seen millions of lines of real code and comparatively few contrived tool-calling traces. Instead of one tool call per step, the model writes a TypeScript program against a generated SDK that wraps all the tools, and that program is executed. The model curates what comes back — only what it `console.log`s and/or returns — instead of every intermediate result. The SDK functions are async, so the model can *express* fan-out (`Promise.all`) naturally in code; this RFC initially **serializes** those dispatches (§ Concurrency) until the tool contract grows concurrency-safety metadata, so the early win is composition and fewer round-trips, not parallelism.
|
||||
|
||||
This RFC proposes an **optional** Code Mode for the DeepSeek Harness, covering **all** tools uniformly — built-in and future MCP — with no per-tool work, implemented Cordis-style with **zero core-package changes**. It fully specifies the code-execution seam and the SDK-generation pipeline, but ships only a minimal `node:vm` reference stub for execution; the hardened, sandboxed execution substrate is **deferred to a follow-up RFC** (see Risks). This RFC does not change the agent loop, and it leaves native tool-calling exactly as it is — Code Mode is a plugin you load, not a replacement.
|
||||
|
||||
## Proposal
|
||||
|
||||
The design follows the codebase's capability-seam pattern ([capability seams](../implemented/2026-06-13-capability-seams.md), the `bash` template) as a three-package split, plus one consumer plugin. Nothing in `dsh-session`, `dsh-agent`, `dsh-agent-loop`, `dsh-llm`, `dsh-tools`, or `dsh-system-prompt` changes.
|
||||
|
||||
**Prior art.** `@cloudflare/codemode` validates this shape directly and several of its decisions are adopted below. Its `Executor` interface is deliberately tiny — `execute(code, fns) → { result, error?, logs? }` — with a production `DynamicWorkerExecutor` (isolated Workers) and a six-line `NodeVMExecutor` example as two implementations behind it: exactly the interface/implementation split [the capability-seam pattern](../implemented/2026-06-13-capability-seams.md) prescribes. It generates TypeScript type definitions from tools for the model's context and runs the generated JavaScript in a sandbox, capturing console output alongside the return value. It normalizes model output into an async arrow function via AST parsing (acorn) and sanitizes tool names into valid JS identifiers (`my-tool` → `my_tool`, `delete` → `delete_`). It blocks outbound network by default. The transferable lessons — minimal executor contract, host-side type derivation, capture-output-and-return-value, name sanitization, AST-normalize the code, isolate by default — are folded into the design below. What does **not** transfer is the substrate: Cloudflare's isolation is Workers-specific; our equivalent hardened substrate is the deferred follow-up.
|
||||
|
||||
**Prompt-budget tradeoff (Code Mode is not unconditionally cheaper).** Deriving the SDK types host-side costs no extra *discovery* round-trip, but the generated `.d.ts` is injected into the system prompt (§3a), so the type definitions themselves **do** consume context — and for an all-tools SDK that cost scales with every registered tool and can be comparable to, or larger than, the native JSON schemas it replaces. Code Mode's saving is on the **output/result** side (the model curates what comes back; intermediate results never re-enter context) and on **round-trips** (compose many calls in one program), not on the input-side tool description. The net win is workload-dependent: it pays off for multi-call, large-intermediate-result workflows and can cost *more* for a single call against a large tool surface. The `.d.ts` section is a prefix-stable prompt prefix, so prompt caching amortizes its per-turn cost across a session; the RFC notes that caching is what keeps the injected SDK affordable, and that a deployment with a very large tool surface should weigh the SDK size against native schemas rather than assume Code Mode is strictly cheaper.
|
||||
|
||||
**1. Interface package `packages/code-runtime/`** — a new package `@deepseek-ai/dsh-code-runtime` owning `ctx.codeRuntime`, depending only on `cordis`. It defines an abstract `CodeRuntime extends Service` plus the execution vocabulary. The runtime knows **nothing** about `ctx.tools`: it is handed a set of named async functions (the resolved SDK bindings), runs the program, and captures output. The result shape mirrors Cloudflare's proven-minimal contract so an error is a *field on a resolved result*, not a throw the runtime is expected to make:
|
||||
|
||||
- `CodeRunRequest = { code: string; sdk: SdkBinding[]; signal?: AbortSignal }`
|
||||
- `CodeRunResult = { result: unknown; logs: string[]; error?: string }`
|
||||
- a readonly `safe: boolean` on the `CodeRuntime` service — `false` for an unsandboxed stub, `true` only for a real isolating substrate; consumers gate on it (§2).
|
||||
- `SdkBinding = { namespace: string; fns: Record<string, (args: unknown) => Promise<unknown>> }`
|
||||
|
||||
Per the "explicit > implicit at seams" convention, the request spells out every field the runtime acts on; defaulting (e.g. an output cap, a timeout derived from `signal`) is the implementation's explicit job, not a hidden `?? default` inside `run()`. The split into interface + implementation is justified under [the capability-seam pattern](../implemented/2026-06-13-capability-seams.md) because there is **genuinely more than one planned implementation** — the node:vm stub *and* the hardened substrate (a real isolate, or the generated program run as a sandboxed process through the existing `ctx.bash` seam) that is scheduled follow-up work, not speculative optionality. The capability-seam pattern warns against splitting preemptively when only one implementation is conceivable; here a second is not just conceivable but required before any untrusted use, so the seam earns its keep.
|
||||
|
||||
**Backends can differ by language/runtime, not only by trust level.** The two implementations above (unsafe stub vs. hardened substrate) differ along the *trust* axis while staying TypeScript/JS, but nothing in the `CodeRuntime` contract — a program string plus a set of named async SDK bindings in, and a `{ result, logs, error? }` out — is bound to one source language. The same seam can host backends that differ along the *language* axis, executing a program written in something other than TypeScript. Two illustrative directions:
|
||||
|
||||
- **An AssemblyScript backend.** AssemblyScript is a strict TypeScript subset that compiles to WebAssembly, so a program stays familiar to a TS-fluent model while the WASM boundary supplies exactly the sandboxing the hardened substrate is meant to provide — memory isolation and no ambient host authority come from the runtime rather than from after-the-fact hardening of `node:vm`. This is an appealing route to a `safe = true` backend.
|
||||
- **A Python backend.** Python is arguably the model's most native language — it has seen far more real Python than any tool-calling trace — which is the same "LLMs write better code than tool calls" argument that motivates Code Mode, taken one step further. A Python backend is itself a sub-seam over different Python *runtimes*: **CPython** (in-process or a sandboxed subprocess via `ctx.bash`) for maximum fidelity and ecosystem access, or a more controllable / embeddable interpreter — Pyodide (CPython on WASM), RustPython, or a restricted embedded interpreter — when isolation, deterministic resource limits, or a clean capability boundary matter more than running arbitrary native extensions.
|
||||
|
||||
These are illustrations of the seam's reach, **not commitments** — the MVP ships only the TypeScript path. The honest caveat is that the *execution* contract is language-agnostic but the *presentation* is not: the SDK-generation pipeline below (§3a and the `jsonSchemaToTs` codegen, which emits a TypeScript `.d.ts`) is TypeScript-specific, so a non-TS backend pairs the shared `CodeRuntime` contract with its own language-appropriate SDK generator and system-prompt section (a `.pyi` stub and Python usage instructions for the Python backend, AssemblyScript-flavored types for that one). The runtime seam is reused as-is; only the codegen/prompt half is per-language.
|
||||
|
||||
**2. Implementation package `packages/code-runtime-vm/`** — a new package `@deepseek-ai/dsh-code-runtime-vm`, the `node:vm` reference stub. It type-erases the model's TypeScript via the compiler's `transpileModule` (or sucrase) — the types exist only to guide the model; the runtime is plain JS — then wraps the body in an async IIFE for top-level `await` (Cloudflare's `NodeVMExecutor` does literally `new AsyncFunction("codemode", "return await (${code})()")`), runs it in a `vm.Context` whose globals are a capturing `console` and the SDK namespace objects, awaits the IIFE, and captures the return value, the buffered logs, and any thrown error (as `error: string`). It applies an **output cap** (truncate captured logs) and a **timeout tied to `request.signal`**. These caps limit blast radius; **they are not a security boundary**. node:vm is **not** isolation: withholding `require`/`process` does not contain anything (code escapes via `constructor`/prototype reflection), and per [AGENTS.md](../../../AGENTS.md) the harness must never hand model output the ambient environment.
|
||||
|
||||
**The unsafe-runtime guard is enforceable, not a README warning.** Because a README caveat is not a control — and AGENTS.md's "never hand model output ambient authority" is a hard rule, not advice — the design makes the danger refuse to run by construction. Two layers:
|
||||
|
||||
- **The runtime declares its trust level.** `CodeRuntime` carries a readonly `safe: boolean` (a `node:vm`-class stub returns `safe = false`; a real isolate/sandboxed-process substrate returns `safe = true`). The `code-runtime-vm` constructor *additionally* requires an explicit opt-in — `new VmCodeRuntime({ unsafe: true })` — and **throws** if that flag is absent, so merely depending on the package and wiring it cannot silently produce a live unsafe runtime; the operator must type the word `unsafe`.
|
||||
- **The consumer refuses to expose `run_code` over an unsafe runtime by default.** When `code-mode` initializes, if `ctx.codeRuntime.safe === false` it does **not** register `run_code` unless the plugin itself is configured with an explicit acknowledgement (e.g. `code-mode` config `allowUnsafeRuntime: true`). Absent that, it logs a typed error and registers nothing — so a real model never reaches an unsandboxed runtime by a single config slip. The refusal path is tested: with the acknowledgement unset and an unsafe runtime, `run_code` is absent (and the wire tool list is unchanged from native); with both opt-ins set, it registers and runs. This keeps the unsafe reference backend usable for tests and trusted local demos while making production misuse take two deliberate, greppable flags rather than one mistake.
|
||||
|
||||
`code-runtime-vm` is therefore documented as **reference / test-only / unsafe-for-untrusted-input**, acceptable in the MVP only because the code runs at harness trust *and* both opt-in flags must be set. Signal handling is best-effort: it aborts in-flight sub-dispatches but cannot reliably interrupt a hot synchronous loop (`while(true){}`) in node:vm — another reason the hardened substrate is deferred, not optional-forever.
|
||||
|
||||
**3. Consumer plugin `packages/code-mode/`** — a new package `@deepseek-ai/dsh-code-mode`, the plugin that wires everything together. It declares `inject = ['tools', 'systemPrompt', 'codeRuntime']` — Cordis throws on access to a service that is not injected, and keeps the plugin inactive until all three exist (the same pattern as `tool-bash`'s `inject = ['tools', 'bash']`), which also gives correct load-ordering relative to `code-runtime`/`code-runtime-vm`. The plugin contributes four things, all through existing seams:
|
||||
|
||||
**3a. Tool presentation — a lazy system-prompt section (the injection seam already exists).** `dsh-system-prompt` already provides the Cordis-idiomatic way for any plugin to inject prompt snippets: `ctx.systemPrompt.section({ name, order, text })`, fiber-scoped and auto-disposed via `ctx.effect()`, where `text` may be a lazy `() => string` re-evaluated at each assembly. No new mechanism is needed or invented. Code Mode registers a lazy section (high `order` so it lands last) whose thunk reads `ctx.tools.schemas()` at assembly time and regenerates the SDK `.d.ts` plus usage instructions from the currently-registered tool set. Because the thunk reads the live registry, coverage of every tool — built-in, MCP, future — is automatic.
|
||||
|
||||
**3b. Wire tool-list enforcement — an `agent/request` listener (the authoritative seam).** The goal "exactly one tool reaches the wire" must be enforced where the wire request is finalized. The loop calls `ctx.systemPrompt.assemble()` first, *then* builds `GenerateOptions` (seeding `tools` from `assembly.tools`), *then* runs the `agent/request` waterfall, *then* calls `ctx.llm.stream()`. A `system-prompt/assemble` listener can only influence the *seed*; `agent/request` is the last seam before the model call, so it is authoritative. The plugin registers an `agent/request` listener that does `const final = await next(); return { ...final, tools: [runCodeSchema] }` — overriding the value *returned by* `next()`, not the inbound argument, so it dominates the cooperative request listeners it wraps. It registers with `prepend: true` to sit at the outer edge of the waterfall chain. One honest caveat, stated in the RFC body: `ctx.llm.stream()` itself runs a further `llm/stream` waterfall before the adapter, so the guarantee is "authoritative within the agent request pipeline," not an absolute wire invariant; if a hard invariant is ever required, a defensive `llm/stream` assertion with a spy adapter covers it in tests.
|
||||
|
||||
**3c. The single tool — `run_code`.** Registered normally in `ctx.tools` with one parameter `{ code: string (required) }`. Because it is an ordinary tool, the unchanged loop dispatches it through the normal path — this is the crux of "zero loop changes." Its `execute(args, exec)`:
|
||||
|
||||
1. Builds the SDK bindings. For each real tool, an async `invoke(callArgs)` that **checks `exec.signal?.aborted` (throwing if set) before and after** calling `ctx.tools.execute({ callId: <deterministic sub-id>, name, arguments: callArgs, agent: exec.agent, signal: exec.signal })`, then maps the resulting `ContentBlock[]` to a simplified `{ output, isError }` (text blocks for the MVP), and emits an observability event. The explicit abort check matters because `ctx.tools.execute()` *catches* thrown tool errors and converts them to `isError` results — without the check, an aborted sub-call would look like ordinary error data and the program would keep running instead of stopping. Sub-dispatch still flows through the `tools/execute` waterfall, so permission/sandbox/hook plugins apply to code-mode calls exactly as to native ones.
|
||||
2. Calls `ctx.codeRuntime.run({ code: args.code, sdk: bindings, signal: exec.signal })`.
|
||||
3. Surfaces the outcome. A *successful* run returns `[{ type: 'text', text: <console logs + return value> }]`. A *runtime-error* result cannot be reported by returning content, because a normal `ToolDefinition.execute()` returns only `Promise<ContentBlock[]>` and `ToolRegistry.execute()` hardcodes `isError: false` on any successful return — `isError: true` arises only from the registry's catch path. So on an error result the tool **throws a `CodeRunError extends HarnessError`** (`HarnessError` is exported from `dsh-llm`; the registry catch turns any throw into `isError: true` with the message as text, and a `HarnessError` additionally carries structured `{ name, code }`). An alternative — registering `run_code` handling as a `tools/execute` listener that returns a full `ToolExecutionResult` and can set `isError` directly — is noted; the throw is simpler and preferred.
|
||||
|
||||
**3d. Result discipline — what the model receives.** The model gets back **only the captured console output and/or the program's return value** (the model chooses which to surface). Intermediate sub-call results are **never** returned to the model. This is the core context-saving benefit: the agent curates its own output, exactly as a script's stdout curates a pipeline's intermediate state.
|
||||
|
||||
**Sub-call CallIds.** Real tool calls dispatched from inside `run_code` need ids, but `CallId` is normally provider-issued (a branded string for correlating a call with its result — only brand-wrapped via `CallId()`, with no generator and no documented session-global-uniqueness guarantee). The plugin mints deterministic sub-ids scoped to the parent: `` `${exec.callId}:code:${n}` `` with a per-run counter `n`. These are unique within one `run_code` run (assuming the parent `callId` is unique, which the provider guarantees per turn); the `code/dispatch` event additionally carries the session log's `seq` so the UI and persistence can order and disambiguate globally without relying on the id alone. `ToolExecution.agent` is optional; the normal loop always supplies it (and with it `exec.agent.session`, the log `code/dispatch` appends to). A `run_code` execution arriving without `exec.agent` still runs (sub-calls propagate `agent: undefined`, exactly as the loop's own contract allows) but **skips session-log observability** — with no session to append to, those direct runs are simply not logged.
|
||||
|
||||
**Observability without context cost.** Each sub-dispatch emits a session event **declared by the `dsh-code-mode` plugin itself** via `SessionEventMap` declaration merging (the map is merge-extensible precisely so plugins can add events without touching `dsh-session`). Shape: `code/dispatch` with `{ parentCallId, subCallId, name, arguments (or redacted), isError, summary }`, ordered by the session log's own `seq`. `deriveMessages()` does **not** translate it into a model message — an unknown event type falls through its `default`, per the merge-extensible-union convention — so the UI and persistence ([session persistence](../implemented/2026-06-14-session-persistence.md)) can render every sub-call while the model's context only ever receives the single `run_code` tool-result. Because the event lives in the plugin, this adds no core change.
|
||||
|
||||
**SDK codegen.** A pure `jsonSchemaToTs(schema)` in `code-mode` maps the JSON-schema subset the `defineTool` DSL produces (object/string/number/boolean/array, `properties`, `required[]`, `enum` → string-literal union, nested objects, array `items`) to a TS type literal. It is **total**: any unsupported construct (`$ref`, `oneOf`/`anyOf`, `integer`, `null`, `additionalProperties`, or any raw MCP shape it does not recognize) degrades to `unknown` without throwing — it never crashes codegen. Typing is best-effort, not a guarantee, because MCP tools accept arbitrary JSON Schema and `ToolSchema.parameters` is typed only as `Record<string, unknown>`. Because `ToolSchema.name` is an arbitrary string (not necessarily a valid TS identifier), the SDK is generated as a **namespace with quoted access** (e.g. `tools["some-mcp-tool"](args)`) plus safe camelCase aliases where the name is a clean identifier; alias collisions and TS reserved words fall back to quoted-only access (no duplicate alias emitted). This mirrors Cloudflare's `sanitizeToolName`. `run_code` itself is filtered out of the SDK. The MVP surfaces text content only; image and other block types in sub-results are deferred (noted as a limitation).
|
||||
|
||||
**Concurrency — serialized by default (the binding must enforce it).** The SDK functions are async, so a model writing `await Promise.all([tools.a(...), tools.b(...)])` would *start both* immediately, and each would call `ctx.tools.execute` right away — i.e. the binding shape makes concurrent dispatch the **default**, not an opt-in. Because the tool contract carries **no concurrency-safety metadata today** (parallel tool execution and a concurrency-safety hint are an open TODO in both `dsh-tools` and [docs/architecture.md](../../architecture.md): "phase 1 executes tool calls sequentially"), concurrent dispatch through a not-yet-hardened tool may race. So a prose "may serialize" is not sufficient. **Decision: the MVP SDK bindings enforce serialization** — each `run_code` invocation owns a per-run dispatch queue, and every `invoke()` chains onto it (`tail = tail.then(() => ctx.tools.execute(...))`), so even `Promise.all` over SDK calls executes them one at a time in submission order. This is a hard acceptance criterion, with a test that issues `Promise.all([...])` from a program and asserts the underlying `ctx.tools.execute` calls did **not** overlap (e.g. a probe tool records enter/exit and the test asserts no interleaving). The `.d.ts` may *describe* the model-visible functions as async (they are), but the implementation guarantees serial execution. Lifting serialization is deferred: only once a tool can declare itself read-only / concurrency-safe does the binding allow those specific tools to overlap. The same per-run queue is where the before/after abort checks (§3c) live, so an aborted run drains no further queued dispatches.
|
||||
|
||||
**Tool visibility tiers (design intentionally skipped).** A natural extension is to mark each tool with a *visibility tier*: some tools "direct-call eligible" (still offered as native wire tools alongside `run_code`), some "code-mode only" (reachable solely from within a `run_code` program, never on the wire), and the default "both." This would let a deployment keep a few high-frequency or approval-gated tools as direct calls while routing the long tail through Code Mode, or hide composition-only primitives from the native surface entirely. This RFC notes the possibility but **intentionally skips the detailed design** — the per-tool metadata, how it interacts with the `agent/request` enforcement in 3b, and the presentation split in 3a are left to a follow-up. The MVP is the simple two-state model: Code Mode on (everything via `run_code`) or off (everything native).
|
||||
|
||||
**Optionality / toggle.** Loading the `code-mode` plugin enables Code Mode for that context; not loading it leaves today's native tool-calling untouched. The two are mutually exclusive within one ctx, because Code Mode rewrites the wire tool list down to `[run_code]`. Per-agent selection via ctx forks, and the visibility tiers above, are future work; the MVP toggle is plugin presence.
|
||||
|
||||
## Alternatives
|
||||
|
||||
**Result elision / summarization over native tool-calling (the narrower route).** The Problem has two halves — context bloat (every intermediate `tool-result` re-enters context) and serial composition (one tool call per round-trip). The context-bloat half can be addressed *without* any code-execution runtime: keep provider tool-calling exactly as it is, and add a plugin on the `agent/request` waterfall (or a compaction pass akin to [the session-persistence work](../implemented/2026-06-14-session-persistence.md)) that elides or summarizes older `tool-result` blocks before they re-enter the model's context — drop them past a window, replace large payloads with a digest, or keep only the blocks the model still references. This is strictly less invasive than Code Mode: no new runtime seam, no model-written programs, no new safety surface. It is the right tool if context growth is the only pain.
|
||||
|
||||
It is insufficient for the **composition / round-trip** half, which is the decisive reason this RFC does not stop there. Elision still pays one model round-trip per tool call: a loop over N items is N turns, a branch on an intermediate value is a turn to fetch then a turn to act, and post-processing (filter, join, reduce) either happens in the model's head over full payloads or not at all. Code Mode collapses all of that into one program — the loop, the branch, the join run in the runtime, and only the curated result returns. Elision also cannot express fan-out or data-dependent control flow; it only shrinks what comes back. So the two are complementary, not competing: elision could even layer *under* Code Mode for the residual native-tool paths. The RFC chooses Code Mode because the round-trip/composition cost is the larger structural limit, and accepts the new code-execution surface as the price — which is exactly why the execution substrate is gated behind the enforceable safety guard (§2) and the hardened backend is a hard prerequisite for untrusted use.
|
||||
|
||||
**Why not change the loop to dispatch native tool calls in parallel instead?** That is the other obvious answer to the round-trip cost, and it remains valid future work (it is the open `dsh-tools`/architecture.md TODO). But it is a core-loop change requiring the same concurrency-safety metadata Code Mode defers, and it still does not give the model *composition* (branch/loop/post-process between calls) — only parallelism of independent calls the model already decided to make in one step. Code Mode delivers composition with zero core change; parallel native dispatch and Code Mode can coexist later.
|
||||
|
||||
## Plan
|
||||
|
||||
1. Scaffold the interface package `packages/code-runtime/` per [the cookbook](../../cookbook/adding-a-package.md): abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) with a readonly `safe: boolean`, the `declare module 'cordis'` ctx key, the `CodeRunRequest`/`CodeRunResult`/`SdkBinding` vocabulary, method contracts documented in JSDoc (what `run` captures, abort semantics, that an error is a result field not a throw, what `safe` means). HMR-safety test (dispose the contributing fiber, assert `ctx.codeRuntime` is gone).
|
||||
2. Scaffold the implementation package `packages/code-runtime-vm/`: the node:vm stub — `safe = false`, a constructor that **throws unless given `{ unsafe: true }`**, transpile/type-erase, async-IIFE wrap, capturing `console`, SDK globals, return-value/logs/error capture, output cap, signal-tied timeout. Tests for output capture, return value, error-as-field, abort, the constructor refusal without `unsafe`, and a README documenting the "not a sandbox, trusted-only" caveat prominently.
|
||||
3. Scaffold the consumer plugin `packages/code-mode/`: `jsonSchemaToTs` codegen with namespace/quoted-access + alias handling (unit tests, including non-identifier MCP names and unsupported-shape → `unknown`); the registered lazy `ctx.systemPrompt.section()` carrying the SDK `.d.ts`; the `agent/request` listener (`prepend: true`) collapsing `request.tools` to `[run_code]` after `await next()`; the **unsafe-runtime gate** (refuse to register `run_code` when `ctx.codeRuntime.safe === false` unless `allowUnsafeRuntime` is set); the `run_code` tool with the dispatch bridge (per-run serialization queue, deterministic sub-call ids, before/after abort checks, `CodeRunError` on error results); and the `code/dispatch` event declared here via `SessionEventMap` merge. Declare `inject = ['tools', 'systemPrompt', 'codeRuntime']`.
|
||||
4. Tests: HMR-safety (dispose removes the tool, the section, and the listener); a waterfall test that the wire tool list is exactly `[run_code]` (spy adapter, asserting via `agent/request` and optionally `llm/stream`); an integration test that a program calling two tools returns only its printed/returned output (verify the world, not the self-report); a **serialization test** that `Promise.all([...])` over SDK calls does not overlap the underlying `ctx.tools.execute` invocations (a probe tool records enter/exit; assert no interleaving); `deriveMessages()` ignores `code/dispatch`; abort mid-program stops further dispatches; `CodeRunError` surfaces as `isError: true`; and the **unsafe-runtime refusal test** (§3, the VM-guard): with the unsafe flag unset, a non-mock agent's `run_code` is refused; with it set, the program runs.
|
||||
5. Wire an example: `examples/coding-agent-code-mode` (or a config flag on the existing example) loading the trio. Running it against the node:vm stub requires both opt-ins (`VmCodeRuntime({ unsafe: true })` and `code-mode`'s `allowUnsafeRuntime`); the example sets them explicitly and comments why, or uses a mock model — a real model never reaches the unsandboxed stub without those deliberate flags. Add a `pnpm run demo:*` entry.
|
||||
6. Docs: update [docs/architecture.md](../../architecture.md) (a `ctx.codeRuntime` row in the service map, a Code Mode note under the tool pipeline / capability seams sections); add a [cookbook](../../cookbook/) note on writing a `CodeRuntime` backend; and **file the follow-up RFC for the hardened execution substrate** (the isolate/sandboxed-process design, the additional-language backends sketched in §1 — AssemblyScript/WASM, Python — with their per-language SDK generators, plus the tool-visibility-tier design skipped here). On landing, move this file to `implemented/` and update its row in [the RFC index](../README.md).
|
||||
|
||||
## Risks
|
||||
|
||||
node:vm is not a sandbox. This is the single biggest caveat. Withholding `require`/`process` is not a boundary; the MVP runs at harness trust only; the hardened substrate is a hard prerequisite before any untrusted use and is the explicit subject of a follow-up RFC. The guard is enforceable, not just documented: the runtime exposes `safe: boolean`, the VM stub throws unless constructed with `{ unsafe: true }`, and `code-mode` refuses to register `run_code` over an unsafe runtime unless separately acknowledged (`allowUnsafeRuntime`) — production misuse requires two deliberate, greppable flags, and the refusal path is tested.
|
||||
|
||||
Wrong seam would leak tools. If the wire tool list were enforced only in `system-prompt/assemble`, a later `agent/request` listener could re-add tools. Mitigation: enforce `request.tools = [run_code]` in the `agent/request` waterfall (the authoritative seam, run last before `llm.stream()`) with `prepend: true`, and assert exactly one wire tool in tests. The residual `llm/stream` caveat is documented, not hidden.
|
||||
|
||||
Concurrency before the contract supports it. The binding shape makes concurrent dispatch the default, and the tool contract has no concurrency-safety metadata yet, so unguarded `Promise.all` over SDK calls could race a not-yet-hardened tool. Mitigation: the MVP bindings enforce a per-run serialization queue (every `invoke` chains onto the previous), with a test asserting `Promise.all` from a program does not overlap the underlying `ctx.tools.execute` calls. Per-tool parallelism is unlocked only once a tool can declare itself concurrency-safe.
|
||||
|
||||
Two presentation modes to keep coherent. A tool added later must work in both native and Code Mode. Mitigation: both the codegen thunk and the `agent/request` listener read `ctx.tools.schemas()`, so coverage is automatic; a test asserts every registered schema produces valid `.d.ts`, including non-identifier MCP names via quoted access.
|
||||
|
||||
Type-erased runtime is not type-checked. The model can write code that type-checks against the advisory `.d.ts` but throws at runtime, and MCP-schema typing is best-effort. Mitigation: errors are captured as `CodeRunResult.error` and surfaced so the model can self-correct; the `.d.ts` is explicitly advisory.
|
||||
|
||||
Lost observability of sub-calls. Routing everything through one `run_code` result hides the individual calls from the model — and could hide them from operators too. Mitigation: the plugin-declared `code/dispatch` event keeps every sub-call in the session log and UI without polluting model context.
|
||||
|
||||
Abort granularity. node:vm cannot reliably interrupt hot synchronous code, and `ctx.tools.execute()` converts thrown aborts into `isError` data. Mitigation: the SDK bindings check `signal.aborted` and throw before/after each dispatch so an aborted sub-call stops the program; the vm stub wraps the run in a signal-tied timeout; the hardened substrate addresses the hot-loop case.
|
||||
|
||||
Unsafe example wiring. A demo running a real model through the node:vm stub would hand model output ambient authority. Mitigation: examples are mock-model or explicitly marked unsafe; `code-runtime-vm` is labeled reference/test-only.
|
||||
|
||||
Non-text sub-results dropped in the MVP. Image and other block types from sub-calls are not surfaced into the program yet. Mitigation: noted as a known limitation; block-type handling deferred.
|
||||
67
docs/rfc/proposed/2026-06-16-typed-event-schemas.md
Normal file
67
docs/rfc/proposed/2026-06-16-typed-event-schemas.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# RFC: Runtime schemas for the event vocabulary (Zod vs the merge-extensible-map pattern)
|
||||
|
||||
Status: proposed
|
||||
|
||||
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
||||
|
||||
## Problem
|
||||
|
||||
The harness models its core vocabulary — content blocks, message sources, finish reasons, turn triggers, turn-end reasons, and session events — as **merge-extensible maps**: a TypeScript `interface` (e.g. `SessionEventMap`, `ContentBlockMap`) that plugins augment via declaration merging, with the public union derived as `Map[keyof Map]`. This is the repo's universal extension pattern, documented in [docs/architecture.md](../../architecture.md) ("The same merge-extensible-map pattern is used for `MessageSource`, `FinishReason`, `TurnTrigger`, and `TurnEndReason`") and relied on by the `defineTool` `InferArgs` DSL and the `assertNever` exhaustiveness convention.
|
||||
|
||||
The pattern is **compile-time only**. The types vanish at runtime: there is no schema object to validate an incoming value against, parse untrusted input with, or enumerate at runtime. Two concrete consequences surfaced in review of [the session-persistence work](../implemented/2026-06-14-session-persistence.md) (#33):
|
||||
|
||||
1. **Persistence treats `event.data` as opaque JSON.** The JSONL/SQLite backends `JSON.stringify`/`JSON.parse` each event verbatim; the only runtime guard is `isJsonValue` (round-trip serializability — rejects BigInt, functions, cycles, non-finite numbers, …), NOT structural validation. A corrupted-but-still-JSON event datum (wrong field types, missing fields) round-trips silently and is only caught later, if at all, by a consumer's `switch`.
|
||||
2. **No runtime contract for plugin-added variants.** A plugin that declaration-merges a new `SessionEventMap` key gets compile-time typing for its own code, but nothing validates that the values it produces match the shape it declared — at the producer, at the persistence boundary, or on reload.
|
||||
|
||||
A reviewer asked whether the project should move "all the JSON serialization/deserialization" — and ultimately the event vocabulary itself — to **Zod** (or a similar runtime-schema library), so the durable boundary and the plugin extension points are backed by runtime schemas rather than erased types.
|
||||
|
||||
This RFC scopes that question. It does **not** propose an implementation; it records the tradeoff so the decision is made deliberately rather than incrementally inside a persistence PR.
|
||||
|
||||
## Why this is not a persistence change
|
||||
|
||||
It is tempting to read "use Zod for serialization" as a local change to `dsh-session-persistence-jsonl/src/format.ts`. It is not, for one structural reason: **a plugin cannot declaration-merge a Zod schema.** Declaration merging is a TypeScript compile-time mechanism; a Zod schema is a runtime value. To validate events with Zod you need a **runtime registry** that every event-producing package contributes its schema to (e.g. `ctx.sessionEvents.register('compaction/marker', z.object({…}))`), and every consumer reads from. That registry — not the persistence backend — becomes the source of truth for the vocabulary, replacing the merge-extensible interface.
|
||||
|
||||
So the real proposal is: **replace the compile-time merge-extensible-map pattern with a runtime schema registry, repo-wide.** That is a core-vocabulary redesign.
|
||||
|
||||
## Blast radius (measured)
|
||||
|
||||
A migration of the event/vocabulary surface to runtime schemas touches, at minimum:
|
||||
|
||||
- **Six merge-extensible maps** (~370 LOC of core types): `ContentBlockMap`, `MessageSourceMap`, `FinishReasonMap` (in `dsh-llm`); `TurnTriggerMap`, `TurnEndReasonMap`, `SessionEventMap` (in `dsh-session`).
|
||||
- **~10 `declare module` augmentation sites** across `dsh-agent`, `dsh-agent-loop`, `dsh-bash`, `dsh-llm`, `dsh-session`, `dsh-session-persistence`, `dsh-system-prompt`, `dsh-tools` — each would move from declaration merging to a runtime `register()` call.
|
||||
- **The event producers** — 16 `session.append(...)` call sites in the loop — unchanged in shape but now validated at the boundary.
|
||||
- **~7 switch-consumers** that branch on these unions: `deriveMessages` (`dsh-session`), `BlockAssembler` (`dsh-llm`), the `dsh-invariants` plugin, both LLM adapters (`dsh-llm-deepseek`, `dsh-llm-pi-ai`), and the tool schema layer (`dsh-tools`). The `assertNever`-on-closed-unions vs fall-through-on-extensible-unions convention (a documented lint rule) would need rethinking — runtime variants are not statically exhaustive.
|
||||
- **The `defineTool` `InferArgs` DSL** (`dsh-tools`), which derives zero-cast `execute` arg types from a compile-time schema spec — the showcase of the current approach.
|
||||
- **Docs**: architecture.md (the pattern is described as foundational), [dev-mode invariants](../implemented/2026-06-11-dev-invariants-over-deep-readonly.md), and any RFC that references the pattern.
|
||||
|
||||
This is a HUGE change. It is not in scope for the RFC-009 session-persistence work and must not be smuggled in through it.
|
||||
|
||||
## Options
|
||||
|
||||
### A. Status quo — merge-extensible types + `isJsonValue` at the durable boundary
|
||||
Keep the compile-time pattern. Persistence stays opaque-JSON + serializability guard. Plugins extend via declaration merging; correctness of event *shape* is the producer's responsibility, enforced by TypeScript at compile time and by the `dsh-invariants` plugin's structural checks in dev.
|
||||
|
||||
- **Pros**: zero churn; plugin extension is a one-line `interface` augmentation with full type inference and no runtime registration ceremony; no new runtime dependency; the `defineTool` DSL and `assertNever` exhaustiveness keep working.
|
||||
- **Cons**: no runtime structural validation at the persistence boundary or at plugin seams; a malformed-but-JSON datum is caught late.
|
||||
|
||||
### B. Header/closed-shape validation only (schemastery), events stay opaque
|
||||
Tighten only the genuinely-closed shapes that already have hand-rolled type guards — e.g. the JSONL `HeaderLine` guard (`isHeaderLine`) — using **schemastery** (the repo's existing schema library, already used for every plugin `static Config`). Leave the merge-extensible event union as-is.
|
||||
|
||||
- **Pros**: small, fits the existing convention (schemastery, not a new lib); replaces hand-rolled guards on closed shapes with declarative schemas; no core redesign.
|
||||
- **Cons**: does not address event-data validation (the thing the reviewer actually asked about); only helps the fixed metadata records.
|
||||
|
||||
### C. Runtime schema registry for the whole vocabulary (Zod or schemastery)
|
||||
Replace the merge-extensible maps with a runtime registry the producers contribute to and the persistence/consumer paths validate against.
|
||||
|
||||
- **Pros**: real runtime validation at the durable boundary and at plugin seams; one source of truth; enables generic tooling (auto-generated docs, fuzzing, wire-format checks).
|
||||
- **Cons**: the full blast radius above; **Zod is not currently a direct dependency** (only a transitive dep of `@earendil-works/pi-ai`) and the repo's chosen schema lib is **schemastery** — adopting Zod broadly is itself a dependency decision; declaration-merge ergonomics (one-line plugin extension, full inference) are replaced by runtime registration + manual type wiring; the `assertNever` exhaustiveness guarantee weakens (runtime variants aren't statically exhaustive).
|
||||
|
||||
## Recommendation
|
||||
|
||||
Defer. Do **not** change #33. If runtime validation is wanted at the durable boundary in the near term, **Option B** (schemastery on the closed header/metadata shapes) is the proportionate step and stays within the existing convention. **Option C** is a genuine architecture decision that should be evaluated on its own merits — including whether the chosen library is Zod or schemastery — and, if accepted, land as its own change with its own RFC, not as a side effect of persistence serialization.
|
||||
|
||||
## Open questions
|
||||
|
||||
- If a registry is adopted, is the library **schemastery** (already in the tree, already the config schema lib) or **Zod** (richer ecosystem, currently only transitive)? Adopting two schema libraries is a cost in itself.
|
||||
- Can a hybrid keep compile-time inference (so `defineTool` and plugin DX survive) while adding an *optional* runtime schema per variant, validated only at the persistence/wire boundary rather than on every in-process append?
|
||||
- Does the `dsh-invariants` plugin already cover enough of the runtime-shape gap in dev that boundary validation is only needed for genuinely untrusted input (reload of an externally-modified log)?
|
||||
@@ -0,0 +1,26 @@
|
||||
# RFC: Agent lifecycle and ownership seams
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
Several ACP and tool-bash limitations are symptoms of the same missing seam: plugins can create or resume agents through `ctx.agents`, but they cannot own and dispose one agent independently, and long-running bash tasks carry no stable owner in the executor itself. ACP currently aborts and awaits agents on disconnect, but cannot unregister just that session's agent; `session/cancel` cannot cancel queued-but-not-yet-started work; and `tool-bash` keeps task ownership in a plugin-local `Map`, so an HMR reload can make an old task look unowned.
|
||||
|
||||
## Proposal
|
||||
|
||||
Add explicit lifecycle ownership to the agent factory and explicit ownership metadata to background tasks.
|
||||
|
||||
1. `ctx.agents.create/resume` should return an `AgentHandle` (or add an adjacent method) that exposes the `Agent` plus an async disposer. The disposer unregisters the agent, aborts queued/running work, and resolves only when the driver loop reaches quiescence.
|
||||
2. Add a queue-aware cancel primitive to the `Agent` interface. It must clear queued work that has not started, abort the current step if one exists, and make `whenIdle()` wait for the post-cancel quiescent state. ACP `session/cancel` and bridge teardown then become honest cancellation, not best-effort pre-step cancellation.
|
||||
3. Move background task ownership into the bash seam. `BashExecSpec` or `BashTask` should carry a stable owner token, preferably the session id rather than the `Agent` object identity. `bash_output`/`bash_kill` then ask the executor for ownership rather than relying on a `tool-bash` instance-local map.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- ACP disconnect/session close leaves no registered agent for that session, even when `session/load` races teardown.
|
||||
- `session/cancel` before a queued prompt starts prevents that prompt from running and cannot batch the next prompt into the cancelled turn.
|
||||
- A `tool-bash` HMR reload does not make an existing background task readable or killable by a different session.
|
||||
- Existing non-ACP demos still work without managing handles explicitly; config-created agents remain owned by the `AgentLoop` plugin fiber.
|
||||
|
||||
## Risks
|
||||
|
||||
This touches public interfaces (`Agent`, `AgentFactory`, and the bash seam), so it should not be smuggled into a local ACP patch. The compatibility trap is preserving the simple synchronous `Agent.send()` ergonomics while adding a robust async lifecycle path for owners that need it.
|
||||
@@ -0,0 +1,24 @@
|
||||
# RFC: Shared persistence write coordinator
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-session-persistence-jsonl` and `dsh-session-persistence-sqlite` intentionally prove the same `SessionPersistence` contract over different storage media, but their write-path orchestration is now duplicated: per-session state, `session/created` adoption, seed-prefix collision checks, write-behind buffers, serialized flush chains, HMR seeding, and dispose drains. That code is correctness-heavy and already receives the same fixes twice.
|
||||
|
||||
## Proposal
|
||||
|
||||
Extract a backend-agnostic coordinator into `dsh-session-persistence`. The coordinator owns live-session adoption, buffering, cursor filtering, per-id serialization, and disposal quiescence. Concrete backends provide small hooks for durable operations: create lazy state, find/load stored prefix, append a contiguous batch, update summary, delete, and list.
|
||||
|
||||
The public `SessionPersistence` service shape can stay the same. The coordinator can be an internal exported helper or protected base class used by first-party backends; third-party backends may still implement the abstract service directly if their write path is different.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- JSONL and SQLite keep passing the existing shared `runPersistenceContract`.
|
||||
- HMR/adoption/collision tests move to a shared coordinator test suite and run once for each backend through hook-driven fixtures.
|
||||
- Backend-specific tests focus on storage mechanics only: JSONL path safety/fsync/sidecar behavior and SQLite schema/WAL/transaction behavior.
|
||||
- A future backend does not need to copy the current `session/event` → buffer → flush orchestration.
|
||||
|
||||
## Risks
|
||||
|
||||
The current duplication is verbose but explicit. A coordinator must not hide storage-specific durability semantics or make unusual backends fight an inheritance hierarchy. Prefer narrow hooks and contract tests over a large framework.
|
||||
Reference in New Issue
Block a user