Merge remote-tracking branch 'origin/master' into scoped-layers-store
# Conflicts: # .agents/notes/proposed/architecture/2026-07-12-scoped-layers-store.i18n.yaml # .agents/notes/proposed/architecture/2026-07-12-scoped-layers-store.md # .agents/notes/proposed/architecture/2026-07-12-scoped-layers-store.zh.md # docs/rfc/INDEX.md
This commit is contained in:
3
.agents/notes/AGENTS.md
Normal file
3
.agents/notes/AGENTS.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# AGENTS.md — Agent Notes
|
||||
|
||||
Agent Notes are effectively RFCs written by agents: durable proposals and decision records that preserve rationale, alternatives, consequences, and verification contracts. Follow the [documentation standard](../../docs/AGENTS.md) and the [Agent Note contract](README.md).
|
||||
111
.agents/notes/README.md
Normal file
111
.agents/notes/README.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# Agent Notes
|
||||
|
||||
One kind of design doc lives here. An **Agent Note** records a decision or proposal that shapes this codebase — the *why* and *what we gave up*, the parts code and docs can't carry. This file is the front door and contract: where Agent Notes live, when to write one, and [the in-file format](#the-file-format).
|
||||
|
||||
## Layout and naming
|
||||
|
||||
Every Agent Note has two axes, both encoded in its **path** — `{lifecycle}/{class}/yyyy-mm-dd-topic-title.md`:
|
||||
|
||||
- **Lifecycle** (the top-level folder) is the Agent Note's status, and an Agent Note moves between folders as that status changes:
|
||||
- **`proposed/`** — proposals reviewed before implementation; not yet built (or only partly).
|
||||
- **`implemented/`** — the decision shipped. The file records what was decided and what was rejected, and is **kept current with what actually shipped**: when the code later moves a file, renames a package, or changes a key/default, the Agent Note is updated in the same change to match (facts only — paths, names, structure — not the decision itself). See [implemented/AGENTS.md](implemented/AGENTS.md).
|
||||
- **`rejected/`** — the proposal was considered and declined. Kept for the record so the rejection isn't re-litigated.
|
||||
- **Class** (the nested folder) is the *kind* of decision — see [Classification](#classification) below.
|
||||
|
||||
The date in the filename is when the topic was **first proposed** (per git history). Cross-references between Agent Notes use relative markdown links (`[topic](../../implemented/architecture/2026-…-….md)`) — never bare prose or numbers — so they are mechanically checkable and survive moves between folders.
|
||||
|
||||
The tree is the inventory: browse its lifecycle/class folders or search the repository. Do not add a centralized `INDEX.md`; the [no-index Agent Note](implemented/process/2026-07-19-remove-generated-agent-note-index.md) owns the rationale.
|
||||
|
||||
## Classification
|
||||
|
||||
Each Agent Note belongs to one path-encoded class from the closed set in `scripts/agent-note-tree.ts`; the classification gate rejects other folders. Adding a class requires updating the canonical set and this section. See the [classification Agent Note](implemented/process/2026-06-20-agent-note-classification.md).
|
||||
|
||||
| Class | What it covers |
|
||||
|---|---|
|
||||
| `feature` | A new user- or model-facing capability. |
|
||||
| `bug-fix` | Corrects a defect or closes a gap a postmortem surfaced. |
|
||||
| `simplification` | Removes code, behavior, or surface area without adding a capability. |
|
||||
| `architecture` | A structural decision about the **shipped source** — how packages relate, what the runtime vocabulary is. |
|
||||
| `process` | Tooling, policy, or workflow **around** the code — gates, the package manager, vendoring — not runtime behavior. |
|
||||
| `testing` | Test infrastructure and strategy. |
|
||||
|
||||
The `architecture` / `process` line: **architecture** is about the source we ship; **process** is the surrounding tooling and workflow. (`refactor` is deliberately absent — it overlaps `simplification`, whose discriminator, "does observable behavior change?", already covers it.)
|
||||
|
||||
## When to write one
|
||||
|
||||
Every non-trivial change MUST add or update at least one Agent Note in the same PR. A change is non-trivial when it alters behavior, architecture, a cross-file or cross-package contract, process or tooling, testing strategy, an on-disk, wire, or configuration format, or another decision a maintainer may reasonably revisit. A proposal for substantial future work starts in `proposed/`; a decision already made starts in `implemented/`. Pick the class folder that matches the decision (see [Classification](#classification)).
|
||||
|
||||
Updating the Agent Note that already owns the decision satisfies the rule; do not create a duplicate. Only a purely mechanical or local edit with no behavioral, contractual, structural, process, or rationale change is exempt. An Agent Note is never edited into a *different decision*: supersede it with a new one and cross-link. Editing an `implemented/` Agent Note to track where its existing decision lives is required, not forbidden; see [implemented/AGENTS.md](implemented/AGENTS.md).
|
||||
|
||||
## The file format
|
||||
|
||||
Every Agent Note follows one in-file format, enforced by `pnpm run verify-agent-note-format` ([scripts/verify-agent-note-format.ts](../../scripts/verify-agent-note-format.ts), part of `doc-sync`); the rationale for the format — and the alternatives it rejected — is [the uniform-format Agent Note](implemented/process/2026-07-05-uniform-agent-note-format.md).
|
||||
|
||||
### The header block
|
||||
|
||||
The first three lines of every Agent Note are exactly:
|
||||
|
||||
```markdown
|
||||
# Agent Note: <title>
|
||||
|
||||
Status: <status>
|
||||
```
|
||||
|
||||
followed by a blank line. The `Status:` value is one of three forms, and must agree with the lifecycle folder the file sits in — the gate cross-checks them:
|
||||
|
||||
- `Status: proposed`
|
||||
- `Status: implemented`
|
||||
- `Status: rejected — <why, in one line>`
|
||||
|
||||
The status carries no dates and no parentheticals: the filename holds the first-proposed date, git holds everything else, and an "accepted in amended form" note is body content (state the amendment where the decision is stated). The rejection reason is the one status with content, because a rejected Agent Note's verdict is the fact readers come for.
|
||||
|
||||
### The body skeleton
|
||||
|
||||
Every Agent Note opens its body with `## Problem` — the motivation, written to stand without the solution. What follows depends on the lifecycle; recurring sections use these canonical names and nothing else, while genuinely bespoke technical sections (package topology, wire contracts, schemas) remain free-form between the required ones.
|
||||
|
||||
#### `proposed/`
|
||||
|
||||
```markdown
|
||||
## Problem
|
||||
## Proposal
|
||||
…bespoke sections…
|
||||
## Alternatives considered
|
||||
## Acceptance criteria
|
||||
## Risks
|
||||
```
|
||||
|
||||
`## Proposal` is the intended change and may legitimately speak in the future tense — plans, migration steps, and open questions belong here while the work is unbuilt. `## Acceptance criteria` says what observable state means done. `## Risks` covers both what could go wrong and what the change knowingly gives up.
|
||||
|
||||
#### `implemented/`
|
||||
|
||||
```markdown
|
||||
## Problem
|
||||
## Decision
|
||||
…bespoke sections…
|
||||
## Alternatives considered
|
||||
## Consequences
|
||||
```
|
||||
|
||||
`## Decision` describes shipped reality in the present tense, and the whole file is kept current with it per [implemented/AGENTS.md](implemented/AGENTS.md). `## Consequences` records what the trade-off cost **and** bought. Proposal-era headings are spec-speak here and the gate rejects them: `## Proposal`, `## Plan`, `## Migration plan`, and `## Acceptance criteria` may not appear in an implemented Agent Note (the [slop checklist](../../docs/AGENTS.md) names why). A `## Testing`, `## Deferred`, or `## Related` section is fine where it states present-tense fact.
|
||||
|
||||
#### `rejected/`
|
||||
|
||||
A rejected Agent Note is the proposal, frozen: it keeps whatever proposal-time sections it had (including `## Acceptance criteria` or `## Plan`), and the verdict lives on the `Status:` line. Only the header block, the `## Problem` opener, a `## Proposal` section, and the Alternatives-considered mandate below apply.
|
||||
|
||||
### Alternatives considered — mandatory
|
||||
|
||||
Every Agent Note carries an `## Alternatives considered` section: each genuine alternative and why it lost, one bold-led paragraph per alternative or a `### Why not <X>?` subsection per contested one. A decision recorded without what it beat invites re-litigation — the failure Agent Notes exist to prevent.
|
||||
|
||||
Alternatives are recorded, never invented. An Agent Note dated before 2026-07-05 whose alternatives are not reconstructible from the record carries this exact comment in place of the section, which the gate accepts for pre-format files only:
|
||||
|
||||
```markdown
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
```
|
||||
|
||||
### Moving between lifecycles
|
||||
|
||||
Moving a file between lifecycle folders means updating the `Status:` line and re-satisfying that folder's skeleton in the same change — the gate fails the move otherwise. Concretely, `proposed/` → `implemented/` rewrites `## Proposal` into a present-tense `## Decision`, folds `## Acceptance criteria` and `## Risks` into `## Consequences` (or a present-tense `## Testing`/`## Verification` section for what now pins the behavior), and drops plans in favor of what shipped — the rewrite [implemented/AGENTS.md](implemented/AGENTS.md) requires, made mechanical. `proposed/` → `rejected/` only adds the reason to the `Status:` line and freezes the file.
|
||||
|
||||
### Chinese counterparts
|
||||
|
||||
A `.zh.md` counterpart mirrors its English sibling's structure section-for-section under the [i18n contract](../../docs/i18n/README.md); the machine-checked header tokens (`# Agent Note: ` and the `Status:` line) stay in English verbatim. The format gate skips `.zh.md` files — the pairing gate owns their consistency.
|
||||
11
.agents/notes/implemented/AGENTS.md
Normal file
11
.agents/notes/implemented/AGENTS.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# AGENTS.md — Implemented Agent Notes
|
||||
|
||||
These Agent Notes describe shipped decisions. Follow the [root instructions](../../../AGENTS.md), [documentation standard](../../../docs/AGENTS.md), and [Agent Note format](../README.md#the-file-format); `verify-agent-note-format` gates the lifecycle-specific structure.
|
||||
|
||||
## Keep an implemented Agent Note current with what actually shipped
|
||||
|
||||
Keep paths, symbols, defaults, and mechanisms current in the same change that alters them. Rewrite stale facts in place; do not append change history.
|
||||
|
||||
### This is not a license to rewrite the *decision*
|
||||
|
||||
Update factual realization in place. A reversal of the decision or its rationale requires a new Agent Note and cross-link; see the [Agent Note contract](../README.md).
|
||||
@@ -0,0 +1,26 @@
|
||||
# Agent Note: Provider-neutral content-block vocabulary owned by dsh-llm
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness needs one internal language for messages that the loop, session log, and all plugins speak.
|
||||
|
||||
## Decision
|
||||
|
||||
Own the vocabulary: messages are arrays of typed content blocks (`text`, `reasoning`, `tool-call`, `tool-result`), with the union derived from the merge-extensible `ContentBlockMap` so plugins add block types via declaration merging. The same merge-extensible-map pattern types every "stringly" field (`MessageSource`, `FinishReason`, `TurnTrigger`, `TurnEndReason`). Streaming is a raw chunk protocol; `BlockAssembler` is the single shared assembly implementation. Adapters translate to provider wire formats — mapping cost lives in adapters, where it belongs.
|
||||
|
||||
In-session context injection (`context/message`) and mid-turn steering (`steering/message`) originally rendered as tagged user-role envelopes (the system-reminder pattern) rather than a new role, so adapters carry zero burden. Both now project as plain user content with no wrapper; see [the injected-content-envelope Agent Note](../simplification/2026-07-20-unwrap-injected-content-envelopes.md). Live-adapter validation confirms this rendering for current DeepSeek behavior; a future provider-specific mismatch belongs in that adapter rather than a new canonical role.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Mirror the DeepSeek/OpenAI chat-completions shape** — zero mapping cost for the first provider, but awkward for rich content (reasoning, tool results as structured blocks).
|
||||
- **Adopt Anthropic's Messages block structure verbatim** — battle-tested, but the canonical types would mirror a third-party API the harness does not target first.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Reasoning has a core home without provider-specific shapes.
|
||||
- Multimodal blocks return only with coordinated adapter, UI, and compaction support; see [the drop-image Agent Note](../simplification/2026-07-04-drop-image-content-block.md).
|
||||
- Cache hints and assistant prefill remain absent until a shipping adapter can honor them; see the [producer-less variants](../simplification/2026-07-04-prune-producerless-vocabulary-variants.md) and [inert request knobs](../simplification/2026-07-04-drop-inert-request-knobs.md) Agent Notes.
|
||||
- Every adapter pays a translation cost; the first real adapters have since validated the streaming protocol, and new adapters should continue proving their provider-specific mapping in adapter-local tests.
|
||||
- IDs that cross package boundaries are branded (`CallId`, the shared agent/session `SessionId`) — nominal typing at zero runtime cost.
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Custom typed tool-schema DSL instead of schemastery
|
||||
# Agent Note: Custom typed tool-schema DSL instead of schemastery
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Tool parameters must reach the model as standard JSON Schema (the wire format), and tool authors deserve typed `execute(args)` without casts. The repo already vendors schemastery (used for plugin Config), so reusing it was the obvious candidate. The user also explicitly preferred per-property `required: true` booleans over JSON Schema's separate `required` array.
|
||||
Tool parameters must reach the model as standard JSON Schema while giving tool authors typed `execute(args)` without casts. Schemastery already serves plugin config, but the tool-author API needs per-property `required: true` booleans rather than JSON Schema's separate `required` array.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -18,4 +18,4 @@ A small custom DSL in dsh-tools: `SchemaSpec` (per-property specs with `required
|
||||
|
||||
- First-party tool authors get zero-cast typed args; the type gymnastics cost stays inside the core package (sanctioned by the AGENTS.md type-safety policy).
|
||||
- The DSL is deliberately small (string/number/boolean/object/array, enum, default, nested properties/items). Gaps vs full JSON Schema (unions, formats, constraints) are accepted until real tools demand them.
|
||||
- The InferArgs mapping is regression-tested at the type level (expectTypeOf) after an early optionality bug shipped and was caught by review.
|
||||
- The `InferArgs` mapping is regression-tested at the type level after an early optionality bug.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Source-owned session immutability and dev-mode invariants
|
||||
# Agent Note: Source-owned session immutability and dev-mode invariants
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -28,11 +28,11 @@ This guarantee belongs in `Session`, not in an optional listener, because every
|
||||
|
||||
`deriveMessages()` projects logged surface events into detached, deep-frozen `Message` objects and returns a fresh array snapshot. Request assembly can therefore combine derived history with other inputs without exposing a path back into the log. The cache reuses safe immutable projections rather than recloning the complete history for each model call.
|
||||
|
||||
### The invariants plugin checks relationships
|
||||
### Package-owned invariant companions check relationships
|
||||
|
||||
`dsh-invariants` is a pure-listener development plugin. It does not freeze records and has no configuration; disposal removes only its assertions. It checks rules that require trace state or observation of another seam, including monotonic sequence numbers, turn and step nesting, tool-call/result pairing, legal agent-status transitions, subject-correct scoped dispatch, and equality between a loop-built request and the request reconstructed from its session-log prefix.
|
||||
`dsh-invariants` registers the configurable `ctx.invariants` service and contains no product checks. Every package publishes a `./invariant` ownership companion; `dsh-session`, `dsh-agent`, `dsh-scope`, and `dsh-agent-loop` currently add the rules that require trace state or observation of another seam: monotonic sequence numbers, turn and step nesting, tool-call/result pairing, legal agent-status transitions, subject-correct scoped dispatch, and equality between a loop-built request and the request reconstructed from its session-log prefix. Global enablement and package-name regex filters belong to the service ([package-owned invariant service](2026-07-19-package-owned-invariant-service.md)).
|
||||
|
||||
When the plugin attaches to an existing or seeded session, it replays the immutable log to rebuild trace state. This makes hot reload safe in the middle of a turn without giving the plugin ownership of session storage.
|
||||
When the session companion attaches to an existing or seeded session, it replays the immutable log to rebuild trace state. The service gives each contribution a disposable child fiber, so hot reload is safe in the middle of a turn without giving diagnostics ownership of session storage.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -53,6 +53,6 @@ Detaching `deriveMessages()` would protect the most common request path but leav
|
||||
- Every accepted live or seeded session event is detached from caller-owned inputs and deeply immutable before any observer can receive it.
|
||||
- `session.events` exposes stable immutable snapshots instead of the private growing array.
|
||||
- Request-side mutation cannot reach stored history through derived messages.
|
||||
- Development builds can enable relational assertions without changing storage behavior, and disposing or omitting the plugin does not weaken log immutability.
|
||||
- `dsh-invariants` has no `Config` surface because it has no behavior to tune.
|
||||
- Development builds can enable relational assertions without changing storage behavior, and disposing or filtering a companion does not weaken log immutability.
|
||||
- `dsh-invariants` configures global enablement plus package allow/block regex lists; each check remains owned and tested by its product package.
|
||||
- The runtime boundary carries a recursive snapshot-and-freeze cost once per accepted event; later readers and cached projections reuse the owned immutable records.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Event-sourced sessions with derived message history
|
||||
# Agent Note: Event-sourced sessions with derived message history
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -12,7 +12,7 @@ A `Session` is an append-only log of typed `SessionEvent`s — the single source
|
||||
|
||||
Appends are synchronous (the hot path never blocks on I/O); `session/event` is a sync notification; persistence plugins buffer write-behind and drain at the awaited `session/flush` checkpoint fired at every turn end.
|
||||
|
||||
Ordering contract: the loop appends to the session *before* emitting the corresponding Cordis event, and the `agent/step-result` waterfall runs before the `assistant/message` append so the log records what tool dispatch actually used (post-review fix; regression-tested).
|
||||
Ordering contract: the loop appends to the session *before* emitting the corresponding Cordis event, and the `agent/step-result` waterfall runs before the `assistant/message` append so the log records the message tool dispatch actually used. Regression tests pin that ordering.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Microkernel — extension via Cordis event taxonomy, one concrete loop
|
||||
# Agent Note: Microkernel — extension via Cordis event taxonomy, one concrete loop
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -10,8 +10,8 @@ The product principle is "everything is a plugin": hooks, /goal, /loop, dynamic
|
||||
|
||||
Pure Cordis event taxonomy. The loop's extension seams are typed events with deliberate dispatch modes:
|
||||
|
||||
- **waterfall** (around-middleware) where plugins transform, veto, or wrap: `agent/prompt-submit`, `agent/request`, `agent/step-result`, `agent/turn-continuation`, `tools/pre-execute`, `tools/execute`, `tools/post-execute`, `llm/stream`, `system-prompt/assemble`.
|
||||
- **serial** (awaited in listener order; a bail value stops later listeners) for ordered checkpoints: every `agent/pre-step` listener runs when all abstain, while the first stop returned from `agent/turn-stop` makes the terminal decision final.
|
||||
- **waterfall** (around-middleware) where plugins transform, veto, recover, or wrap: `agent/prompt-submit`, `agent/request`, `agent/request-error`, `agent/step-result`, `agent/turn-continuation`, `tools/pre-execute`, `tools/execute`, `tools/post-execute`, `llm/stream`, `system-prompt/assemble`.
|
||||
- **serial** (awaited in listener order; a bail value stops later listeners) for ordered checkpoints: every `agent/pre-step` and `agent/post-step` listener runs when all abstain, while the first stop returned from `agent/turn-stop` makes the terminal decision final.
|
||||
- **parallel** (awaited fan-out) where every listener must get an independent chance: the `session/flush` durability checkpoint.
|
||||
- **emit** (synchronous fire-and-forget) for notifications: turn/step boundaries, stream chunks, lifecycle, errors, and the contained immutable `tools/result` observation.
|
||||
|
||||
@@ -23,7 +23,7 @@ The event vocabulary lives in interface packages (dsh-agent declares the agent/*
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every MVP feature maps to a listener (the [feature → mechanism map](../../../cookbook/extension-cookbook.md#the-feature--mechanism-map) is the proof obligation, kept current).
|
||||
- Every MVP feature maps to a listener (the [feature → mechanism map](../../../../docs/cookbook/extension-cookbook.md#the-feature--mechanism-map) is the proof obligation, kept current).
|
||||
- HMR and disposal come free: listeners and registrations are Cordis effects.
|
||||
- Waterfall semantics (call `next()` or short-circuit) are non-obvious and must be taught — documented in AGENTS.md and covered by composition tests.
|
||||
- The loop must be defensive: plugin exceptions are contained at turn level, steering from any seam is never stranded (regression-tested).
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Runtime arg validation at the model boundary
|
||||
# Agent Note: Runtime arg validation at the model boundary
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -19,4 +19,4 @@ The validator mirrors `schemaSpecToJsonSchema` semantics exactly — same struct
|
||||
- `ToolArgsError` is a plain `Error` with a `code` field for now; if a harness-wide error taxonomy lands it becomes a subclass without changing callers that read `.message`.
|
||||
- Validation cost is negligible next to a model call.
|
||||
|
||||
<!-- rfc-format: alternatives-not-recorded (pre-format RFC) -->
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Structured error taxonomy
|
||||
# Agent Note: Structured error taxonomy
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -6,13 +6,11 @@ Status: implemented
|
||||
|
||||
Failures crossed seams as bare strings. A tool error flattened to a text block — name, code, and stack lost — so a future sandbox/retry plugin couldn't tell ENOENT from EACCES, and the model got less actionable feedback than it could. A non-Error throw degraded further: the loop wrapped it in `new Error(String(x))`, dropping any code. And `LlmError` was the only typed error in the system, with no shared base, so there was nothing for a consumer to `instanceof` against generically.
|
||||
|
||||
This is the last of the runtime-validation / error-taxonomy pieces and the one the user was most skeptical of, so it was deliberately built **last and in isolation**: the earlier PRs (arg validation, dev invariants) threw plain `Error`s with a `code` field, decoupled from any shared base, so this change is a pure upgrade and is independently revertible without unpicking them.
|
||||
|
||||
## Decision
|
||||
|
||||
A single `HarnessError extends Error` base in `dsh-llm` (the leaf package every other imports — no new dependency edge): a stable `code` distinct from `message`, `cause` chaining via `ErrorOptions`, and `name` defaulting to the subclass. `isHarnessError` narrows at seams.
|
||||
|
||||
- `LlmError`, `ToolArgsError` (dsh-tools), and `InvariantError` (dsh-invariants) now extend it, keeping their existing codes.
|
||||
- `LlmError` and `ToolArgsError` (dsh-tools) extend it, keeping their existing codes.
|
||||
- `ToolExecutionResult` gains optional `error: { name, code }`, populated in the registry's catch when the thrown value is a `HarnessError`. The agent loop forwards it onto the `tool/result` session event (which gained the same optional field), so the structured failure survives into the log for retry/sandbox plugins and replay. The model-facing text block is unchanged.
|
||||
- The loop's `toError` wraps a non-Error throw in a `HarnessError` (`code: 'UNKNOWN'`, original chained as `cause`) instead of a bare `Error`, so even a bad throw carries a routable code into the session `error` event (which already surfaced `code`).
|
||||
|
||||
@@ -21,6 +19,6 @@ A single `HarnessError extends Error` base in `dsh-llm` (the leaf package every
|
||||
- Errors are machine-routable end-to-end: a plugin can branch on `error.code` rather than substring-matching a message.
|
||||
- One base class is imported widely, but it lives in the package everyone already depends on, so the cost is a single import, not a new edge.
|
||||
- `deriveMessages` does not surface `error` into model history — the model still sees the text block; the structured field is for code and replay.
|
||||
- Reverting this PR returns the earlier errors to plain `Error`+`code` form; nothing else in the stack depends on the shared base.
|
||||
- Argument validation retains its existing code and behavior; package-owned diagnostic invariants carry their stable code independently so the invariant registry does not import a product package. The shared base adds cross-seam routing metadata without changing model-facing text.
|
||||
|
||||
<!-- rfc-format: alternatives-not-recorded (pre-format RFC) -->
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Tool schemas are part of the system-prompt assembly
|
||||
# Agent Note: Tool schemas are part of the system-prompt assembly
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Capability seams — interface / implementation / consumer split
|
||||
# Agent Note: Capability seams — interface / implementation / consumer split
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -6,15 +6,15 @@ Status: implemented
|
||||
|
||||
The harness has swappable capabilities — bash execution today, sandboxed/remote executors and alternative model providers tomorrow. A capability has three concerns that change at different rates and for different reasons: the *contract* (what the capability is), the *implementation* (how it runs), and the *consumer surface* (what the model and other plugins program against). Bundling them in one package couples those rates of change — swapping a local executor for a sandboxed one would churn the tool schemas the model sees, even though the model-facing contract never changed.
|
||||
|
||||
This is distinct from "who provides vs. needs a capability at runtime", which Cordis already answers with services + `inject` (a provider registers `ctx.bash`; a consumer declares `inject: ['bash']` and its fiber pends until the service exists). That mechanism is necessary but doesn't dictate package boundaries; this RFC does.
|
||||
This is distinct from "who provides vs. needs a capability at runtime", which Cordis already answers with services + `inject` (a provider registers `ctx.bash`; a consumer declares `inject: ['bash']` and its fiber pends until the service exists). That mechanism is necessary but doesn't dictate package boundaries; this Agent Note does.
|
||||
|
||||
## Decision
|
||||
|
||||
A swappable capability is **three packages**:
|
||||
|
||||
1. **Interface** — an abstract service + the vocabulary types, owning the `ctx.<key>` and depending only on cordis (e.g. `dsh-bash`: `BashExecutor`, `BashRunResult`, `BashTask`).
|
||||
1. **Interface** — an abstract service + the vocabulary types, owning the `ctx.<key>` and depending only on its vocabulary dependencies (e.g. `dsh-bash`: `BashExecutor`, `BashRunResult`, `BashProcess`).
|
||||
2. **Implementation** — a concrete subclass loaded as a plugin (e.g. `dsh-bash-local`: subprocesses, process-group kills, spill-file truncation). Sandboxed/remote backends are sibling packages implementing the same interface.
|
||||
3. **Consumer** — what the model and plugins see (e.g. `dsh-tool-bash`: the `bash`/`bash_output`/`bash_kill` tool schemas). Consumers `inject` the interface key and never import implementation types.
|
||||
3. **Consumer** — what the model and plugins see (e.g. `dsh-tool-bash`: the `bash` schema, with background handles registered into the generic task runtime). Consumers `inject` the interface key and never import implementation types.
|
||||
|
||||
Implementation and consumer then evolve independently: a sandboxed executor replaces `dsh-bash-local` without touching a tool schema.
|
||||
|
||||
@@ -23,8 +23,8 @@ The split is not mandatory when the parts are genuinely one concern: the LLM sea
|
||||
## Alternatives considered
|
||||
|
||||
- **One combined package** — rejected because it recouples the three rates of change the split exists to separate (the whole point).
|
||||
- **`@cordisjs/plugin-capability`** — a different axis entirely: it is a permission/capability-*security* service (named permissions with inheritance, tested against a session via `ctx.capability.test`), a candidate for the deferred permissions/sandbox work on the `tools/pre-execute` deny/ask seam, NOT a mechanism for swapping implementations. Confusing the two ("capability") is the trap this RFC names.
|
||||
- **`@cordisjs/plugin-capability`** — a different axis entirely: it is a permission/capability-*security* service (named permissions with inheritance, tested against a session via `ctx.capability.test`), a candidate for the deferred permissions/sandbox work on the `tools/pre-execute` deny/ask seam, NOT a mechanism for swapping implementations. Confusing the two ("capability") is the trap this Agent Note names.
|
||||
|
||||
## Consequences
|
||||
|
||||
More packages and more boilerplate per capability (a `package.json`/`tsconfig`/README trio, the inject wiring). Bought: implementations and consumers ship and version independently, and a new backend never risks the model-facing contract. The rule is documented in [AGENTS.md](../../../../AGENTS.md) § Conventions ("Capability seams are three packages") and [architecture.md](../../../architecture.md) § "Capability seams"; the bash trio is the reference template. When to fold vs. split is a judgment call the architecture doc spells out — this RFC records *why* the default is to split.
|
||||
More packages and more boilerplate per capability (a `package.json`/`tsconfig`/README trio, the inject wiring). Bought: implementations and consumers ship and version independently, and a new backend never risks the model-facing contract. The rule is documented in [AGENTS.md](../../../../AGENTS.md) § Conventions ("Capability seams are three packages") and [architecture.md](../../../../docs/architecture.md) § "Capability seams"; the bash trio is the reference template. When to fold vs. split is a judgment call the architecture doc spells out — this Agent Note records *why* the default is to split.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Two LLM adapters as a design-verification twin
|
||||
# Agent Note: Two LLM adapters as a design-verification twin
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -22,4 +22,4 @@ The rule they enforce: **anything the StreamChunk vocabulary cannot express for
|
||||
|
||||
## Consequences
|
||||
|
||||
Double the adapter maintenance and double the key-gated e2e surface (both adapters cover V4 Flash and Pro across representative thinking/effort modes). Bought: a continuously-verified neutrality guarantee for the most leak-prone abstraction in the codebase, and a worked second example for adapter authors. The two share the core Config shape (`apiKey`/`baseURL`/`models`) so a deployment swaps mostly one line, but the reasoning knob differs — `dsh-llm-deepseek` takes `thinking`/`reasoningEffort`, `dsh-llm-pi-ai` takes a single `reasoning` level — so a swap translates that field. If the maintenance cost ever outweighs the verification value (e.g. once conformance tests from [architectural conformance](../../proposed/process/2026-06-11-architectural-conformance.md) cover the contract mechanically), retiring the twin to a single adapter + the conformance kit would be a new RFC superseding this one.
|
||||
The twin doubles adapter and key-gated e2e maintenance—both cover V4 Flash and Pro across representative reasoning modes—in exchange for continuous seam-neutrality validation and a second implementation example. Both use `apiKey`, `baseURL`, and `models`; the hand-rolled adapter exposes `thinking`/`reasoningEffort`, while pi-ai exposes one `reasoning` level. A future conformance suite could justify retiring one adapter through a superseding Agent Note.
|
||||
@@ -1,12 +1,10 @@
|
||||
# RFC: Session persistence as an abstract service over the existing `SessionEvent`
|
||||
# Agent Note: Session persistence as an abstract service over the existing `SessionEvent`
|
||||
|
||||
Status: implemented
|
||||
|
||||
> Merges the original proposal and the decision record for one topic. The proposal's full method-surface and write-path detail lives in git history; this records the decision and the durable, contested choices.
|
||||
|
||||
## Problem
|
||||
|
||||
Sessions lived only in memory. The example `session-jsonl.ts` plugin (duplicated byte-for-byte in both examples) was write-only telemetry: it buffered `session/event` and appended JSON lines, with no read/replay path, no crash-safety (no fsync, no atomic write, a fire-and-forget dispose drain), no listing, and no format versioning. Nothing could rehydrate a past session from disk into a live agent, so durable resume ("continue yesterday's task"), durable forking, and the ACP `session/load` method ([ACP support](../../implemented/feature/2026-06-14-acp-agent-client-protocol.md)) were all impossible.
|
||||
Sessions lived only in memory. The example `session-jsonl.ts` plugin (duplicated byte-for-byte in both examples) was write-only telemetry: it buffered `session/event` and appended JSON lines, with no read/replay path, no crash-safety (no fsync, no atomic write, a fire-and-forget dispose drain), no listing, and no format versioning. Nothing could rehydrate a past session from disk into a live agent, so durable resume ("continue yesterday's task"), durable forking, and the ACP `session/load` method ([ACP support](../feature/2026-06-14-acp-agent-client-protocol.md)) were all impossible.
|
||||
|
||||
The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append-only log the single source of truth and derives LLM history from it. Persistence had to stay faithful to that: persist the existing `SessionEvent` directly, with no parallel "persisted message" type that the log is converted to and from. The backend also had to be swappable — a file store now, a database store later — behind one interface.
|
||||
|
||||
@@ -15,15 +13,15 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append
|
||||
Persistence is an abstract **capability seam** ([capability seams](2026-06-13-capability-seams.md), the `dsh-bash` template), not loop or core logic:
|
||||
|
||||
1. **Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`append`/`load`/`list`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
|
||||
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**).
|
||||
2. **Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session (a `SessionHeader` line then one `SessionEvent` per line, verbatim **including `assistant/chunk`**), encoded as [checksummed Zstandard frames by default](2026-07-19-zstandard-jsonl-session-logs.md) or raw lines by configuration.
|
||||
|
||||
Key choices recorded here because they are durable, contested, and surprising:
|
||||
|
||||
- **The canonical durable log persists every `SessionEvent` verbatim, including `assistant/chunk`.** `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and the load-validation `events[i].seq === i` require a *contiguous* log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
|
||||
- **Append-only; a crashed turn is closed, never truncated.** Committed events — those at or below a flushed `turn/end` — are never rewritten. The loop only flushes at `turn/end`, so a crash can leave a durable log whose final turn never closed: real, fully-written events sit after the last `turn/end`. **A single turn can be huge in a long-horizon task** (many steps, large tool output spanning a long autonomous run), so discarding the interrupted turn would silently destroy a large amount of real work — truncating a turn is wrong. Instead, on reload `load` PRESERVES those events and CLOSES the orphaned turn by durably appending the minimal synthetic boundary events: an error `tool/result` for every `tool-call` the crash left unanswered, then a `step/end` if a step was still open, then a `turn/end` carrying the merge-extensible `{ kind: 'interrupted' }` reason (a marker that records the turn was cut short by a crash, not completed by the model — no loop ever emits it). The synthetic tool results matter for resume correctness: the loop logs the `assistant/message` (carrying the `tool-call` blocks) BEFORE running the tools, so a crash mid-tool leaves calls without results; `deriveMessages()` would then replay a dangling assistant tool-call, which every provider rejects as an invalid transcript on the next request. Answering each orphaned call with an error result keeps the rehydrated history valid. `load` returns the balanced log, so a resumed session is immediately usable. The ONLY thing discarded is a never-fully-written **torn tail fragment** — a final record whose bytes (JSONL) or row were never completely flushed; that fragment is not a valid event and is dropped before the synthetic closers are written. A parse error or `seq` gap in the COMMITTED region (at or before the last real `turn/end`) is genuine corruption and makes the session unloadable.
|
||||
- **Append-only; a crashed turn is closed, never truncated.** Events through a flushed `turn/end` are never rewritten, and the loop flushes only at turn end. Because one interrupted turn may contain substantial valid work, `load` preserves its contiguous, parseable events and appends error results for unanswered tool calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }`. The synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable.
|
||||
- **File backend canonical, DB backend a proven drop-in.** `SessionEvent` maps 1:1 onto a row `(session_id, seq, type, time, data)` — `append` is INSERT (in a transaction asserting the contiguous-seq contract), `load` is SELECT … ORDER BY seq. `dsh-session-persistence-sqlite` is exactly this: a `SessionPersistence` subclass with no interface change (opencode runs this exact shape on SQLite/WAL), and it passes the same `runPersistenceContract` suite as the JSONL backend — so the contract holds both backends to identical semantics (lazy materialization, interrupted-turn close on load, contiguous-seq), expressed once over file bytes and once over rows.
|
||||
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header seam is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).)
|
||||
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and starts a fresh agent on the resumed id (NOT `${agentId}-session`). The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
|
||||
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` awaits `ctx.sessionPersistence.load`, recreates the live session with the loaded events (so `lastTurnNumber`/`deriveMessages` continue), and registers the fresh agent under the exact resumed id. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -33,4 +31,4 @@ Format versioning: the header carries a `version`; `load` rejects any non-curren
|
||||
|
||||
## Consequences
|
||||
|
||||
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and the foundation the ACP `session/load` ([ACP support](../../implemented/feature/2026-06-14-acp-agent-client-protocol.md)) needs — all over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only / contiguous-seq / lazy-materialization / serializability semantics. This completes [event-sourced sessions](2026-06-11-event-sourced-sessions.md)'s deferred "real persistence backend" and resolves its `TODO(review)` on the event vocabulary: persisting the log freezes its shape, and the `assistant/chunk` fidelity question is answered above (persist verbatim).
|
||||
Two new packages and the metadata seam in `dsh-session` (`session.header`, the `create(id?, options?)` signature). Bought: durable resume/fork, a read/replay path, crash tolerance, and the foundation the ACP `session/load` ([ACP support](../feature/2026-06-14-acp-agent-client-protocol.md)) needs — all over the existing event-sourced log, with the backend swappable behind one interface. The reusable `runPersistenceContract` suite holds every backend to the same append-only, contiguous-seq, lazy-materialization, and serializability semantics. Persisting the full log also settles event fidelity: `assistant/chunk` remains verbatim.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Every session event is enclosed in a turn
|
||||
# Agent Note: Every session event is enclosed in a turn
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -18,10 +18,10 @@ In case 2, if the injected `context/message` is the last event before a flush/di
|
||||
**Every session event lives inside a turn** — between a `turn/start` and its matching `turn/end`. Concretely:
|
||||
|
||||
- The loop appends queued `user/message` events **after** `turn/start` (inside the turn), not before it. `turn/end` is therefore owed the moment those messages are recorded, and the existing finalizer guarantees it.
|
||||
- An `agent.inject()` made while the agent is **running** appends its `context/message` into the already-open turn (unchanged).
|
||||
- An `agent.inject()` made while the agent is **running** joins the already-open turn. While the current step executes assistant tool calls, accepted context waits in arrival order until that batch settles, then appends after every recorded result and before the turn closes even when execution is interrupted.
|
||||
- An `agent.inject()` made while **idle** wraps its `context/message` in a one-shot turn: `turn/start{trigger:{kind:'injection'}}` → `context/message` → `turn/end{completed}`. A new `injection` variant joins the merge-extensible `TurnTriggerMap`.
|
||||
- The loop derives the next turn number from the log each iteration (`lastTurnNumber(session) + 1`) instead of keeping a private counter, so an idle injection's one-shot turn cannot collide with the next real turn's number.
|
||||
- The `dsh-invariants` plugin **enforces** the invariant in dev: a `user/message` / `context/message` / `steering/message` appended while no turn is open throws an `InvariantError`.
|
||||
- The `dsh-session/invariant` companion registers the check with `ctx.invariants`: when selected, a `user/message` / `context/message` / `steering/message` appended while no turn is open throws an `InvariantError` attributed to `@deepseek-ai/dsh-session`.
|
||||
|
||||
The serializability invariant is enforced at the same source boundary (`Session.append` throws on non-JSON-serializable data), so "what may enter the log" is now governed in one place rather than discovered downstream by whichever backend happens to be watching.
|
||||
|
||||
@@ -37,4 +37,4 @@ Costs: `agent.inject()` while idle now writes three log lines instead of one, an
|
||||
|
||||
The rule is intentionally producer-enforced and dev-checked rather than reader-tolerated: a future backend (SQLite/WAL) inherits the same clean boundary for free, and a plugin that records an event outside a turn fails loudly in dev instead of silently losing data on the next reload.
|
||||
|
||||
The invariant also constrains where the loop may record an `error` event. A failure detected while a turn is open is appended INSIDE the turn (before `turn/end`); but a failure that surfaces once the turn is already closed — a rejecting `session/flush`, which runs as the post-`turn/end` durability checkpoint — has no in-turn position left. Appending an `error` there would land it past the last `turn/end`, exactly the crash-tail position a backend discards. So that post-turn failure is reported via the `agent/error` event and the logger only, never as a `SessionEvent`; the turn stays balanced and persistence keeps its buffered events for the next checkpoint. If durable operational diagnostics are ever needed, they belong on a separate telemetry channel, not the replayable session log.
|
||||
Failures detected during a turn are logged before `turn/end`. A later flush failure has no valid in-turn position, so it is reported through `agent/error` and logging rather than appended as a session event. This preserves a balanced replay log; durable operational diagnostics require a separate telemetry channel.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Filesystem capability seam — ctx.fs, local backend, and model-facing filesystem tools
|
||||
# Agent Note: Filesystem capability seam — ctx.fs, local backend, and model-facing filesystem tools
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -18,7 +18,7 @@ We need the filesystem tools to land in the same capability-seam shape as bash b
|
||||
|
||||
## Decision
|
||||
|
||||
Filesystem access is a first-class capability seam following [the capability-seam RFC](../../implemented/architecture/2026-06-13-capability-seams.md):
|
||||
Filesystem access is a first-class capability seam following [the capability-seam Agent Note](2026-06-13-capability-seams.md):
|
||||
|
||||
1. `@deepseek-ai/dsh-fs` (`packages/fs/fs`) owns the abstract `ctx.fs` service, the filesystem vocabulary types, and the `fs/*` policy event vocabulary.
|
||||
2. `@deepseek-ai/dsh-fs-local` (`packages/fs/fs-local`) provides the first implementation, backed by the local filesystem.
|
||||
@@ -26,7 +26,7 @@ Filesystem access is a first-class capability seam following [the capability-sea
|
||||
|
||||
The consumer package depends only on the interface package, never on `dsh-fs-local`. A deployment that wants a different backend loads a different provider for `ctx.fs` without changing the tool schemas or model-facing prompt guidance.
|
||||
|
||||
The read-before-write/edit and observed-state policy is a fourth package, `@deepseek-ai/dsh-fs-policy` (`packages/fs/fs-policy`), contributed through the `fs/*` event gate rather than living on `ctx.fs`; a deployment loading `dsh-tool-fs` also loads `dsh-fs-policy` to get read-before-write/edit. This RFC established the three-package seam; the split of policy off the provider base class is decided by [the split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md), and its realization as an event-gate plugin (not a method service) by [the event-gate RFC](2026-06-26-file-context-as-event-gate.md). This document is updated to describe that landed four-package shape.
|
||||
The read-before-write/edit and observed-state policy is a fourth package, `@deepseek-ai/dsh-fs-policy` (`packages/fs/fs-policy`), contributed through the `fs/*` event gate rather than living on `ctx.fs`; a deployment loading `dsh-tool-fs` also loads `dsh-fs-policy` to get read-before-write/edit. This Agent Note established the three-package seam; the split of policy off the provider base class is decided by [the split-fs-seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md), and its realization as an event-gate plugin (not a method service) by [the event-gate Agent Note](2026-06-26-file-context-as-event-gate.md). This document is updated to describe that landed four-package shape.
|
||||
|
||||
The first backend is deliberately local-only: `dsh-fs-local` implements `ctx.fs` against the host filesystem. Future sibling backends can provide sandboxed, remote, virtual, or project-scoped filesystems behind the same interface.
|
||||
|
||||
@@ -34,7 +34,7 @@ The first consumer is deliberately text-file-only: `dsh-tool-fs` exposes model-f
|
||||
|
||||
Filesystem permissions and sandboxing are not implied by this split. The local backend resolves relative paths from its configured base directory, but containment policy is a separate decision: either a stricter `ctx.fs` implementation enforces it, or a permission/sandbox plugin wraps `tools/execute` and vetoes calls before they reach the consumer.
|
||||
|
||||
Read-before-write/edit and observed-state are policy, contributed by the `dsh-fs-policy` plugin through the `fs/*` event gate — NOT stored on `ctx.fs`. The provider seam offers an optional version guard on its mutations (`writeText`/`editText` take an optional expectation); the policy plugin decides that guard by listening on `fs/write-intent`/`fs/edit-intent` and records observed versions on `fs/observed`. The executor (`dsh-tool-fs`) passes the current tool execution context as the opaque event actor; the policy plugin derives the observed-state owner from it, normally `exec.agent.session`. `dsh-fs` treats the actor as opaque and never reads it; `dsh-tool-fs` never reaches into the policy plugin. Authorization is version freshness: any read records the file's version, and a later write/edit is authorized as long as the file is unchanged. (This RFC first placed the observed-state store on `ctx.fs`; the split to `dsh-fs-policy` on the `fs/*` event gate is decided by [the split-fs-seam](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [event-gate](2026-06-26-file-context-as-event-gate.md) RFCs.)
|
||||
Read-before-write/edit and observed state belong to `dsh-fs-policy`, not `ctx.fs`. Through the `fs/*` event gate, the policy records versions per opaque actor and supplies optional mutation expectations; the provider enforces freshness atomically. `dsh-tool-fs` emits the events without depending on the policy. See the [split-seam](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [event-gate](2026-06-26-file-context-as-event-gate.md) Agent Notes.
|
||||
|
||||
## Package topology
|
||||
|
||||
@@ -71,7 +71,7 @@ The provider seam also carries the freshness hooks that policy builds on — but
|
||||
- `writeText`/`editText` take an OPTIONAL version expectation: omit it for an unconditional bare-provider mutation, or supply it to guard the mutation inside the backend's atomic critical section.
|
||||
- The `dsh-fs-policy` plugin decides that expectation on `fs/write-intent`/`fs/edit-intent` and records observed versions on `fs/observed`, keyed by an owner it derives from the opaque event actor (normally `exec.agent.session`).
|
||||
|
||||
Authorization is version freshness, not a full/partial view distinction: any read records the target's version, and a later write/edit is authorized as long as the file is still at that version — so a windowed read of lines 100-150 authorizes an edit of line 120. The observed-state store is a `WeakMap<owner, Map<targetKey, version>>` inside `dsh-fs-policy`; `dsh-fs` holds none of it and treats the actor as opaque. (This RFC first modeled a `FileState` cache with `full`/`partial` views on `ctx.fs`; the split-fs-seam and event-gate RFCs replaced that with the freshness-based policy plugin described here.)
|
||||
Authorization is version freshness, not a full/partial view distinction: any read records the target's version, and a later write/edit is authorized as long as the file is still at that version — so a windowed read of lines 100-150 authorizes an edit of line 120. The observed-state store is a `WeakMap<owner, Map<targetKey, version>>` inside `dsh-fs-policy`; `dsh-fs` holds none of it and treats the actor as opaque. (This Agent Note first modeled a `FileState` cache with `full`/`partial` views on `ctx.fs`; the split-fs-seam and event-gate Agent Notes replaced that with the freshness-based policy plugin described here.)
|
||||
|
||||
Path resolution is explicit and allowed to be async. Local resolution may only normalize a path, but sandboxed/remote/project-scoped backends may need I/O to resolve a user-supplied path into a stable target identity.
|
||||
|
||||
@@ -81,7 +81,7 @@ Resolved targets must expose at least three concepts:
|
||||
- An opaque `targetKey`, used for stale guards and file-state lookup. The local backend might use a realpath-like key; a remote backend might use a workspace URI or file id. Consumers must not parse or assume this is a local absolute path.
|
||||
- A `displayPath`, used for model/UI-facing output. It may be a local absolute path, workspace-relative path, or remote URI depending on the backend.
|
||||
|
||||
Read and mutation results must include an opaque file `version`. A local backend can use mtime/size or a hash-like token; a remote backend can use a revision id. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token.
|
||||
Read and mutation results must include an opaque file `version`. The local backend derives its token from bigint stat metadata (`dev`, `ino`, `size`, `mtimeNs`, and `ctimeNs`) so same-size rewrites and inode replacement invalidate consumers reliably; a remote backend can use a revision id or hash-like token. The `dsh-fs-policy` plugin records versions for stale checks; consumers may display related metadata but must not interpret the version token.
|
||||
|
||||
The provider hands back decoded text: `readText` returns a whole regular text file, `streamText` streams the same text semantics for large files. Both own regular-file checks, bounded line/output handling is NOT theirs — line windowing, numbered-line rendering, and total-line accounting live in the executor (`dsh-tool-fs`), which reads through `ctx.fs` and renders the model-facing window. The provider owns UTF-8 decoding and binary/NUL rejection; it does not know about line windows or views.
|
||||
|
||||
@@ -135,7 +135,7 @@ The defensive-pattern classes this repo has been bitten by are pinned directly:
|
||||
|
||||
- **Model-facing tools directly over `node:fs`** — the tool package would own execution policy, path resolution, atomic writes, text decoding, and edit semantics at once, coupling the three independently-changing concerns the Problem names and churning schemas on any backend swap.
|
||||
- **One combined `dsh-fs-tools` package** — the pre-seam shape; rejected for the same interface/implementation/consumer split as bash, and the combined name never became public surface.
|
||||
- **Observed-state on `ctx.fs`** — the shape this RFC first landed; superseded by [the split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [the event-gate RFC](2026-06-26-file-context-as-event-gate.md): a sandboxed/remote backend must not inherit model-facing observation policy, so the provider keeps only the version token and the optional version-guarded mutation.
|
||||
- **Observed-state on `ctx.fs`** — the shape this Agent Note first landed; superseded by [the split-fs-seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [the event-gate Agent Note](2026-06-26-file-context-as-event-gate.md): a sandboxed/remote backend must not inherit model-facing observation policy, so the provider keeps only the version token and the optional version-guarded mutation.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -143,11 +143,11 @@ The defensive-pattern classes this repo has been bitten by are pinned directly:
|
||||
|
||||
**The interface can become too local.** Returning fields such as `absolutePath` from `ctx.fs` would make remote, sandboxed, or virtual backends awkward. The contract should expose display metadata without requiring consumers to understand host paths.
|
||||
|
||||
**The interface can become too thin.** If `ctx.fs` only mirrors `node:fs` primitives, `tool-fs` will reimplement binary detection, pagination, atomic writes, and edit semantics. That recreates the coupling this RFC is trying to avoid.
|
||||
**The interface can become too thin.** If `ctx.fs` only mirrors `node:fs` primitives, `tool-fs` will reimplement binary detection, pagination, atomic writes, and edit semantics. That recreates the coupling this Agent Note is trying to avoid.
|
||||
|
||||
**Edit semantics are race-prone by nature.** Literal edit is a read-modify-write operation; the guard is the backend's atomic mutation critical section plus the optional version expectation, so concurrent edits settle deterministically — one wins, the other gets `FS_STALE_VERSION`.
|
||||
|
||||
**Observed state does not belong on `ctx.fs`.** Recording what an execution context has seen is workflow policy, not raw filesystem I/O. This RFC first placed it inside the filesystem seam; the split-fs-seam RFC then established that a sandboxed/remote backend should not inherit model-facing observation policy, and moved it into the `dsh-fs-policy` plugin. The provider seam keeps only what write/edit safety genuinely needs at the storage layer — a backend-minted version token and an optional version-guarded mutation — while the policy plugin owns owner derivation, observed-state, and read-before-edit gating over the `fs/*` events.
|
||||
**Observed state does not belong on `ctx.fs`.** Recording what an execution context has seen is workflow policy, not raw filesystem I/O. This Agent Note first placed it inside the filesystem seam; the split-fs-seam Agent Note then established that a sandboxed/remote backend should not inherit model-facing observation policy, and moved it into the `dsh-fs-policy` plugin. The provider seam keeps only what write/edit safety genuinely needs at the storage layer — a backend-minted version token and an optional version-guarded mutation — while the policy plugin owns owner derivation, observed-state, and read-before-edit gating over the `fs/*` events.
|
||||
|
||||
**The `resolve`-then-operate shape costs an extra round-trip per call.** Each tool may resolve a path to an `FsTarget` and then issue the read/write/edit as a separate `ctx.fs` call. For the local backend this is negligible (resolution is in-memory path normalization), but a remote/sandboxed backend may turn each step into its own request, so a single `read` can become two network round-trips. Backends where the round-trip matters can cache or fold resolution internally while preserving the observable contract.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Agent lifecycle and ownership seams
|
||||
# Agent Note: Agent lifecycle and ownership seams
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -10,9 +10,9 @@ Several ACP and tool-bash limitations were symptoms of the same missing seam: pl
|
||||
|
||||
Three seams: the queue-aware cancel, the `AgentHandle` disposer, and the bash owner token.
|
||||
|
||||
### 1. Queue-aware `Agent.cancel(reason?)`
|
||||
### 1. Queue-aware `Agent.cancel(cause?)`
|
||||
|
||||
A new `cancel()` verb on the `Agent` interface — the single public stop primitive. (It originally shipped alongside a narrower step-only `abort()`; that verb was later removed as unused, leaving `cancel()` the only public way to stop work.) It clears the inbox's queued + steering FIFOs, aborts the in-flight step if any, and drives a **turn-scoped cancellation marker** the driver loop checks at every turn-decision point — so a prompt that is queued-but-not-yet-started never runs, a cancel landing in the pre-step / continuation window drops the about-to-run turn (ending it `aborted`), and a later prompt cannot be batched into the cancelled turn. `whenIdle()` reaches post-cancel quiescence. ACP `session/cancel` maps to `cancel()`. The marker is armed ONLY when there is something to cancel, so an idle no-op cancel cannot strand the next prompt.
|
||||
A new `cancel()` verb on the `Agent` interface — the single public stop primitive. (It originally shipped alongside a narrower step-only `abort()`; that verb was later removed as unused, leaving `cancel()` the only public way to stop work.) It clears the inbox's queued + steering FIFOs, aborts the active turn if any, and keeps a cause-less pre-run marker so a prompt cancelled before claim never runs while a later prompt remains independent. An effective call emits `agent/cancel-requested` with the typed `user | parent` cause before clearing or aborting; idle cancellation emits nothing and cannot strand the next prompt. `whenIdle()` reaches post-cancel quiescence, and ACP `session/cancel` maps to `user`. The [explicit turn-cancellation decision](2026-07-16-explicit-turn-cancellation.md) owns the current cause, signal-lifetime, and cooperative-settlement contract.
|
||||
|
||||
### 2. `AgentHandle` async disposer
|
||||
|
||||
@@ -22,26 +22,26 @@ A new `cancel()` verb on the `Agent` interface — the single public stop primit
|
||||
|
||||
### 3. Bash owner token in the seam
|
||||
|
||||
Background-task ownership moved from a `tool-bash` plugin-local `Map<string, Agent>` into the executor. `BashExecRequest` gains an optional `owner?: string`; the resolved `BashExecSpec` carries it as required-but-nullable `owner: string | undefined` (a forgotten owner is a visible `undefined`, never a silently-absent property). The executor stores the token on its task and exposes it via a new `BashExecutor.ownerOf(id): string | undefined` seam (NOT on the public `BashTask` — one read path, no redundant API). `tool-bash` deletes its `Map` entirely: it stamps `exec.agent?.session.header.id` as the owner at `start`, and `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to the caller's token with `!== undefined` semantics (an empty-string token is still a real owner). The completion notice finds the live agent by scanning `ctx.get('agents')?.list()` for `agent.session.header.id === ownerToken` (read via `ctx.get` — `onTaskDone` runs on the bash fiber, a foreign fiber, where the `ctx.agents` proxy would throw). Because ownership now lives on the task in the executor (disposed with the `dsh-bash` fiber), it SURVIVES a `tool-bash` HMR reload — closing the old `XXX(tool-bash-owner-hmr)` gap. (The `onTaskDone` listener is still effect-scoped to `tool-bash`'s `apply`, so a completion landing during the reload gap still drops its one notice — the pre-existing reload-gap drop — but the ownership fence itself is HMR-proof.)
|
||||
Background-task ownership moved from a `tool-bash` plugin-local `Map<string, Agent>` into the executor. `BashExecRequest` gains an optional `owner?: string`; the resolved `BashExecSpec` carries it as required-but-nullable `owner: string | undefined` (a forgotten owner is a visible `undefined`, never a silently-absent property). The executor stores the token on its task and exposes it via a new `BashExecutor.ownerOf(id): string | undefined` seam (NOT on the public `BashTask` — one read path, no redundant API). `tool-bash` deletes its `Map` entirely: it stamps `exec.agent?.id` (the shared registry/session id) as the owner at `start`, and `bash_output`/`bash_kill` compare `ctx.bash.ownerOf(id)` to the caller's token with `!== undefined` semantics (an empty-string token is still a real owner). The completion notice finds the live agent by scanning `ctx.get('agents')?.list()` for `agent.id === ownerToken` (read via `ctx.get` — `onTaskDone` runs on the bash fiber, a foreign fiber, where the `ctx.agents` proxy would throw). Because ownership now lives on the task in the executor (disposed with the `dsh-bash` fiber), it SURVIVES a `tool-bash` HMR reload — closing the old `XXX(tool-bash-owner-hmr)` gap. (The `onTaskDone` listener is still effect-scoped to `tool-bash`'s `apply`, so a completion landing during the reload gap still drops its one notice — the pre-existing reload-gap drop — but the ownership fence itself is HMR-proof.)
|
||||
|
||||
## Verification
|
||||
|
||||
These invariants hold and are pinned by tests:
|
||||
|
||||
- ACP disconnect/session close leaves no registered agent AND no session-store entry for that session, even when `session/load` races teardown.
|
||||
- `session/cancel` before a queued prompt starts prevents that prompt from running and cannot batch the next prompt into the cancelled turn.
|
||||
- `session/cancel` before a queued prompt starts prevents that prompt from running; a later accepted prompt remains an independent queued turn.
|
||||
- A `tool-bash` HMR reload does NOT make an existing background task readable or killable by a different session (ownership survives on the executor).
|
||||
- Existing non-ACP demos still work without managing handles explicitly; config-created agents remain owned by the `AgentLoop` plugin fiber.
|
||||
|
||||
## Session owner tokens are unique among live agents
|
||||
|
||||
The bash owner-token comparison relies on `session.header.id` being unique among live agents. Concurrent same-ID operations may both prepare privately, but publication enters the session and agent in order; `SessionStore.enter()` rejects a duplicate live session id, and every losing transaction rolls its private state back. A programmatic caller therefore cannot publish two live agents with one session token. The access *policy* (token comparison) stays in `tool-bash` (the consumer); the bash seam stores only an opaque `owner` string and never interprets it — the correct interface/implementation/consumer split.
|
||||
The bash owner-token comparison relies on the shared `Agent.id`/`SessionId` being unique among live agents. Concurrent same-ID operations may both prepare privately, but publication enters the session and agent in order; `SessionStore.enter()` rejects a duplicate live session id, and every losing transaction rolls its private state back. A programmatic caller therefore cannot publish two live agents with one session token. The access *policy* (token comparison) stays in `tool-bash` (the consumer); the bash seam stores only an opaque `owner` string and never interprets it — the correct interface/implementation/consumer split.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **A public `BashTask.owner` field** instead of the `BashExecutor.ownerOf(id)` seam — rejected: one read path, no redundant API.
|
||||
- **Sibling cordis effects for the agent's session lifecycle** — rejected: a fiber unload disposes sibling effects concurrently (`Promise.all`), racing removal of the store-owned append publication hooks against the loop's closing `session/flush`; the single composite effect's ordered LIFO chain is what captures the closing `turn/end` on both disposal paths.
|
||||
- **A separate step-only `abort()` beside `cancel()`** — shipped originally, then removed as unused; `cancel()` is the single public stop primitive ([the public-stop-surface RFC](../simplification/2026-06-20-public-agent-stop-surface.md)).
|
||||
- **A separate step-only `abort()` beside `cancel()`** — shipped originally, then removed as unused; `cancel()` is the single public stop primitive ([the public-stop-surface Agent Note](../simplification/2026-06-20-public-agent-stop-surface.md)).
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Agent Note: Session surface — an ordered projection over the event log
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The event log is authoritative, but history manipulation had no durable shared mechanism. Plugins such as compaction would otherwise rewrite derived requests through order-sensitive listeners, leave no provenance, and require repeated changes to `deriveMessages()`.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a **surface** — a derived, cached order of event sequences (the subset of events that produce LLM messages) — maintained by `surfaceOp` markers in the event log.
|
||||
|
||||
### Two new top-level fields on `SessionEvent`
|
||||
|
||||
Every `SessionEvent` gains two optional fields (structural metadata, like `seq`/`time`):
|
||||
|
||||
- **`sourceEventSeqs?: number[]`** — seq numbers of events that are provenance sources (e.g., the `assistant/chunk` seqs that built an `assistant/message`, or the surface nodes shadowed by a compaction marker). A present `[]` is valid only on `assistant/message` and records a known empty provider stream; omission there means legacy or otherwise unrecorded provenance. Other surface events require a non-empty list when the field is present. Provenance is a core design principle; without it, the replace-range operation cannot be validated on replay.
|
||||
- **`surfaceOp?: SurfaceOp`** — how this event entered the surface. Absent for non-surface events.
|
||||
|
||||
### SurfaceOp: two operations
|
||||
|
||||
```ts
|
||||
export type SurfaceOp =
|
||||
| 'append' // normal tail append
|
||||
| { op: 'replace'; start: number; end: number } // shadow [start, end] inclusive
|
||||
```
|
||||
|
||||
1. **Append** — add the new event seq to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`. The loop passes `surfaceOp: 'append'` on all such appends and records `sourceEventSeqs` where applicable: every successful `assistant/message` records its complete `assistant/chunk` source set, including `[]`, while `tool/result` records its `tool/call` source.
|
||||
|
||||
2. **Replace** — remove entries from `start` through `end` (both inclusive) and insert the new event seq in their place. Both `start` and `end` must be present in the current surface; `start === end` replaces one entry. The event's `sourceEventSeqs` must contain every shadowed surface seq. The shadowed events remain in the log but are no longer on the surface.
|
||||
|
||||
### SurfaceManager: delta-based, not full rebuild
|
||||
|
||||
A `Session` owns one `SurfaceManager` that maintains an ordered `number[]` of event seqs. The manager validates each seed or append candidate without applying it before commit, then processes only committed events since its previous synchronization rather than rescanning the entire log. `Session.surface` exposes the same manager through the readonly `SessionSurface` contract, so acceptance, derived history, compaction, and workspace context share one incremental state. Replace locates its inclusive endpoints by array position and splices the replacement seq into that range; no second manager, link objects, or seq-to-node map duplicates the order.
|
||||
|
||||
Delta processing is O(1) when no new events and O(new events) when new events arrive.
|
||||
|
||||
`deriveMessages()` uses the surface when surface markers exist, falling back to the existing linear scan for sessions without markers (backward compatibility).
|
||||
|
||||
### Persistence
|
||||
|
||||
The new fields are serialized as top-level JSON properties. The JSONL backend requires zero changes — `JSON.stringify`/`JSON.parse` preserve everything transparently. The SQLite backend's `events` table carries two nullable TEXT columns (`source_event_seqs`, `surface_op`). The on-disk `SCHEMA_VERSION` is bumped to reflect the column set, and — per the pre-release bump-and-reject policy — a database written by any other build is REJECTED on open rather than migrated (there is no persisted user data to upgrade). The session format `version` is pinned at `SESSION_FORMAT_VERSION = 0` (the "unstable / pre-release" stance): the optional surface fields are absorbed without bumping it.
|
||||
|
||||
### Crash recovery
|
||||
|
||||
The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls after a crash. These closers carry `surfaceOp: 'append'` and `sourceEventSeqs` pointing to the orphaned `tool/call` event, so the rehydrated surface is valid.
|
||||
|
||||
### Invariants
|
||||
|
||||
`Session` validates `sourceEventSeqs` and `surfaceOp` at the always-on seed/append boundary: only `assistant/message` may use an empty provenance list; references are unique, earlier, and known; replacement endpoints exist in surface order; and provenance covers every shadowed node. These are single-record acceptance and storage-projection rules, not optional invariant-service contributions.
|
||||
|
||||
Every surface-eligible event must carry `surfaceOp` or it would disappear from derived history. Typed `append` overloads enforce this for literal event types; runtime checks in `append` and the seed constructor cover widened unions and loaded logs. Invalid seeds are rejected rather than upgraded under the pre-release format policy.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Per-plugin `agent/request` wrapping** (the pre-surface pattern for history manipulation) — listener-ordering fragility, no durable record of what was changed, and every new manipulation forces another change to core `deriveMessages()`.
|
||||
- **Half-open `[start, endExclusive)` replace ranges** — rejected: endpoints are named by surface event seqs, and single-entry replacement (`start === end`) reads naturally with inclusive semantics.
|
||||
- **Linked node objects plus a seq map** — rejected: production did not read predecessor links, the only successor use was the next array position, and replacement already required linear `indexOf` lookup. A single seq array preserves the same asymptotic behavior with one representation to validate.
|
||||
- **Full rebuild behind a dirty flag** instead of delta processing — O(N²) over a session's lifetime: every single-event append would rescan all prior events.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`packages/core/session`**: `surface.ts` (`SurfaceManager`) maintains one ordered seq array for candidate acceptance and live projection; `SessionSurface` is its readonly public view. `SurfaceOp`/`SurfaceIntent` and the top-level session-event fields record how entries join it. `append()` requires a `SurfaceIntent` for surface events, `deriveMessages()` walks the surface as the sole derivation path, and `repair.ts` emits surface-aware closers. The seed constructor rejects a surface-eligible seed event missing its `surfaceOp` marker (see § Invariants).
|
||||
- **`packages/core/agent-loop`**: All surface-capable appends pass surface opts. Chunk seqs are collected for `assistant/message` provenance; `tool/call` seqs are captured for `tool/result` provenance.
|
||||
- **`packages/session-persistence/session-persistence-sqlite`**: Two new nullable TEXT columns (`source_event_seqs`, `surface_op`) on the `events` table; `SCHEMA_VERSION` bumped (bump-and-reject, no migration).
|
||||
- **`packages/session-persistence/session-persistence-jsonl`**: No changes required.
|
||||
- **`packages/session-persistence/session-persistence`**: Abstract interface unchanged.
|
||||
|
||||
The surface is the foundation for future history manipulation. A compaction or tool-result-prune plugin appends one of the existing message-producing event types (a `user/message` carrying the summary, say) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed entries — the new event takes the range's place on the surface while the plugin's own trace events (e.g. `compaction/start`, `compaction/end`) stay off it. Replay preserves the decision deterministically.
|
||||
|
||||
A `tool/result` replacement may rewrite exactly one current `tool/result` and must preserve every data field except `content`. Session acceptance enforces this rule together with positional range and provenance validation, independent of optional diagnostic plugins.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Shared persistence write coordinator
|
||||
# Agent Note: Shared persistence write coordinator
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -10,7 +10,9 @@ Status: implemented
|
||||
|
||||
Extract a backend-agnostic `PersistenceCoordinator` into `dsh-session-persistence`. The coordinator owns the orchestration once; each first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements a small `PersistenceBackend` hook interface, and delegates its four public service methods (`create`/`append`/`load`/`list`) to it.
|
||||
|
||||
Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The RFC's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks; it cannot reach the coordinator's private orchestration state, and the public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator at all.
|
||||
Composition, not inheritance. The coordinator is a concrete class the backend holds, not a base class the backend extends. The Agent Note's risk — "a coordinator must not make unusual backends fight an inheritance hierarchy" — is avoided: a backend exposes only the hooks; it cannot reach the coordinator's private orchestration state, and the public `SessionPersistence` service shape is unchanged, so a third-party backend MAY still implement the abstract service directly without the coordinator at all.
|
||||
|
||||
The coordinator retires each live session from its `session/disposed` notification: it waits for that exact Session object's initialization, serializes a final drain, and then removes the owned state, buffer, and init entries. Failed drains retain their buffers for backend teardown to retry. Settled per-id chain tails remove themselves only when they are still the current tail, so a completion cannot erase a newer operation for the same id. Backend teardown unregisters the write-path listeners before awaiting all admitted retirements, remaining buffers, and chains, then closes the backend.
|
||||
|
||||
### The hook interface (`PersistenceBackend<TornMarker>`)
|
||||
|
||||
@@ -26,11 +28,11 @@ Six methods (five required + an optional lifecycle hook) — the only seam betwe
|
||||
|
||||
### The opaque torn marker
|
||||
|
||||
The single design choice that keeps the seam clean: the crash-repair "where is the torn tail" token is OPAQUE to the coordinator. The coordinator computes the synthetic closers (it owns `interruptedTurnClosers` from `dsh-session`), but it only ever tests `tornMarker !== undefined` and passes the value straight back to `commitRepair` — it never inspects it. Each backend picks its own marker type: JSONL uses the byte offset to truncate to, SQLite the seq to delete from (both happen to be `number`). The JSONL backend folds its `committedBytes < buffer.byteLength` comparison INSIDE the hook so the returned marker is already `number | undefined`; without that fold the coordinator would have to know about byte lengths.
|
||||
The single design choice that keeps the seam clean: the crash-repair "where is the torn tail" token is OPAQUE to the coordinator. The coordinator computes the synthetic closers (it owns `interruptedTurnClosers` from `dsh-session`), but it only ever tests `tornMarker !== undefined` and passes the value straight back to `commitRepair` — it never inspects it. Each backend picks its own marker type: JSONL carries the byte offset to truncate to plus any complete events decoded from an incomplete final frame, while SQLite carries the seq to delete from. The coordinator therefore knows neither byte lengths nor frame recovery state.
|
||||
|
||||
## Testing
|
||||
|
||||
The shared `runPersistenceContract` (public-API contract) keeps running for every backend. A new `runCoordinatorContract` (`tests/coordinator-contract.ts`) holds the write-path orchestration — adoption, HMR, collision, dispose-drain, crash-tail repair — and runs once per backend through a `CoordinatorFixture` (an in-memory reference + jsonl + sqlite). The per-backend specs shrank to storage mechanics only (JSONL: path safety, fsync rollback, bucket listing; SQLite: schema version, `scanRows`, transaction rollback). A through-coordinator torn-tail→load→`commitRepair` test per real backend (via a `corruptTail` fixture hook) keeps the coordinator's torn-marker repair branch covered under the 100% per-file gate — the contract crash test only produces synthetic closers, never a torn marker, so it could not reach that branch.
|
||||
The shared `runPersistenceContract` (public-API contract) keeps running for every backend. `runCoordinatorContract` (`tests/coordinator-contract.ts`) holds the write-path orchestration — adoption, HMR, collision, session and backend disposal drains, and crash-tail repair — and runs once per backend through a `CoordinatorFixture` (an in-memory reference + jsonl + sqlite). Coordinator-specific tests pin retirement map cleanup, same-id chain-tail races, failed-drain retry, and close ordering. The per-backend specs retain storage mechanics only (JSONL: path safety, fsync rollback, bucket listing; SQLite: schema version, `scanRows`, transaction rollback). A through-coordinator torn-tail→load→`commitRepair` test per real backend (via a `corruptTail` fixture hook) keeps the coordinator's torn-marker repair branch covered under the 100% per-file gate — the contract crash test only produces synthetic closers, never a torn marker, so it could not reach that branch.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -39,4 +41,4 @@ The shared `runPersistenceContract` (public-API contract) keeps running for ever
|
||||
|
||||
## Consequences
|
||||
|
||||
The pre-extraction duplication was verbose but explicit — each backend read top-to-bottom. The coordinator adds one indirection (the hook seam) and one new concept (the opaque torn marker). This clears the bar because the centralized logic is the correctness-heavy part that was already being fixed twice, and the hook set is narrow (six methods, no inheritance). The hook surface was deliberately held to the minimum: the create-collision probe is NOT a separate hook — it folds into `loadStored(id) !== undefined`; there is no separate `materialize` hook (folded into `appendBatch` for atomicity); `list()` stays a backend method with no coordinator pass-through (listing needs none of the orchestration). The net effect is a reduction: one orchestration copy instead of two, the backends shrank by ~1200 lines of duplicated churn, and a future backend implements a handful of small primitives instead of copying the entire `session/event` → buffer → flush machinery.
|
||||
The coordinator adds one indirection, an opaque torn marker, and detached session-retirement tasks, but centralizes correctness-heavy orchestration previously duplicated by every backend. Session disposal remains an observe-only event, so the session owner does not await persistence retirement; the coordinator contains failures, preserves uncommitted buffers, and makes backend teardown the quiescence boundary. Its hook surface stays narrow: collision checks reuse `loadStored`, materialization stays atomic inside `appendBatch`, and listing bypasses the coordinator. New backends implement storage primitives rather than copy the event-buffer-flush lifecycle.
|
||||
@@ -1,26 +1,26 @@
|
||||
# RFC: Branded IDs everywhere they belong
|
||||
# Agent Note: Branded IDs everywhere they belong
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness already brands three identifiers — `CallId` (`packages/llm/llm/src/brand.ts`), `SessionId` (`packages/core/session/src/types.ts`), and `AgentId` (`packages/core/agent/src/types.ts`) — using the `Branded<B> = string & { readonly [BRAND]: B }` machinery (owned by the type-only `@deepseek-ai/dsh-brand` package at `packages/util/brand/` — see its [README](../../../../packages/util/brand/README.md)) and a zero-cost cast factory per type. `dsh-brand` also states the governing policy: *"Branding is for ids that cross package boundaries and could plausibly be confused; not every string needs a brand."* That policy is right; the problem is that it is only half-applied. Two gaps let a structurally-identical-but-semantically-wrong string slip through the type checker today.
|
||||
The harness brands `CallId` (`packages/llm/llm/src/brand.ts`) and the shared agent/session `SessionId` (`packages/core/session/src/types.ts`) using the `Branded<B> = string & { readonly [BRAND]: B }` machinery (owned by the type-only `@deepseek-ai/dsh-brand` package at `packages/util/brand/` — see its [README](../../../../packages/util/brand/README.md)) and a zero-cost cast factory per type. `dsh-brand` also states the governing policy: *"Branding is for ids that cross package boundaries and could plausibly be confused; not every string needs a brand."* That policy is right; the problem is that it is only half-applied. Two gaps let a structurally-identical-but-semantically-wrong string slip through the type checker today.
|
||||
|
||||
**Gap 1 — unbranded cross-boundary IDs in the bash seam.** The background-task id is a plain `string`: `BashTask.id: string` (`packages/bash/bash/src/types.ts`), carried as `string` through the whole executor seam (`BashExecutor.get`/`ownerOf`/`readOutput`/`kill(id: string)` in `packages/bash/bash/src/index.ts`) and validated/passed as `string` by the model-facing tools (`validateTaskId`, `assertTaskAccess`, the `task_id` schema arg in `packages/bash/tool-bash/src/index.ts`). It is generated by a per-executor counter — `` `bash-${this.nextTaskId++}` `` in `packages/bash/bash-local/src/index.ts` — which gives it **exactly the same `name-N` shape as `SessionId`'s default** (`` `session-${++counter}` `` in `packages/core/session/src/index.ts`). A bash task id and a session id are trivially swappable at a call site and the compiler says nothing. This is the headline case the user asked about, and it is a model-facing id (the model passes `task_id` back to `bash_output`/`bash_kill`), so a confusion here is reachable from untrusted input.
|
||||
|
||||
The bash **owner token** is the related sub-case: `BashExecRequest.owner?: string` and `BashExecSpec.owner: string | undefined` (`packages/bash/bash/src/types.ts`) are documented as a deliberately *opaque* isolation key, but in every live caller the value IS the owning agent's `session.header.id` (`callerToken = (exec) => exec.agent?.session.header.id` in `packages/bash/tool-bash/src/index.ts`) — i.e. a `SessionId` wearing a `string` disguise. It is compared for access control (`owner !== callerToken(exec)`), so a mismatched-but-well-typed string here is a cross-session isolation bug the type system currently cannot catch. This is the same `session.header.id`-as-owner alias that the [unify-the-agent-id-and-the-session-id](../../proposed/simplification/2026-06-20-unify-agent-and-session-id.md) proposal calls the "bash owner-token alias hole".
|
||||
The bash **owner token** is the related sub-case: `BashExecRequest.owner?: string` and `BashExecSpec.owner: string | undefined` (`packages/bash/bash/src/types.ts`) are documented as a deliberately *opaque* isolation key, but in every live caller the value IS the owning agent's shared `Agent.id`/`SessionId` (`callerToken = (exec) => exec.agent?.id` in `packages/bash/tool-bash/src/index.ts`) wearing a different seam-local name. It is compared for access control (`owner !== callerToken(exec)`), so a mismatched-but-well-typed string here is a cross-session isolation bug the type system currently cannot catch. This is the shared id alias covered by the [unified agent/session identity decision](../simplification/2026-06-20-unify-agent-and-session-id.md).
|
||||
|
||||
**Gap 2 — brand erosion at the seams of the *already-branded* IDs.** Even `CallId`/`SessionId`/`AgentId` decay back to bare `string` at exactly the places confusion is most likely: the registry/store `Map` key types and most public method params. Representative sites: `SessionStore.store = new Map<string, Session>()` and `create`/`prepare(id?: string)`/`get(id: string)` (`packages/core/session/src/index.ts`); `AgentRegistry.store = new Map<string, Agent>()` and `register`/`get(id: string)` (`packages/core/agent/src/index.ts`); `ToolPresenter.pending = new Map<string, …>()` keyed by call id and `call(callId: string)`/`result(callId: string)` (`packages/ui/acp/src/index.ts`); the ACP session-id surface beyond the store map — `SessionRecord.sessionId: string`, `bySession = new WeakMap<Agent, string>()`, `loadingIds = new Set<string>()`, `requireSession(sessionId: string)`, and the exported `streamSessionEventUpdate(sessionId: string, …)` (`packages/ui/acp/src/index.ts`); and the persistence coordinator's `Map<string, …>` keyed by session id (`packages/session-persistence/session-persistence/src/coordinator.ts`). A brand that is dropped at the `Map` key buys nothing on lookups — the value of the existing brands is partly unrealized.
|
||||
**Gap 2 — brand erosion at the seams of the *already-branded* IDs.** Even `CallId` and `SessionId` decay back to bare `string` at exactly the places confusion is most likely: registry/store key types and public method params. Representative sites include the session store, the agent registry (both keyed by the shared `SessionId`), `ToolPresenter`'s call-id map, ACP's session-id records and loading set, and the persistence coordinator. A brand that is dropped at a collection key buys nothing on lookups — the value of the existing brands is partly unrealized.
|
||||
|
||||
## Decision
|
||||
|
||||
A type-only change. Brands are zero-cost casts; nothing about runtime behavior, serialization, comparison, or the wire format changes. The work is in three parts, all honoring the existing "not every string" policy.
|
||||
|
||||
- **Brand the bash task id.** Add `BashTaskId = Branded<'BashTaskId'>` plus its same-named factory in `packages/bash/bash/src/types.ts` (the package that *owns* the id), importing `Branded` from `@deepseek-ai/dsh-brand` exactly as `SessionId`/`AgentId` already do. The brand primitive lives in the dependency-free `dsh-brand` utility package precisely so `dsh-bash` can brand its ids by depending on it alone — it never pulls in `dsh-llm` (or `dsh-session`) just to reach `Branded`. Thread it through `BashTask.id`, the `BashExecutor` seam methods (`get`/`ownerOf`/`readOutput`/`kill`), the generation site in `dsh-bash-local` (brand the counter output once, at creation), and the `dsh-tool-bash` validate/access surface (`validateTaskId` returns a `BashTaskId`; `task_id` is branded at the tool boundary where the model's string arrives).
|
||||
- **Brand the bash task id.** Add `BashTaskId = Branded<'BashTaskId'>` plus its same-named factory in `packages/bash/bash/src/types.ts` (the package that *owns* the id), importing `Branded` from `@deepseek-ai/dsh-brand` exactly as `SessionId` does. The brand primitive lives in the dependency-free `dsh-brand` utility package precisely so `dsh-bash` can brand its ids by depending on it alone — it never pulls in `dsh-llm` (or `dsh-session`) just to reach `Branded`. Thread it through `BashTask.id`, the `BashExecutor` seam methods (`get`/`ownerOf`/`readOutput`/`kill`), the generation site in `dsh-bash-local` (brand the counter output once, at creation), and the `dsh-tool-bash` validate/access surface (`validateTaskId` returns a `BashTaskId`; `task_id` is branded at the tool boundary where the model's string arrives).
|
||||
|
||||
- **Mint a distinct `OwnerToken` brand.** Add `OwnerToken = Branded<'OwnerToken'>` in `packages/bash/bash/src/types.ts`; type `BashExecRequest.owner` / `BashExecSpec.owner` / `BashExecutor.ownerOf` as `OwnerToken | undefined`. The `dsh-tool-bash` consumer casts the agent's `session.header.id` (a `SessionId`) into an `OwnerToken` at the boundary — the one place the two vocabularies meet. The bash seam never imports `dsh-session`. (Rationale in the next section.)
|
||||
- **Mint a distinct `OwnerToken` brand.** Add `OwnerToken = Branded<'OwnerToken'>` in `packages/bash/bash/src/types.ts`; type `BashExecRequest.owner` / `BashExecSpec.owner` / `BashExecutor.ownerOf` as `OwnerToken | undefined`. The `dsh-tool-bash` consumer casts the agent's shared `id` (`SessionId`) into an `OwnerToken` at the boundary — the one place the two vocabularies meet. The bash seam never imports `dsh-session`. (Rationale in the next section.)
|
||||
|
||||
- **Stop the brand erosion.** Propagate the existing brands to the `Map` key types and public method params listed under Gap 2 — `Map<SessionId, Session>`, `get(id: SessionId)`, `Map<AgentId, Agent>`, `Map<CallId, …>`, the ACP `SessionRecord.sessionId: SessionId` surface, the coordinator's `Map<SessionId, …>`. This is the larger mechanical share of the diff and the part that makes the *existing* brands actually load-bearing on lookups, not just on the struct fields.
|
||||
- **Stop the brand erosion.** Propagate the existing brands to the `Map` key types and public method params listed under Gap 2 — `Map<SessionId, Session>`, `Map<SessionId, Agent>`, `get(id: SessionId)`, `Map<CallId, …>`, ACP's `SessionId` surface, and the coordinator's `Map<SessionId, …>`. This is the larger mechanical share of the diff and the part that makes the *existing* brands actually load-bearing on lookups, not just on struct fields.
|
||||
|
||||
Illustrative shape (the factory pattern is identical to the three existing brands):
|
||||
|
||||
@@ -50,18 +50,18 @@ The obvious shortcut is to type `owner` as `SessionId` directly — it always *i
|
||||
|
||||
Kept deliberately narrow per the "not every string needs a brand" policy. Each of these is a plausible future brand, deferred with a reason, not a commitment:
|
||||
|
||||
- **`ModelId`** (`GenerateOptions.model`, the `LlmService` adapter-registry key) — a real cross-package lookup key (config → agent → llm → adapter); a reasonable next brand, left out only to keep this RFC's blast radius focused.
|
||||
- **`ModelId`** (`GenerateOptions.model`, the `LlmService` adapter-registry key) — a real cross-package lookup key (config → agent → llm → adapter); a reasonable next brand, left out only to keep this Agent Note's blast radius focused.
|
||||
- **`ToolName`** (the `ToolRegistry` key) — author-defined, human-readable, and rarely confused with another id; the weakest candidate, likely not worth a brand.
|
||||
- **`ErrorCode`** (`HarnessError.code`) — a closed vocabulary (`ABORTED`, `NO_ADAPTER`, …), not a per-instance id; better served by a string-literal union than a brand, if anything.
|
||||
- **Numeric ordinals** — turn number, step number, and the event `seq` are `number`, not `string`, so `Branded<string>` does not apply; a parallel `number & { readonly [BRAND]: B }` variant could brand them, but they are positional ordinals rarely passed across boundaries, so the payoff is low.
|
||||
- **Validated construction** — the brand factories are pure casts with no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string today. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a *runtime-behavior* change with its own design (what is "malformed"? what do we do on failure?) and belongs in its own RFC, not bundled into this type-only pass.
|
||||
- **Validated construction** — the brand factories are pure casts with no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string today. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a *runtime-behavior* change with its own design (what is "malformed"? what do we do on failure?) and belongs in its own Agent Note, not bundled into this type-only pass.
|
||||
|
||||
## Verification
|
||||
|
||||
The landed invariants: `BashTaskId` and `OwnerToken` are defined in `dsh-bash` and threaded end-to-end (executor seam, the `dsh-bash-local` generation site, the `dsh-tool-bash` model-facing surface) with no `dsh-bash` dependency on `dsh-session`; no collection keyed by an in-scope branded id (`CallId`/`SessionId`/`AgentId`/`BashTaskId`) is keyed by bare `string` — `Map` keys, `WeakMap` value slots, `Set` membership (the ACP `bySession`/`loadingIds`), public method params, and exported signatures (`streamSessionEventUpdate`) all take the brand; and brands are constructed via the cast factory at each boundary where a raw string enters (provider call id, ACP session id, model-supplied `task_id`), never as scattered `as` casts.
|
||||
The landed invariants: `BashTaskId` and `OwnerToken` are defined in `dsh-bash` and threaded end-to-end (executor seam, the `dsh-bash-local` generation site, the `dsh-tool-bash` model-facing surface) with no `dsh-bash` dependency on `dsh-session`; no collection keyed by an in-scope branded id (`CallId`/`SessionId`/`BashTaskId`) is keyed by bare `string`; public method params and exported signatures keep the brand; and brands are constructed via the cast factory at each boundary where a raw string enters (provider call id, ACP session id, model-supplied `task_id`), never as scattered `as` casts.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Mechanical churn across two surfaces.** Propagating brands touches the bash seam (interface + impl + consumer) and the ACP session-id surface plus the persistence coordinator. The churn is broad but low-severity: a missed site is a compile error, not a silent bug. The change is observably type-only — no snapshot or e2e behavioral diff. It sits next to the [unify-the-agent-id-and-the-session-id](../../proposed/simplification/2026-06-20-unify-agent-and-session-id.md) proposal (both touch the session-id / owner-token boundary); if that proposal lands, `OwnerToken` still stays distinct from the unified id for the decoupling reason above.
|
||||
- **Brands do not validate.** A brand is a confusability guard, not a correctness proof: a *wrong* session id that is still a well-formed string passes the type checker exactly as before. This RFC does not close that gap (see Out of scope) — it only stops the *category* error of passing the wrong *kind* of id.
|
||||
- **The "where to stop" line stays a judgment call.** Branding `BashTaskId` but not `ToolName`, `OwnerToken` but not `ModelId`, is a taste call about which strings "could plausibly be confused." Reasonable reviewers may want more or fewer; the policy in `brand.ts` is the tie-breaker, and this RFC errs toward the ids that are model-facing or used for access control.
|
||||
- **Mechanical churn across two surfaces.** Propagating brands touches the bash seam (interface + impl + consumer) and the ACP session-id surface plus the persistence coordinator. The churn is broad but low-severity: a missed site is a compile error, not a silent bug. The change is observably type-only — no snapshot or e2e behavioral diff. It sits next to the [unified agent/session identity decision](../simplification/2026-06-20-unify-agent-and-session-id.md) because both touch the session-id / owner-token boundary; `OwnerToken` stays distinct from the unified id for the decoupling reason above.
|
||||
- **Brands do not validate.** A brand is a confusability guard, not a correctness proof: a *wrong* session id that is still a well-formed string passes the type checker exactly as before. This Agent Note does not close that gap (see Out of scope) — it only stops the *category* error of passing the wrong *kind* of id.
|
||||
- **The "where to stop" line stays a judgment call.** Branding `BashTaskId` but not `ToolName`, `OwnerToken` but not `ModelId`, is a taste call about which strings "could plausibly be confused." Reasonable reviewers may want more or fewer; the policy in `brand.ts` is the tie-breaker, and this Agent Note errs toward the ids that are model-facing or used for access control.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Agent Note: Extract example apps into packages
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
An example folder is supposed to be *thin* — the variable wiring of a demo, not the demo's machinery. Before this change it was thick. Each example carried a hand-rolled `start.ts` boot bootstrap, an infra preamble (`timer`, and — for the stdio demos — `logger` + `hmr`), nested includes of three shared YAML fragments (`base.yml` / `base-core.yml` / `acp-agent/acp-tail.yml`), and per-example `agent-loop`/persistence/system-prompt config. The actual app — the spine of services every agent needs — was spread across the leaf and those includes.
|
||||
|
||||
The leaf configs also owned coupled front doors. ACP requires stdout purity and creates agents through `session/new`; terminal and Headless apps pre-create `main` but have different process I/O contracts. Prose warnings were the only guard against combining these incorrectly, while three `start.ts` files duplicated the Loader bootstrap and lifecycle code.
|
||||
|
||||
## Decision
|
||||
|
||||
Each example is now **mostly an invocation of an app package**, splitting the wiring along the existing [interface / implementation / consumer seam](2026-06-13-capability-seams.md): the **app package owns the composition**, the leaf `cordis.yml` owns only the **swappable choices** (which LLM adapter, which bash executor, model, prompt, persistence root).
|
||||
|
||||
- **`@deepseek-ai/dsh-agent-spine-demo`** ([packages/examples/agent-spine-demo](../../../../packages/examples/agent-spine-demo)) composes the providerless, executor-less, UI-less spine and forwards the loop's agent-list config. Its dependency on the concrete loop is intentional because this package composes the spine rather than extending it; swapping the loop means supplying another bundle.
|
||||
- **`@deepseek-ai/dsh-tui-demo`**, **`@deepseek-ai/dsh-cli-demo`**, and **`@deepseek-ai/dsh-acp-demo`** bake in their process roles. TUI includes the full-screen UI and a pre-created `main`; Headless includes the one-shot driver and a pre-created `main`; ACP includes the bridge and no pre-created agent. All three include JSONL persistence and omit stdout loggers.
|
||||
- **`start.ts` is gone.** Each app package exposes a bin; the `demo:*` scripts invoke it. Loader boot, `.env` loading, and fail-loud guards live in the shared [`@deepseek-ai/dsh-app-boot`](../../../../packages/ui/app-boot) package (unit-tested under the per-file coverage gate — see [share the app bins' boot glue](../simplification/2026-07-04-share-app-bin-boot-glue.md)); the thin self-executing entries are driven by keyless Loader-path tests.
|
||||
- **Each leaf `cordis.yml` collapses** to backends, optional product tools, and one app entry carrying the app config. TUI and Headless route model/session choices onto a pre-created agent; ACP routes the initial provider/model onto its bridge.
|
||||
- **`base.yml`, `base-core.yml`, and `acp-agent/acp-tail.yml` are retired** — the spine they shared now lives in `dsh-agent-spine-demo`.
|
||||
|
||||
`bash-local` and the LLM adapter stay **leaf choices**: the bundle ships `tool-bash` (the consumer schema), the leaf picks the executor implementation, so a sandboxed executor or replay adapter swaps in without touching the app.
|
||||
|
||||
### Amendment on implementation: `hmr` stays a leaf entry
|
||||
|
||||
The proposal listed `hmr` among the interactive app's baked-in front-door cluster. Validating against the code, baking `hmr` into the app package fights Cordis in two ways, so it ships as a **leaf `cordis.yml` entry** instead:
|
||||
|
||||
1. `@cordisjs/plugin-hmr` is a Loader-only, subprocess-only dev plugin — its constructor throws without `node --expose-internals` + a live `loader` service, so it can only run in the real `demo:*`/bin subprocess, never in the in-process unit/coverage tier.
|
||||
2. The in-process test tier (vitest) cannot even *import* the vendored `hmr` module (its class-decorator `@Inject` form fails under Vite's transform), so a package whose `apply` statically imported it could never satisfy the per-file 100% coverage gate on its headline function.
|
||||
|
||||
Crucially, `hmr` is not a stdout-purity footgun: a stray entry in the ACP config does not corrupt JSON-RPC frames. Every shipped app omits a stdout console logger; the app or protocol driver alone owns stdout.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Why not keep the wiring in shared YAML includes?
|
||||
|
||||
The old `base*.yml`/`acp-tail.yml` includes already deduped the *config*, but a YAML include cannot **encapsulate** the front-door coupling — it can only describe it in a comment and trust every leaf to obey. It also cannot own a `bin`, so the boot glue stayed copied across three `start.ts` files. A package turns "the ACP app never logs to stdout" from a prose warning into a property of the artifact: there is no logger entry in the leaf to get wrong.
|
||||
|
||||
## Verification
|
||||
|
||||
- Example directories contain only their config, README, and tests: `start.ts`, the infrastructure preamble, and the shared YAML includes are gone.
|
||||
- `demo:tui`, `demo:headless`, and `demo:acp` invoke the app-package bins.
|
||||
- Each new package has a README and per-file 100% coverage; each app package also has a keyless real-Loader-path bin smoke that catches export-shape failures described in [postmortem 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md).
|
||||
- The ACP replay transcript remains unchanged because the plugin set and load order did not change.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The bare-plugin-tree pedagogy.** The spine lives behind a bundle, so seeing the whole tree means opening `dsh-agent-spine-demo`. The app package's README carries that teaching weight.
|
||||
- **A layer of indirection.** "What does this demo load?" becomes a package read, not a single YAML scan.
|
||||
|
||||
## Related
|
||||
|
||||
- Supersedes [Make the shared example base providerless](../../rejected/architecture/2026-06-20-providerless-example-base.md): renaming `base.yml` to the providerless core is moot once the spine moves into `dsh-agent-spine-demo` and the `base*.yml` files are deleted.
|
||||
- Builds on the [capability-seams](2026-06-13-capability-seams.md) interface/implementation/consumer split — backends and presentation stay leaf choices; the spine is the shared bundle.
|
||||
- Complements [Reorganize packages into a modular hierarchy](2026-06-20-package-hierarchy.md): the new app/core packages slot into existing groups under that hierarchy (`core` for the reusable spine bundle, `ui` for the app-specific front doors).
|
||||
- The later [redundant-agent removal](../simplification/2026-07-20-remove-stdio-and-echo-agents.md) owns the final TUI/Headless split and removes the line-oriented and mock-only leaves.
|
||||
@@ -0,0 +1,128 @@
|
||||
# Agent Note: The background task runtime (`ctx.tasks`) and generic task control tools
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Background bash originally combined two responsibilities: the bash executor ran processes and also managed task ids, ownership, incremental reads, cancellation, completion listeners, and model-facing control tools. Adding background subagents required the same lifecycle and interaction contract. Implementing that contract independently for every long-running capability would duplicate isolation, cleanup, notification, and prompt behavior while teaching the model a different collect-and-stop protocol for each producer.
|
||||
|
||||
The task registry, control tools, and completion notices form one harness capability. Bash and subagents should supply execution-specific hooks without owning generic task behavior.
|
||||
|
||||
## Decision
|
||||
|
||||
The `tasks/` package group owns background-task semantics:
|
||||
|
||||
- `@deepseek-ai/dsh-tasks` registers running work as `ctx.tasks` and owns task ids, authorization, snapshots, reads, cancellation, waiting, completion listeners, and cleanup.
|
||||
- `@deepseek-ai/dsh-tool-tasks` exposes `task_output`, `task_list`, and `task_kill`, injects completion notices, and supplies the background-task system-prompt guidance.
|
||||
|
||||
Long-running tools are producers. `dsh-tool-bash` adapts a `BashProcess` into incremental output and process cancellation; `dsh-tool-subagent` adapts a child run into final output and child disposal. The execution seams remain independent of sessions and the task registry.
|
||||
|
||||
`TaskService` is a concrete, process-local service. TODO(task-service-backend): separate its public contract from the implementation when a second backend defines the required lifecycle; a systemd-backed runtime is one plausible driver, but this PR does not speculate about its durability, reconnect, ownership, or observation semantics.
|
||||
|
||||
## Runtime contract
|
||||
|
||||
The literal types live in the [task data-structure catalog](../../../../docs/core-data-structures/tasks.md). A producer calls `ctx.tasks.start()` with a kind, label, optional owning `Agent`, and a `run()` function. The runtime completes all failable preflight work before calling `run()` and invokes it once. After `run()` returns hooks, registration commits without another failable step; a producer cannot start work that lacks a collectable task id.
|
||||
|
||||
The producer hooks define three responsibilities:
|
||||
|
||||
- `cancel(reason?)` synchronously requests termination, is idempotent, and must cause `done` to settle.
|
||||
- `done` never rejects and settles only after the producer has released the task's resources.
|
||||
- Optional `readOutput()` returns the next consuming output delta. Omitting it declares a final-output task whose terminal result comes from `TaskOutcome.output`.
|
||||
|
||||
Statuses are `running`, `stopping`, `completed`, `killed`, and `failed`. Producer-specific information such as an exit code or stop reason belongs in `detail`; the registry does not interpret it. Task kinds form a merge-extensible string union, and task ids are branded and generated as `<kind>-N`, with a counter per kind.
|
||||
|
||||
The runtime attaches one continuation to `done`, records the first terminal outcome, resolves waiters, and invokes completion listeners with per-listener error containment. First-wins settlement matters during teardown: if `cancel` throws, the runtime force-fails the record and warns that work may be orphaned rather than waiting forever for a promise that may never settle. A later producer outcome cannot overwrite that diagnosis or notify twice. A `cancel` that returns without eventually settling `done` still blocks teardown because the runtime cannot distinguish it from a slow, valid stop.
|
||||
|
||||
Task registrations are not effects of the producer tool fiber. Reloading a tool or control-surface plugin therefore does not kill work owned by an agent and backend. The task service's own disposal cancels all live tasks and awaits contract-compliant producers.
|
||||
|
||||
## Authorization and owner lifecycle
|
||||
|
||||
Task ids are runtime-global and predictable, so every access is authorized by the registry. `get`, `read`, `wait`, and `kill` accept the calling `Agent`; `list` returns only tasks visible to that caller. An owned task is accessible only to the exact owning session. Unowned tasks are open to non-agent callers and die with the task service.
|
||||
|
||||
The snapshot stores the owner's branded `SessionId` for authorization, while lifecycle operations retain the exact live `Agent` instance. These identities serve different purposes: session equality grants access, but exact object identity selects cleanup and completion delivery. Reusing an agent or session id cannot redirect an old scope's cleanup or notices to a replacement.
|
||||
|
||||
The first task for an owner attaches one asynchronous effect to `owner.ctx`. Agent-scope disposal cancels that owner's live tasks, awaits their terminal records, and removes their snapshots. This effect survives producer reloads and joins the agent's existing quiescence boundary. The task service retains the effect disposer so service reload can detach callbacks from still-live agent scopes after global teardown.
|
||||
|
||||
For contract-compliant producers, `AgentHandle.dispose()` resolves only after owned background work has stopped. Work intended to outlive an agent must be started unowned; survival across runtime restarts requires a separate durable-job design.
|
||||
|
||||
## Service surface
|
||||
|
||||
`TaskService` provides:
|
||||
|
||||
- `start(spec)` for preflighted, atomic registration.
|
||||
- `get(id, caller?)` and `list(caller?)` for non-consuming snapshots.
|
||||
- `read(id, caller?)` for a consuming stream delta or an idempotent final result.
|
||||
- `kill(id, caller?, reason?)` for cancellation.
|
||||
- `wait(id, timeoutMs, caller?, signal?)` for bounded terminal waiting.
|
||||
- `onTaskDone(listener)` for effect-scoped observation with exact-owner delivery and listener containment.
|
||||
- `attachSurface(name)` for the control-surface availability fence.
|
||||
|
||||
`wait` returns the terminal snapshot when the task settles or the live snapshot when its timeout expires. Aborting a wait cancels only that wait. If settlement has already assigned terminal delivery to the waiter, the terminal snapshot still wins. Waiters unregister synchronously on abort so a same-tick settlement cannot suppress a completion notice on behalf of a reader that receives nothing.
|
||||
|
||||
A producer loaded without any control surface would let callers start work they cannot collect or stop. `dsh-tool-tasks` therefore calls `attachSurface()` for its lifetime, and `start()` fails before producer execution when no surface is attached. This check occurs at start rather than plugin load because sibling plugins may activate concurrently. Custom non-model surfaces can attach themselves without teaching the registry tool names.
|
||||
|
||||
## Model-facing control surface
|
||||
|
||||
`dsh-tool-tasks` registers three kind-independent tools with generic ACP cards:
|
||||
|
||||
- `task_output(task_id, wait?, timeout_ms?)` reads output and always appends `[status: ...]`. Stream tasks return only output since the previous read; final-output tasks return their result after settlement. Reads are non-blocking unless `wait: true`, whose timeout is defaulted and capped by plugin config. A wait timeout reports the still-running status and does not stop the task.
|
||||
- `task_list()` returns caller-visible tasks as `<id> [<kind>] <status> — <label>`, or `(no background tasks)`.
|
||||
- `task_kill(task_id, reason?)` requests cancellation immediately. The optional logged reason is forwarded to the producer. Terminal tasks report their existing status; a throwing producer cancel fails the call and leaves the task running.
|
||||
|
||||
Stream reads share one task-scoped consuming cursor because the owning model is the intended reader. A UI or multiple independent readers need a separate non-consuming observation API; sharing this cursor would let readers consume one another's output.
|
||||
|
||||
The system prompt tells the model to retain task ids, continue independent work instead of busy-polling or duplicating a running task, collect relevant tasks before its final answer, and kill work that no longer matters. Completion injects a logged `context/message` into the exact owner's session; it becomes durable context for the next request but does not wake an idle agent.
|
||||
|
||||
The runtime marks a terminal task `reported` when a read or wait delivers it, when a live waiter has claimed delivery at settlement, or when the model explicitly kills it. Reported tasks do not inject redundant completion notices. Listener failures are logged independently, do not stop later listeners, and are not awaited by waiters or teardown.
|
||||
|
||||
## Producer opt-in
|
||||
|
||||
Each producer owns whether its schema exposes `run_in_background` through defaulted config. `dsh-tool-bash` and each `dsh-tool-subagent` instance use `enableRunInBackground`, defaulting to true. A disabled instance omits the parameter and also rejects a forced background argument at execution because the generic argument validator permits undeclared keys. Schema omission advertises the capability; the execution check enforces it.
|
||||
|
||||
`ctx.tasks` does not rewrite producer schemas. A bundle forwards configuration only for producers it owns. If a background call reaches `start()` without an attached surface, the runtime fence fails before execution.
|
||||
|
||||
## Producer integrations
|
||||
|
||||
The bash seam exposes `resolve`, `run`, and `start`. `start(spec)` returns a `BashProcess` with incremental reads, cancellation, exit facts, and a non-rejecting quiescence promise. The local executor retains live handles only so its own disposal can kill and join processes. Foreground callers continue to use `resolve` and `run` directly.
|
||||
|
||||
For background bash, `dsh-tool-bash` registers the calling agent as owner. Its hooks map `kill()` to cancellation, `done` to a completed or killed `TaskOutcome`, and `readOutput()` to the process's bounded incremental output plus spill and sandbox notices. Generic task tools own ids, status lines, listing, waiting, and completion notices.
|
||||
|
||||
For background subagents, `dsh-tool-subagent` creates a task-owned `AbortController` and begins provider startup inside the task starter. Cancellation aborts the same signal before or after provider readiness. `done` awaits both the child result and child disposal, maps completed output to a final result, maps abort to `killed`, and maps other stop reasons or infrastructure failures to `failed`. Intermediate child history remains in the child session and is not exposed through `readOutput()`.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Per-capability control tools
|
||||
|
||||
Separate bash and subagent output/stop tools duplicate ids, isolation, cleanup, notification, and guidance while increasing the model's schema and protocol burden. One runtime keeps execution-specific behavior in producers without cloning the task lifecycle.
|
||||
|
||||
### An immediate abstract task-runtime backend
|
||||
|
||||
The current `TaskStart.run()` contract passes in-process callbacks and exact `Agent` objects. A durable backend changes identity, restart, ownership, and observation semantics, so extracting an interface before a second implementation exists would freeze the wrong boundary.
|
||||
|
||||
### Consumer-owned authorization or cleanup events
|
||||
|
||||
Consumer-owned checks invite inconsistent or missing isolation on each new surface. A broadcast cleanup event makes every listener filter every agent and provides no registration disposer. Central authorization plus one owner-scoped effect gives every consumer the same fence and an awaited, removable lifecycle hook.
|
||||
|
||||
### Blocking output or a separate wait tool
|
||||
|
||||
Blocking by default would serialize the parent while background work runs. Waiting without reading would add another model call and schema without returning useful information. `task_output(wait: true)` makes blocking explicit and combines it with result delivery.
|
||||
|
||||
The wait uses the shared deadline primitives but not the generic tool-timeout policy. A wait timeout is a successful observation that returns `[status: running]`; the generic policy would replace it with a timeout error. No tool-call timeout controls task lifetime after a task id has been returned.
|
||||
|
||||
### Runtime-owned output sinks
|
||||
|
||||
A push sink would centralize buffering, but bash already owns bounded buffers, truncation, and spill files behind its executor seam. Pulling formatted deltas preserves that ownership. A durable backend that owns storage may justify revisiting the producer interface.
|
||||
|
||||
### Random ids, promotion, or lifecycle session events
|
||||
|
||||
Authorization, not unguessability, is the access boundary, and ids do not derive filesystem paths; sequential branded ids keep transcripts readable. Foreground-to-background promotion requires a user interaction contract the SDK does not prescribe. Starts, reads, and notices are already logged as tool and context events, so dedicated task session events would duplicate model-visible facts.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit coverage pins preflight atomicity, per-kind ids, stream and final reads, wait timeout and abort races, cancellation, first-wins settlement, listener containment, notice suppression, owner isolation, stale owner instances, owner cleanup, service teardown, and the no-surface fence. Producer tests cover bash process mapping, subagent startup cancellation, terminal mapping, and disposal. Snapshot coverage pins the control-tool schemas and prompt guidance.
|
||||
|
||||
## Consequences
|
||||
|
||||
Bash commands and subagents share one id vocabulary, listing, notice format, prompt habit, and set of control tools. New long-running producers implement execution hooks instead of another registry and tool family. The [tool cookbook](../../../../docs/cookbook/adding-a-tool.md) points producers to this contract.
|
||||
|
||||
Owned background bash now stops with its agent instead of surviving it. Background processes have no executor timeout; callers must kill irrelevant work or rely on owner/service disposal. Stream reads support one consuming reader, completion notices do not wake idle agents, and a producer that returns from `cancel` without settling `done` can still stall teardown. Durable jobs, independent observation cursors, and foreground promotion remain separate designs.
|
||||
@@ -1,7 +1,9 @@
|
||||
# RFC: Reorganize packages into a modular hierarchy
|
||||
# Agent Note: Reorganize packages into a modular hierarchy
|
||||
|
||||
Status: implemented
|
||||
|
||||
The later [fold-stdio-helper](../simplification/2026-07-04-fold-stdio-ui-helper.md) decision superseded the original `support/ui-stdio` placement, and the [redundant-agent removal](../simplification/2026-07-20-remove-stdio-and-echo-agents.md) subsequently removed that surface entirely. The uniform depth-two hierarchy remains the decision owned here.
|
||||
|
||||
## Problem
|
||||
|
||||
`packages/` was flat: 18 packages all sat at `packages/<name>/`, so a package's location said nothing about whether it was core product API, a swappable capability seam, a provider adapter, a product integration, or example/test support. The package README carried a `FIXME(package-hierarchy)` and `scripts/publint-all.ts` a `TODO(package-inventory)` flagging exactly this. Core packages, provider integrations, capability seams, example UI support, and snapshot-only replay support all looked equally foundational.
|
||||
@@ -0,0 +1,146 @@
|
||||
# Agent Note: Bounded recovery for transient LLM request failures
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-llm` can report provider failures either by throwing during adapter dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary tags thrown failures so `dsh-agent-loop` can distinguish them from middleware and result-processing defects, and the loop normalizes both delivery forms into `agent/request-error` after closing the failed step. The default decision is `fail`; `dsh-compact-basic` is the only shipped recovery listener, and it retries a canonical context-window overflow only after compaction proves that the durable surface shrank.
|
||||
|
||||
That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry opens a new numbered step from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate.
|
||||
|
||||
The prior boundary left three narrower gaps.
|
||||
|
||||
- Provider failures retain only a message and usually a code. HTTP status, retry delay, and provider request id are discarded or recoverable only through provider-specific error objects, so generic recovery cannot make or explain a decision without parsing text.
|
||||
- Retry ownership differs by adapter. The hand-written DeepSeek adapter makes one attempt, while pi-ai profiles can enable opaque library retries. Combining hidden transport retries with an `agent/request-error` listener would multiply attempts and omit intermediate failures from the session log.
|
||||
- A recovered failure has no durable status fact. The failed step and chunks remain reconstructable, but an observer cannot tell whether the agent is deliberately backing off, for how long, or why. A long silent wait looks like a stalled loop.
|
||||
|
||||
The goal is bounded recovery from transient failures of the same explicit provider/model request. Provider or model failover, response splicing, and semantic output repair are different problems and have no current consumer.
|
||||
|
||||
## Decision
|
||||
|
||||
### Preserve failure facts without embedding policy
|
||||
|
||||
`@deepseek-ai/dsh-llm` exports one JSON-serializable `LlmFailure` payload:
|
||||
|
||||
```ts ignore-check
|
||||
type ProviderRequestId = Branded<'ProviderRequestId'>
|
||||
|
||||
interface LlmFailure {
|
||||
message: string
|
||||
code: string
|
||||
status?: number
|
||||
providerRetryAfterMs?: number
|
||||
requestId?: ProviderRequestId
|
||||
}
|
||||
```
|
||||
|
||||
`code` remains the provider-neutral machine-routing taxonomy established by `HarnessError`; the new fields are observations from the provider boundary. `ProviderRequestId` is owned and constructed by `dsh-llm`, then serializes as its provider-issued string. The payload deliberately has no `retryable`, `failover`, `partialOutput`, provider, model, phase, or route id fields. Retryability belongs to policy, provider/model are already in the durable request header, and partial output is derived from the failed step's `assistant/chunk` events.
|
||||
|
||||
`LlmError` carries `failure: LlmFailure` and preserves `failure.code === error.code`. `FinishReasonMap.error` and `FinishReasonMap.aborted` carry the same payload instead of parallel failure shapes. An adapter-thrown `Error` keeps its exact object identity: the final-adapter scope associates the normalized facts with that object in call-local sidecar state and rethrows it unchanged; a non-`Error` throw is wrapped as today. `llmFailureOf(stream, error)` retrieves those facts alongside the existing provenance check, while an in-band finish without an error object becomes a new `LlmError`. This preserves listeners that key on error type or identity while giving all final-adapter failures, including unknown SDK exceptions, an `UNKNOWN` terminal payload.
|
||||
|
||||
The agent loop keeps `RequestError` as that exact error object and passes `LlmFailure` as a separate argument to `agent/request-error`; it does not mutate possibly frozen third-party errors. It also uses the payload when converting an in-band finish and when recording an unrecovered `turn/end.reason`.
|
||||
|
||||
Adapters extract structured facts before falling back to message inspection. They validate HTTP status, parse `Retry-After` seconds or dates into a positive finite millisecond delay, brand the provider request id when exposed, and distinguish their own timeout from the caller's abort. Provider-specific codes and messages may refine a mapping, but no recovery listener parses them.
|
||||
|
||||
The initial shared transient-code set is intentionally small: the adapters' existing `RATE_LIMIT` and `SERVER` mappings plus explicit `TIMEOUT` and `TRANSPORT` codes for the two missing remote-failure families. Authentication, quota, invalid request, context overflow, protocol, abort, and unknown failures keep distinct stable codes and are not transient by default. Adding a code requires adapter fixtures and a documented policy decision; it does not require expanding a second failure-class enum.
|
||||
|
||||
### Put retry policy on the existing failed-step seam
|
||||
|
||||
`@deepseek-ai/dsh-llm-retry` is a function plugin that listens to `agent/request-error`. It introduces no service or new loop branch; the agent-loop package changes only the data carried through its existing failed-step recovery control flow.
|
||||
|
||||
The `agent/request-error` seam carries the current `LlmFailure` and an immutable list of prior failures that led to another request attempt in this consecutive recovery sequence. `dsh-llm-retry` counts only prior failures whose codes are in its configured transient set, while `dsh-compact-basic` counts only prior context-overflow failures. A successful model request clears the history. Alternating transient and context-overflow failures therefore consume their owning policy budgets independently; the maximum request count is one plus the sum of the finite budgets of the loaded recovery policies.
|
||||
|
||||
The plugin resolves and validates this deployment configuration at load:
|
||||
|
||||
```ts ignore-check
|
||||
interface Config {
|
||||
maxTransientRetries?: number
|
||||
initialDelayMs?: number
|
||||
maxDelayMs?: number
|
||||
jitterRatio?: number
|
||||
retryableCodes?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
The defaults are two transient retries, a 500 millisecond initial delay, a 10 second delay cap, 10 percent jitter, and the four transient codes above. The count and delay bounds match the conservative edge of the inspected implementations: [OpenCode uses two request retries with 500 ms/10 s bounds](https://github.com/anomalyco/opencode/blob/9976269ab1accfc9f9dc98a4a688c516934de422/%70ackages/llm/src/route/executor.ts#L36-L39), [Pi separates three agent-level retries from provider retries and defaults provider retries to zero](https://github.com/earendil-works/pi/blob/3da591ab74ab9ab407e72ed882600b2c851fae21/%70ackages/coding-agent/docs/settings.md#L139-L147), and [Codex uses finite request/stream budgets plus a five-minute idle timeout](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/model-provider-info/src/lib.rs#L25-L33). Ten percent follows [Codex's bounded jitter](https://github.com/openai/codex/blob/0fb559f0f6e231a88ac02ea002d3ecd248e2b515/codex-rs/codex-client/src/retry.rs#L40-L47). Two retries mean at most three provider requests when no other recovery policy applies. `maxTransientRetries` is a non-negative integer, delays are positive finite numbers with `initialDelayMs <= maxDelayMs`, `jitterRatio` is in `[0, 1]`, and codes are non-empty and unique. These are Cordis config fields rather than hidden constants so deployments can choose different cost and latency budgets.
|
||||
|
||||
For an eligible failure with budget remaining, the one-based transient retry count uses bounded exponential backoff. A valid `providerRetryAfterMs` replaces exponential backoff only when it does not exceed `maxDelayMs`; a longer provider delay causes delegation instead of an earlier retry that violates the provider instruction. Local backoff multiplies by an injected random factor in `[1 - jitterRatio, 1 + jitterRatio]` and clamps the final value to `maxDelayMs`; provider delay is not jittered.
|
||||
|
||||
The plugin owns a lifetime `AbortController` and tracks every active backoff callback. Each wait fuses the waterfall's turn signal with that lifetime signal. Effect cleanup first unregisters the listener, then aborts and awaits the active callbacks; a captured callback whose lifetime signal aborts returns `fail` and can neither retry nor enter the rest of its captured waterfall after disposal. This makes HMR disposal quiescent even though Cordis has already captured the listener.
|
||||
|
||||
Before sleeping, `dsh-llm-retry` appends one non-surface `llm/retry` session event containing the turn, failed step, one-based transient retry number, configured maximum, scheduled delay, and `LlmFailure`. The plugin owns the `SessionEventMap` augmentation; `dsh-session` remains generic persistence and does not absorb the optional policy's vocabulary. The event says what was scheduled, not that the next request completed; cancellation during the delay is subsequently visible on `turn/end`. The event ships only with a production renderer and replay/snapshot coverage, because its purpose is operational state rather than trace collection.
|
||||
|
||||
The listener calls `next()` for a non-transient code, an exhausted policy budget, or an over-cap provider delay. This preserves composition with context-overflow recovery and later policy plugins. It returns `{ action: 'retry' }` only after the delay completes under both signals; turn cancellation and plugin disposal return `fail`, after which the loop's cancellation/disposal checks remain authoritative.
|
||||
|
||||
The agent-spine demo bundle loads the plugin so the shared stdio/TUI, one-shot CLI, and ACP example compositions use the same bounded policy. Library consumers retain explicit plugin composition: omitting the plugin leaves `agent/request-error` at its current fail default.
|
||||
|
||||
### Make one layer own visible attempts
|
||||
|
||||
Adapters perform one provider request per `stream()` call. The pi-ai adapter removes public `maxRetries` and `maxRetryDelayMs` profile fields and disables library retries; the hand-written adapter keeps its current single-attempt behavior. This prevents an SDK budget from multiplying the agent budget and ensures every transient retry is represented by a closed failed step plus `llm/retry`.
|
||||
|
||||
`ctx.llm.stream()` remains the raw one-attempt waterfall. Direct callers such as compaction summarization receive the structured failure but do not gain automatic retry, because they have no agent step boundary or general durable place to separate attempts. A future direct-call consumer may justify a buffering helper that retries only before emitting a chunk; this decision adds no such helper.
|
||||
|
||||
### Bound stalled streams where they can be stopped
|
||||
|
||||
Each adapter exposes a validated `streamIdleTimeoutMs` configuration field with the five-minute prior-art default cited above. The interval is capped at Node's maximum timer delay so it cannot be clamped to one millisecond. It covers each outstanding iterator `next()` from demand to the next valid `StreamChunk`; time a consumer spends between `next()` calls is not provider idle time.
|
||||
|
||||
`@deepseek-ai/dsh-timeout` exposes a rearmable idle-watchdog primitive. One stable local `AbortController` is fused with the caller signal and passed to the transport for the whole adapter call; each outstanding `next()` arms the watchdog, resolution disarms it, and the next demand rearms it. Timeout aborts that stable controller with a capability-owned `TimeoutReason`, and `finally` clears the timer. The adapter classifies its watchdog as `TIMEOUT` and an earlier upstream abort as `ABORTED`. The existing one-shot `deadline()` is not presented as a sliding timer.
|
||||
|
||||
Boundary tests prove termination at both actual transports. The hand-written adapter aborts its fetch/reader, and the pi-ai adapter maps the stable signal through the SDK and proves the SDK closes the response. A timer that merely rejects a consumer promise while leaving the request running does not satisfy the contract.
|
||||
|
||||
### Keep attempts separate in the existing log
|
||||
|
||||
A failed attempt may leave `assistant/chunk` events in its closed step, but it never appends `assistant/message` and never dispatches a tool. A retry opens the next numbered step, reconstructs the request from the durable surface, and produces its own chunks. UIs may render live chunks while a step is open, then mark or clear that transient view when `llm/retry` identifies the failed step or `turn/end` records terminal failure; message derivation continues to ignore the failed chunks.
|
||||
|
||||
If recovery is exhausted, the final failure is stored once on `turn/end.reason` with the structured facts. If transient recovery continues, `llm/retry` is the durable home for that attempt's failure and delay. No standalone final-error event or response-id vocabulary is added.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Automatic provider or model failover. Requests already select one explicit provider and model, and the provider registry deliberately has one adapter owner per provider.
|
||||
- Retrying or continuing after a successful terminal finish, or splicing chunks from two attempts into one assistant message.
|
||||
- Repairing malformed tool arguments, refusals, content filters, or other semantic model output.
|
||||
- Unbounded retries, unattended retry-until-cancelled behavior, circuit breakers, shared provider health, or cross-agent retry budgets.
|
||||
- Changing `llm/stream` into a response lifecycle or adding convenience generation APIs without a production consumer.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Retry inside `llm/stream` or the provider SDK** — rejected because a raw stream has no durable attempt boundary after emitting chunks, hidden SDK retries multiply budgets, and neither path can record each failed attempt consistently.
|
||||
- **Add response start, interrupted, discarded, failed, and committed events to `dsh-llm`** — rejected because the agent log already separates raw chunks, successful messages, and numbered attempts. A second state machine would duplicate ownership without enabling the bounded same-route retry.
|
||||
- **Add logical routes, capability matrices, and failover selection** — rejected because current requests already name provider and model explicitly, one adapter owns each provider, and no current consumer requires automatic fallback or can prove semantic compatibility.
|
||||
- **Put `retryable` or `failover` on `LlmFailure`** — rejected because adapters report facts while deployment policy decides action. The same 429 may be retried in an interactive bundle and rejected in a cost-capped batch.
|
||||
- **Retry forever while the caller remains active** — rejected because it gives one request unbounded cost and latency. Visible status makes bounded waiting understandable; it does not make an unlimited budget safe.
|
||||
- **Log retry status only through the process logger** — rejected because process logs do not reconstruct session behavior and cannot drive replayed UI state.
|
||||
- **Keep only flat codes** — rejected because retry delay and provider request id are structured provider facts, and HTTP status is necessary for diagnosis when different wire failures share one stable code.
|
||||
|
||||
## Verification
|
||||
|
||||
- `LlmFailure` is the single serializable payload for thrown, error-finish, and aborted-finish final-adapter failures; normalization preserves stable code, status, retry delay, branded provider request id, error cause, and caller-abort versus adapter-timeout classification where available.
|
||||
- An adapter-thrown `Error` reaches `agent/request-error` as the exact same object while its sidecar `LlmFailure` reaches the adjacent argument; tests retain the existing identity assertion for extensible and frozen third-party errors.
|
||||
- DeepSeek and pi-ai adapter tests cover representative 400, 401/403, 429, 5xx, connection, malformed/truncated stream, timeout, abort, retry-after seconds/date, request-id, and unknown-SDK-error paths without recovery policy parsing message text.
|
||||
- Pi-ai pins the SDK option to zero retries and performs one observed wire attempt for a retryable provider response; separate tests make removing either boundary fail.
|
||||
- `agent/request-error` carries current failure facts plus immutable prior-retried failure facts; a success clears that history, and alternating transient/context-overflow integration tests prove the two policies consume only their own finite budgets.
|
||||
- `dsh-llm-retry` validates every config field at Loader startup, delegates all ineligible paths with `next()`, and makes at most `maxTransientRetries + 1` provider requests when no other policy applies.
|
||||
- HMR-during-backoff tests prove disposal unregisters the listener, aborts and awaits its captured callbacks, emits no retry decision after disposal, and leaves no timer or promise alive.
|
||||
- Pure unit tests cover transient-code selection, exponential backoff and jitter bounds, valid and over-cap `Retry-After`, exhausted budgets, deterministic timer/random seams, and abort during backoff.
|
||||
- Real agent-loop tests cover failure before chunks, partial chunks then failure, thrown and in-band failures, retry to success in a new step, exhaustion to structured `turn/end.reason`, and composition with `dsh-compact-basic` context-overflow recovery.
|
||||
- The partial-chunk integration test proves failed chunks remain attributed to the failed step, no assistant message or tool side effect is committed for that step, and the successful retry has distinct provenance.
|
||||
- The plugin-owned `llm/retry` event is non-surface, survives JSONL and SQLite round trips, is ignored by message derivation, and drives TUI retraction plus durable discarded-attempt markers in append-only ACP and stdio streams. Keyless snapshots cover scheduling, cancellation, success, and exhaustion.
|
||||
- Idle-watchdog tests prove the stable signal is rearmed only while `next()` is outstanding, disarmed during consumer think time and in `finally`, and classified separately from a total-call deadline and an earlier caller abort; adapter tests prove the signal stops the underlying request rather than merely detaching it.
|
||||
- Direct `ctx.llm.stream()` callers remain single-attempt and receive the same structured failure facts.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every transient recovery attempt is visible as a closed step plus `llm/retry`, and the bounded policy prevents hidden SDK retries from multiplying cost. A retry can still duplicate provider billing even when no chunk arrived; the finite attempt budget limits but cannot remove that risk.
|
||||
- Provider SDKs may hide status or retry headers. Those adapters retain the stable facts they expose and otherwise use a coarse code rather than letting recovery policy parse fragile text.
|
||||
- Durable retry events expand the session protocol and UI state machine. Shipping the event and its consumer together prevents an unused telemetry vocabulary, but later schema changes still require persistence and replay work.
|
||||
- Clearing a failed step's live chunks can visibly retract output. That is preferable to presenting discarded text or partial tool JSON as committed history, and snapshots pin the transition.
|
||||
- Adapter-local idle enforcement stops stalled transports without counting consumer think time. Contract tests at each transport boundary guard against SDK drift.
|
||||
- Multiple recovery plugins add their finite budgets. Their classifiers remain disjoint here; an overlapping classifier would be registration-order policy and must be documented and tested by the plugins that introduce it.
|
||||
|
||||
## Related
|
||||
|
||||
- [Structured error taxonomy](../../implemented/architecture/2026-06-11-structured-error-taxonomy.md) owns stable machine-routable codes and cause chaining.
|
||||
- [Reconstructable requests](../../implemented/architecture/2026-07-05-reconstructable-requests.md) makes provider/model and complete request inputs durable before dispatch.
|
||||
- [Timeout deadline library](../../implemented/architecture/2026-07-06-timeout-deadline-library.md) separates shared deadline classification from capability-owned termination.
|
||||
- [After-call compaction pressure and context-overflow recovery](../../implemented/architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) owns the current closed-step request-recovery seam and bounded overflow retry.
|
||||
- [Provider-routed LLM adapters](../../implemented/architecture/2026-07-14-provider-routed-llm-adapters.md) owns explicit provider/model routing and the one-adapter-per-provider invariant.
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Mandatory `User-Agent` attribution for provider requests
|
||||
# Agent Note: Mandatory `User-Agent` attribution for provider requests
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
LLM provider requests should identify the product making them. That is useful for provider-side support, abuse investigation, compatibility debugging, and traffic analytics. Before this RFC the harness only partially did this: the hand-rolled DeepSeek adapter sent a hand-copied `User-Agent` constant (`packages/llm/llm-deepseek/src/adapter.ts`), while the pi-ai-backed twin sent no harness-owned headers at all (`packages/llm/llm-pi-ai/src/adapter.ts`). New adapters could therefore omit attribution silently, and a library-backed adapter could drift from the hand-rolled adapter even though [the twin-adapter RFC](2026-06-13-twin-llm-adapters.md) exists to keep the provider seam honest across both implementations.
|
||||
LLM provider requests should identify the product making them. That is useful for provider-side support, abuse investigation, compatibility debugging, and traffic analytics. Before this Agent Note the harness only partially did this: the hand-rolled DeepSeek adapter sent a hand-copied `User-Agent` constant (`packages/llm/llm-deepseek/src/adapter.ts`), while the pi-ai-backed twin sent no harness-owned headers at all (`packages/llm/llm-pi-ai/src/adapter.ts`). New adapters could therefore omit attribution silently, and a library-backed adapter could drift from the hand-rolled adapter even though [the twin-adapter Agent Note](2026-06-13-twin-llm-adapters.md) exists to keep the provider seam honest across both implementations.
|
||||
|
||||
The immediate prompt came from OpenRouter's [App Attribution](https://openrouter.ai/docs/app-attribution) docs. OpenRouter creates app pages and rankings from `HTTP-Referer` plus display/category headers. That is valuable, but it is not the HTTP standard for application identity. The risk is adopting OpenRouter's exact header set as if it were universal, then leaking provider-specific headers to direct DeepSeek requests, future OpenAI/Anthropic/Vertex adapters, test servers, or proxies that log unknown fields indefinitely.
|
||||
|
||||
@@ -12,7 +12,7 @@ The immediate prompt came from OpenRouter's [App Attribution](https://openrouter
|
||||
|
||||
- **OpenRouter's mechanism is provider-specific.** Their current docs say app attribution is tracked through `HTTP-Referer` (required), `X-OpenRouter-Title`, and `X-OpenRouter-Categories`; `X-Title` is only accepted for backward compatibility. Their API reference calls the headers optional and says they make the app discoverable on OpenRouter. This is a concrete OpenRouter contract, not an IETF or OpenAI-compatible API standard.
|
||||
- **In agent tooling, `HTTP-Referer` is an OpenRouter-aware convention, not a general agent convention.** It is common enough that OpenRouter SDKs and OpenRouter examples expose it directly, and frameworks that target OpenRouter usually need a way to pass it through. But agent protocols such as ACP negotiate names, versions, and capabilities in their own initialize messages, while model-provider requests still need HTTP-level identity. "Accepted in the agent world" therefore means "recognized by OpenRouter integrations," not "portable across agent runtimes or providers."
|
||||
- **Observed coding agents use product/version `User-Agent` strings, sometimes with environment context.** A non-exhaustive public-code survey found OpenAI Codex building `{originator}/{version} ({os} {os_version}; {arch}) ...` and carrying an `originator` header; Google Gemini CLI sending `GeminiCLI[-clientName]/{version}/{model} ({platform}; {arch}; {surface})` or a Cloud Code VS Code variant; Cline's Codex backend client sending `cline/{version} ({platform} {release}; {arch}) node/{nodeVersion}` plus `originator: cline`; SWE-agent setting `swe-agent/{version}` unless the user already supplied a header; Continue setting `Continue/{version}` for its ClawRouter provider plus `X-Continue-Provider`. Aider also appends `Aider/{version} +{website}` to browser-like user agents for web scraping, but that is not a model-provider request path. The pattern is not one exact format; it is product identity in `User-Agent`, with provider-specific side headers only where a provider/backend asks for them.
|
||||
- **Coding agents identify the product and version in `User-Agent`.** Public implementations vary in environment detail and provider-specific side headers, but product identity is the common contract; there is no universal exact format.
|
||||
- **The standards-track general client identity header is `User-Agent`.** RFC 9110 section 10.1.5 defines `User-Agent` as the user-agent software identity, says it is used for interoperability reports and analytics, and says a user agent SHOULD send it on each request unless configured not to. This is the only standard header that directly matches "what product is making this HTTP request."
|
||||
- **`Referer` is standard, but OpenRouter's `HTTP-Referer` is not the standard field.** RFC 9110 section 10.1.3 defines `Referer` as the URI from which the target URI was obtained and spends significant text on privacy restrictions. OpenRouter instead asks for `HTTP-Referer`, using it as an app URL identifier. That name and meaning are OpenRouter-specific even though it resembles the CGI environment variable form of the standard `Referer` header.
|
||||
- **`From` is standard but not suitable as a mandatory default.** RFC 9110 section 10.1.2 defines `From` as an email address for the human responsible for a user agent. Robotic agents SHOULD send it so servers can contact an operator, but non-robotic agents should not send it without explicit user configuration because of privacy and security policy concerns. The harness can support an operator contact later, but must not invent one or require it globally.
|
||||
@@ -24,11 +24,11 @@ The immediate prompt came from OpenRouter's [App Attribution](https://openrouter
|
||||
|
||||
Provider request attribution is mandatory at the LLM adapter boundary, using the standard `User-Agent` header only. The rule: every product LLM adapter sends a static, non-secret application identity on every provider HTTP request, and every adapter has tests proving that `User-Agent` reaches the wire (a mock server asserting received headers; for a library-backed adapter, the library's header hook feeding the same mock-server assertion).
|
||||
|
||||
Do **not** implement OpenRouter app attribution in this RFC. `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, and `X-OpenRouter-Categories` are OpenRouter-specific product-surface headers, not provider-neutral model-request attribution. They can be proposed later by an OpenRouter adapter or explicit OpenRouter mode, with its own privacy/product decision, tests, and docs. Until then, even requests pointed at OpenRouter send only the shared `User-Agent` attribution from this RFC.
|
||||
Do **not** implement OpenRouter app attribution in this Agent Note. `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, and `X-OpenRouter-Categories` are OpenRouter-specific product-surface headers, not provider-neutral model-request attribution. They can be proposed later by an OpenRouter adapter or explicit OpenRouter mode, with its own privacy/product decision, tests, and docs. Until then, even requests pointed at OpenRouter send only the shared `User-Agent` attribution from this Agent Note.
|
||||
|
||||
The provider-neutral identity is owned by `dsh-llm` (`packages/llm/llm/src/attribution.ts`), not by individual adapters. `AppIdentity` contains only public product facts needed to build `User-Agent`, and the default `APP_IDENTITY` settles the values the proposal left open:
|
||||
|
||||
- product token for `User-Agent`: `deepseek-harness` (continuity with the pre-RFC wire value and the repo/org identity)
|
||||
- product token for `User-Agent`: `deepseek-harness` (continuity with the pre-Agent Note wire value and the repo/org identity)
|
||||
- version: read from the owning package's manifest via `createRequire`, never a hand-copied constant
|
||||
- app URL: `https://github.com/deepseek-ai/deepseek-harness-sdk` - the planned public home; a `FIXME` in `attribution.ts` blocks release until that repository actually exists
|
||||
|
||||
@@ -40,10 +40,10 @@ Wire mapping (`attributionHeaders`; header names lowercase in code - HTTP field
|
||||
|---|---|
|
||||
| All HTTP-based adapters | `User-Agent: {product}/{version} (+{url})` - the parenthesized `+url` comment stays within RFC 9110's conservative product/comment syntax. |
|
||||
| Direct DeepSeek endpoint | `User-Agent`; do not send OpenRouter-only headers unless DeepSeek documents an equivalent contract. |
|
||||
| OpenRouter endpoints | `User-Agent` only for now. Do not send `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, or `X-OpenRouter-Categories` under this RFC. |
|
||||
| Future providers | `User-Agent` only unless a later provider-specific RFC accepts additional headers. Do not reuse `HTTP-Referer` by analogy. |
|
||||
| OpenRouter endpoints | `User-Agent` only for now. Do not send `HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, or `X-OpenRouter-Categories` under this Agent Note. |
|
||||
| Future providers | `User-Agent` only unless a later provider-specific Agent Note accepts additional headers. Do not reuse `HTTP-Referer` by analogy. |
|
||||
|
||||
Endpoint detection is not part of this RFC because no endpoint-specific mapping is accepted here. If OpenRouter support lands later, detection must be explicit: either a dedicated OpenRouter provider package or an explicit `provider: 'openrouter'` / `attributionTarget: 'openrouter'` config, not arbitrary path fragments or model names.
|
||||
Endpoint detection is not part of this Agent Note because no endpoint-specific mapping is accepted here. If OpenRouter support lands later, detection must be explicit: either a dedicated OpenRouter provider package or an explicit `provider: 'openrouter'` / `attributionTarget: 'openrouter'` config, not arbitrary path fragments or model names.
|
||||
|
||||
## Verification
|
||||
|
||||
@@ -53,19 +53,19 @@ The landed contract:
|
||||
- A shared helper (`attributionHeaders` / `userAgent`) constructs the app identity and the standard `User-Agent` value from package metadata, so adapters do not hand-copy version constants.
|
||||
- `dsh-llm-deepseek` sends the shared `User-Agent` on every request and its mock-server suite asserts the exact value.
|
||||
- `dsh-llm-pi-ai` sends the same `User-Agent` through pi-ai's `StreamOptions.headers` hook and its mock-server suite asserts the exact value.
|
||||
- No adapter sends OpenRouter-specific attribution headers (`HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, `X-OpenRouter-Categories`) as part of this RFC.
|
||||
- No adapter sends OpenRouter-specific attribution headers (`HTTP-Referer`, `X-OpenRouter-Title`, `X-Title`, `X-OpenRouter-Categories`) as part of this Agent Note.
|
||||
- No app-attribution field carries secrets, local paths, session ids, prompt text, model output, user email, or per-user stable identifiers.
|
||||
- The adapter READMEs state the `User-Agent` attribution policy and explicitly avoid documenting OpenRouter app attribution as implemented behavior.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**OpenRouter app attribution now.** Rejected for this RFC. Sending `HTTP-Referer` plus `X-OpenRouter-Title` would satisfy OpenRouter rankings, but those headers are a provider-specific product feature, not the provider-neutral model-request attribution this RFC is trying to standardize. Supporting them should be an explicit OpenRouter adapter/mode decision later, not hidden inside the first shared attribution helper.
|
||||
**OpenRouter app attribution now.** Rejected for this Agent Note. Sending `HTTP-Referer` plus `X-OpenRouter-Title` would satisfy OpenRouter rankings, but those headers are a provider-specific product feature, not the provider-neutral model-request attribution this Agent Note is trying to standardize. Supporting them should be an explicit OpenRouter adapter/mode decision later, not hidden inside the first shared attribution helper.
|
||||
|
||||
**OpenRouter headers everywhere.** Rejected. It would treat a custom OpenRouter contract as a universal standard and send fields with misleading semantics to providers that did not ask for them. It also risks using `HTTP-Referer` as a generic app URL field even though standard HTTP already has `User-Agent` for product identity and `Referer` for a different browsing-context concept.
|
||||
|
||||
**Only provider account/project identity.** Rejected. Organization/project headers, API keys, cloud accounts, and billing projects identify who pays or owns the request, not which application is sending traffic. They also expose no public app title/category and do not help gateways like OpenRouter build app rankings.
|
||||
|
||||
**End-user `user`/`metadata` fields.** Rejected for this RFC. Those are valuable for abuse monitoring and customer support but describe the human or tenant behind a request. App attribution must be static product identity and safe to send on every request.
|
||||
**End-user `user`/`metadata` fields.** Rejected for this Agent Note. Those are valuable for abuse monitoring and customer support but describe the human or tenant behind a request. App attribution must be static product identity and safe to send on every request.
|
||||
|
||||
**Config-only opt-in attribution.** Rejected. A default-off setting is exactly how adapters keep drifting. The policy is mandatory default attribution with overrideable public values, not optional attribution.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Web capability seam - stable tools over multiple providers
|
||||
# Agent Note: Web capability seam - stable tools over multiple providers
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -14,7 +14,7 @@ There is also a provider-selection question. Existing `tool-bash` and `tool-fs`
|
||||
|
||||
## Decision
|
||||
|
||||
Web access is a first-class capability seam following [the capability-seam RFC](../../implemented/architecture/2026-06-13-capability-seams.md):
|
||||
Web access is a first-class capability seam following [the capability-seam Agent Note](2026-06-13-capability-seams.md):
|
||||
|
||||
1. `@deepseek-ai/dsh-web` (`packages/web/web`) owns `ctx.web`, provider registration, provider selection, shared request/result vocabulary, and web-specific errors.
|
||||
2. Provider packages implement concrete backends and register capabilities with `ctx.web`, for example `@deepseek-ai/dsh-web-search-exa`, `@deepseek-ai/dsh-web-search-perplexity`, `@deepseek-ai/dsh-web-search-deepseek`, and `@deepseek-ai/dsh-web-fetch-local`.
|
||||
@@ -22,7 +22,7 @@ Web access is a first-class capability seam following [the capability-seam RFC](
|
||||
|
||||
Providers do not register tools. Providers register capabilities. `dsh-tool-web` is the only owner of model-facing names, descriptions, prompt guidance, JSON schemas, and presentation.
|
||||
|
||||
Search and fetch are separate capabilities and separate model-facing tools, but they are deliberately one seam. `ctx.web` is a single web-access middle layer between provider packages on one side and the tool consumer on the other: one service to inject, one provider-selection policy owner, one abort/error vocabulary, one place a product configures "how this harness reaches the web." The two halves do not share a request schema and have no shared business logic — search normalizes provider-backed discovery into a portable result with optional answer text and citeable sources, while fetch retrieves a concrete public HTTP(S) URL and returns a status code plus bounded decoded content — but they are parallel registries on one capability surface, not two surfaces. The cost is a `WebService` whose registry/exec methods come in `Search`/`Fetch` pairs; that parallelism is intentional, not a missed extraction. Splitting into `dsh-search` and `dsh-fetch` is the rejected alternative below.
|
||||
Search and fetch are separate tools but one web-access seam. `ctx.web` owns provider selection, abort/error vocabulary, and deployment configuration for both parallel registries. Their request schemas and provider logic remain separate; the shared service is the product boundary for reaching the web.
|
||||
|
||||
`dsh-tool-web` registers model-facing web tools when the product has enabled those tools and the `ctx.web` seam is present. Backend availability is an execution-time concern, not a schema-registration concern:
|
||||
|
||||
@@ -32,7 +32,7 @@ Search and fetch are separate capabilities and separate model-facing tools, but
|
||||
|
||||
This keeps the model schema stable without making plugin load order, credential state, or HMR timing part of the model-facing contract. If web search is enabled but no usable search provider exists, `web_search` remains visible and execution fails with a structured `WebError` such as `WEB_PROVIDER_UNAVAILABLE` or `WEB_PROVIDER_CONFIGURED_UNAVAILABLE`. If a provider appears after `dsh-tool-web`, the next execution can use it without changing the schema. If a provider disappears mid-call, execution fails with a structured `WebError` instead of silently choosing another provider or falling through to `UNKNOWN_TOOL`.
|
||||
|
||||
The seam deliberately exposes no observation surface — no registry-change event and no aggregated capability-status query. Unavailability is a fact a caller observes by executing: `search()`/`fetch()` resolve the provider at call time and throw the structured `WebError` that names what failed. [The observation-surface RFC](../simplification/2026-07-04-drop-unconsumed-web-observation-surface.md) records that judgment: derived-on-call selection and enablement-based registration leave no consumer that needs a change signal or an availability probe distinct from executing and routing the error, and a future provider-status panel reintroduces the smallest signal or query it actually consumes.
|
||||
The seam deliberately exposes no observation surface — no registry-change event and no aggregated capability-status query. Unavailability is a fact a caller observes by executing: `search()`/`fetch()` resolve the provider at call time and throw the structured `WebError` that names what failed. [The observation-surface Agent Note](../simplification/2026-07-04-drop-unconsumed-web-observation-surface.md) records that judgment: derived-on-call selection and enablement-based registration leave no consumer that needs a change signal or an availability probe distinct from executing and routing the error, and a future provider-status panel reintroduces the smallest signal or query it actually consumes.
|
||||
|
||||
## Package topology
|
||||
|
||||
@@ -64,9 +64,9 @@ flowchart LR
|
||||
toolWeb -->|ctx.tools.register| webFetch["tool: web_fetch"]
|
||||
```
|
||||
|
||||
`@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, the provider status type, and error codes. It does not import tool, agent, session, LLM, or provider packages.
|
||||
`@deepseek-ai/dsh-web` depends only on Cordis and low-level harness support. It declares `ctx.web`, provider interfaces, request/result types, the provider availability contract, and error codes. It does not import tool, agent, session, LLM, or provider packages.
|
||||
|
||||
Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credentials, endpoint config, provider-specific request mapping, provider-specific response parsing, and provider-specific error translation into `WebError`. They issue network requests with platform-native `fetch` at the repo's Node floor, mirroring `@deepseek-ai/dsh-llm-deepseek`'s adapter, NOT a cordis HTTP-client service (`ctx.http`/`@cordisjs/plugin-http`) — even where a Perplexity provider's request is shaped like an OpenAI-compatible chat completion, that wire shape is a provider-private detail and does not make the provider depend on `ctx.llm`. A provider does NOT own the `ctx.web` key (two search providers cannot both own it): like `dsh-llm-deepseek`, each provider package is a function/namespace plugin (`inject: ['web']`) whose `apply` constructs the backend and calls `ctx.web.registerSearchProvider` / `registerFetchProvider`. `@deepseek-ai/dsh-web` is the `export default` service that owns the key.
|
||||
Provider packages depend only on `dsh-web` and Cordis. They own credentials, endpoints, wire mapping, parsing, and `WebError` translation, using platform `fetch`. Each provider injects the shared service and registers a backend; only `dsh-web` owns the `ctx.web` key. Provider-private protocol shapes do not create dependencies on `ctx.llm` or a Cordis HTTP service.
|
||||
|
||||
`@deepseek-ai/dsh-tool-web` depends on `@deepseek-ai/dsh-web`, `@deepseek-ai/dsh-tools`, `@deepseek-ai/dsh-system-prompt`, and Cordis. It never imports concrete provider packages.
|
||||
|
||||
@@ -77,52 +77,42 @@ Provider packages depend on `@deepseek-ai/dsh-web` and Cordis. They own credenti
|
||||
```ts
|
||||
interface WebSearchProvider {
|
||||
readonly id: string
|
||||
status(): WebProviderStatus
|
||||
search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult>
|
||||
available(): boolean
|
||||
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>
|
||||
}
|
||||
|
||||
interface WebFetchProvider {
|
||||
readonly id: string
|
||||
status(): WebProviderStatus
|
||||
fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult>
|
||||
available(): boolean
|
||||
fetch(request: WebFetchRequest, signal?: AbortSignal): Promise<WebFetchResult>
|
||||
}
|
||||
|
||||
interface WebService {
|
||||
registerSearchProvider(provider: WebSearchProvider): () => void
|
||||
registerFetchProvider(provider: WebFetchProvider): () => void
|
||||
|
||||
search(request: WebSearchRequest, exec?: WebExecContext): Promise<WebSearchResult>
|
||||
fetch(request: WebFetchRequest, exec?: WebExecContext): Promise<WebFetchResult>
|
||||
}
|
||||
|
||||
interface WebExecContext {
|
||||
readonly signal?: AbortSignal
|
||||
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>
|
||||
fetch(request: WebFetchRequest, signal?: AbortSignal): Promise<WebFetchResult>
|
||||
}
|
||||
```
|
||||
|
||||
`WebExecContext` is execution control, not business input. It carries only `signal`, so `tool-web` propagates turn cancellation, tool timeout, and agent disposal into provider network requests, SSE readers, and expensive decoding. It does not pass `ToolExecution` through the seam — that would make `dsh-web` depend on `dsh-tools`.
|
||||
The optional signal is execution control, not business input: `tool-web` passes `exec.signal` directly so turn cancellation, tool timeout, and agent disposal reach provider network requests, stream readers, and expensive decoding. The seam does not pass `ToolExecution` through — that would make `dsh-web` depend on `dsh-tools`.
|
||||
|
||||
Provider ids are stable strings and unique within their capability kind. Registering a duplicate search provider id or duplicate fetch provider id fails rather than silently replacing the old provider. Provider registration returns a disposer and follows the existing `ctx.tools.register()` / `ctx.systemPrompt.section()` pattern: the mutation is wrapped in `ctx.effect()` so the registration is torn down with the contributing fiber.
|
||||
|
||||
## Provider status and selection
|
||||
## Provider availability and selection
|
||||
|
||||
Provider status and capability selection are separate concepts, but both stay minimal. A provider reports only whether that concrete implementation is usable by cheap local checks such as credential presence or parseable endpoint config. A provider `status()` must not make network calls.
|
||||
Provider availability and capability selection are separate concepts, but both stay minimal. A provider reports only whether that concrete implementation is usable by cheap local checks such as credential presence or parseable endpoint config. A provider `available()` must not make network calls.
|
||||
|
||||
`LlmService` has no status type at all: availability is expressed as registry membership plus a resolution-time throw. `ctx.web` follows the same discipline. The seam exposes no aggregated capability-status query — `search()` / `fetch()` derive the selection on each call from the configured provider id, the registered providers, and each provider's cheap local `status()`, and a selection failure is the structured `WebError` thrown at execution time, whose code answers "in which broad category does this capability fail" and whose message answers "exactly which provider/ids/reason." A caller that needs to know whether a capability can run executes and routes that error; nothing is stored as mutable service state.
|
||||
`LlmService` has no status type at all: availability is expressed as registry membership plus a resolution-time throw. `ctx.web` follows the same discipline. The seam exposes no aggregated capability-status query — `search()` / `fetch()` derive the selection on each call from the configured provider id, the registered providers, and each provider's cheap local `available()` boolean, and a selection failure is the structured `WebError` thrown at execution time. A caller that needs to know whether a capability can run executes and routes that error; nothing is stored as mutable service state.
|
||||
|
||||
`WebProviderStatus` is an input to selection, not a health system. `tool-web` never calls a provider's `status()` directly — its only path into the seam is `search()` / `fetch()` — so selection policy has one owner.
|
||||
|
||||
```ts
|
||||
type WebProviderStatus =
|
||||
| { readonly available: true }
|
||||
| { readonly available: false; readonly reason: 'missing-credential' | 'misconfigured' }
|
||||
```
|
||||
The boolean is an input to selection, not a health system. `tool-web` never calls a provider's `available()` directly — its only path into the seam is `search()` / `fetch()` — so selection policy has one owner.
|
||||
|
||||
Selection must not depend on registration order. Cordis load order, config ordering, and HMR timing are not product semantics.
|
||||
|
||||
| Situation | Execution behavior |
|
||||
|---|---|
|
||||
| A configured provider id is registered and `status().available === true` | runs that provider |
|
||||
| A configured provider id is registered and `available() === true` | runs that provider |
|
||||
| A configured provider id is not registered | fails with `WEB_PROVIDER_CONFIGURED_MISSING` |
|
||||
| A configured provider id is registered but unavailable | fails with `WEB_PROVIDER_CONFIGURED_UNAVAILABLE` |
|
||||
| No provider id is configured and exactly one provider for that kind is registered and available | runs that single provider |
|
||||
@@ -184,8 +174,6 @@ interface WebSearchRequest {
|
||||
}
|
||||
|
||||
interface WebSearchResult {
|
||||
readonly providerId: string
|
||||
readonly query: string
|
||||
readonly content?: string
|
||||
readonly sources: readonly WebSearchSource[]
|
||||
readonly truncated: boolean
|
||||
@@ -212,20 +200,17 @@ The `web_fetch` implementation is an anonymous public HTTP(S) fetch provider, `l
|
||||
The seam request stays smaller than OpenCode's model-facing tool:
|
||||
|
||||
- `url`: required HTTP(S) URL.
|
||||
- `timeoutMs`: optional positive number capped by the provider.
|
||||
|
||||
The seam request deliberately does not include `format`, `prompt`, or provider-specific extraction controls. `format` is a presentation decision over a fetched resource; `prompt` is a higher-level LLM summarization instruction; extraction APIs such as Firecrawl, Exa, Tavily, or Parallel may not expose a concrete HTTP response. If the product later needs provider-backed page extraction, that is a separate `web_extract` capability or a deliberate widening of this seam — extract semantics are never smuggled into `web_fetch` by making every HTTP field optional.
|
||||
The seam request deliberately does not include a per-call timeout, `format`, `prompt`, or provider-specific extraction controls. Cancellation is the direct optional execution signal, while the fetch provider owns one deployment-configured timeout backstop. `format` is a presentation decision over a fetched resource; `prompt` is a higher-level LLM summarization instruction; extraction APIs such as Firecrawl, Exa, Tavily, or Parallel may not expose a concrete HTTP response. If the product later needs provider-backed page extraction, that is a separate `web_extract` capability or a deliberate widening of this seam — extract semantics are never smuggled into `web_fetch` by making every HTTP field optional.
|
||||
|
||||
HTTP status is part of the fetched resource state, not automatically a tool failure. A successful network fetch of a `404` or `500` response returns `WebFetchResult` with the status code and a bounded decoded body when the content type is supported. `WebError` is for failures to safely retrieve or represent the resource: invalid or blocked URL, redirect policy violation, timeout, abort, response too large, unsupported content type, provider failure, or network failure.
|
||||
|
||||
```ts
|
||||
interface WebFetchRequest {
|
||||
readonly url: string
|
||||
readonly timeoutMs?: number
|
||||
}
|
||||
|
||||
interface WebFetchResult {
|
||||
readonly providerId: string
|
||||
readonly url: string
|
||||
readonly statusCode: number
|
||||
readonly body: WebFetchBody
|
||||
@@ -239,7 +224,7 @@ type WebFetchBody =
|
||||
|
||||
`WebFetchResult.url` is the final URL after allowed redirects. The request URL is already present in `WebFetchRequest`, so there is no separate `requestedUrl`/`finalUrl` pair.
|
||||
|
||||
`WebFetchBody` is a CLOSED discriminated union owned by `dsh-web`, not a merge-extensible map. The merge-extensible pattern (`ContentBlockMap`) exists for variants that independent plugins introduce and the seam cannot foresee; body kinds are not that — `dsh-web` declares the kind, the fetch provider decodes it, and `dsh-tool-web` renders it, so a new kind is a coordinated change across three known packages, not a plugin extension. Keeping it closed buys compile-time exhaustiveness: consumers `switch` on `kind` ending in `default: assertNever(body, …)`, so adding a kind breaks compilation at every consumer that must render it (e.g. `tool-web`'s `html`→markdown vs `text` passthrough) until that arm is written. Each arm stays its own object literal even when the fields coincide today, leaving room for arm-specific fields (a future `pdf` body's `pageCount`, a `json` body's parsed value) without reshaping the type. Since the harness is unreleased, extending this closed union later is free (no migration, no compat shim).
|
||||
`WebFetchBody` is a closed discriminated union because body kinds require coordinated changes to the seam, provider, and tool rather than independent plugin extension. Exhaustive switches make a new kind fail compilation at every renderer until handled. Separate object arms leave room for kind-specific fields.
|
||||
|
||||
The provider owns safe resource retrieval: URL validation, HTTP transport, redirect policy, timeout, abort propagation, byte caps, charset decoding, content-type classification, and binary rejection. `dsh-tool-web` owns presentation: HTML-to-markdown, HTML-to-text, truncation formatting for the model, and future summaries.
|
||||
|
||||
@@ -257,11 +242,11 @@ SSRF / private-network protection (blocking private, loopback, link-local, multi
|
||||
|
||||
`dsh-tool-web` owns two `ToolDefinition`s: `web_search` and `web_fetch`. It owns model-facing JSON schemas, snake_case argument names, prompt sections, result rendering to `ContentBlock[]`, `presentCall`, and `presentResult`.
|
||||
|
||||
`dsh-tool-web` must not enumerate providers or call provider `status()` directly. Its only path into the seam is `ctx.web.search()` / `ctx.web.fetch()`. That keeps provider selection in one layer; otherwise the tool package could decide one provider is usable while execution resolves a different state.
|
||||
`dsh-tool-web` must not enumerate providers or call provider `available()` directly. Its only path into the seam is `ctx.web.search()` / `ctx.web.fetch()`. That keeps provider selection in one layer; otherwise the tool package could decide one provider is usable while execution resolves a different state.
|
||||
|
||||
Tool registration is a minimal stable sync: on plugin startup the `dsh-tool-web` `Config` (`search?: boolean`, `fetch?: boolean`, both default `true`) enables or disables each web tool; an enabled tool is registered with a fiber-scoped disposer via the effect-based registry; neither tool is disposed merely because its selected provider is missing, unusable, or ambiguous; disposing the `tool-web` fiber tears down its registrations automatically.
|
||||
|
||||
Provider status changes affect execution results and diagnostics, not whether the model-facing schema exists. If a product wants no web tools at all, it disables `dsh-tool-web` or the individual web tool in config; if it wants web tools but the backend is misconfigured, the model sees a structured tool error at execution time.
|
||||
Provider availability changes affect execution results and diagnostics, not whether the model-facing schema exists. If a product wants no web tools at all, it disables `dsh-tool-web` or the individual web tool in config; if it wants web tools but the backend is misconfigured, the model sees a structured tool error at execution time.
|
||||
|
||||
The prompt guidance explains the semantic split — `web_search` for discovery and current information, `web_fetch` when the model needs the content of a specific URL — and the prompt and tool result tell the model to cite relevant URLs with markdown links.
|
||||
|
||||
@@ -291,7 +276,7 @@ Tool execution lets these errors flow through `ToolRegistry.execute()`, which al
|
||||
|
||||
## Testing
|
||||
|
||||
Each layer is pinned at its own seam: the registry/selection/truncation/abort contract and the `WebError` codes in `dsh-web`; per-provider request/response mapping over recorded fixtures (Perplexity fixtures include URL-only citations so the optional source fields stay honest) plus a self-skipping with-key smoke per real provider; real local-HTTP behavior in `web-fetch-local`; and enablement-driven registration, structured execution errors, and result formatting through the real tool registry in `dsh-tool-web`. A real-Loader smoke guards the two export shapes ([postmortem 0001](../../../postmortem/0001-acp-default-export-drops-inject.md)): `dsh-web` is a default-exported service, while the providers and `tool-web` are namespace plugins where a stray `export default` would drop `inject`.
|
||||
Each layer is pinned at its own seam: the registry/selection/truncation/abort contract and the `WebError` codes in `dsh-web`; per-provider request/response mapping over recorded fixtures (Perplexity fixtures include URL-only citations so the optional source fields stay honest) plus a self-skipping with-key smoke per real provider; real local-HTTP behavior in `web-fetch-local`; and enablement-driven registration, structured execution errors, and result formatting through the real tool registry in `dsh-tool-web`. A real-Loader smoke guards the two export shapes ([postmortem 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)): `dsh-web` is a default-exported service, while the providers and `tool-web` are namespace plugins where a stray `export default` would drop `inject`.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Make `dsh-fs-policy` an event-gate plugin, not a method interface
|
||||
# Agent Note: Make `dsh-fs-policy` an event-gate plugin, not a method interface
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
[The split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md) put `ctx.fileContext` between the model-facing tools and the `ctx.fs` provider: `dsh-tool-fs` injects `fileContext` and routes every `read`/`write`/`edit` through its methods. That makes `fileContext` **in-path and mandatory**. The tool cannot reach `ctx.fs` without it, the policy layer owns the fs I/O and the read windowing, and a deployment that does not want observed-state policy cannot simply drop the package — `dsh-tool-fs` would fail to resolve `ctx.fileContext`.
|
||||
[The split-fs-seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) put `ctx.fileContext` between the model-facing tools and the `ctx.fs` provider: `dsh-tool-fs` injects `fileContext` and routes every `read`/`write`/`edit` through its methods. That makes `fileContext` **in-path and mandatory**. The tool cannot reach `ctx.fs` without it, the policy layer owns the fs I/O and the read windowing, and a deployment that does not want observed-state policy cannot simply drop the package — `dsh-tool-fs` would fail to resolve `ctx.fileContext`.
|
||||
|
||||
This couples three things that should be separable:
|
||||
|
||||
@@ -29,7 +29,7 @@ provider seam dsh-fs ctx.fs: text IO + ATOMIC mutation primitives who
|
||||
provider dsh-fs-local local implementation of ctx.fs
|
||||
```
|
||||
|
||||
The model is **additive, not subtractive**: `ctx.fs` on its own is a complete, unconstrained text-storage seam — `read` reads, `write` unconditionally creates-or-overwrites, `edit` unconditionally replaces literal text in the current content. There is no "先读后写", no version check, nothing to remove; the bare provider just does the I/O atomically. `dsh-fs-policy` is a plugin that *adds* constraints on top: observed-state, read-before-edit, and "write/edit must be based on the version you read". So removing `dsh-fs-policy` does not break `dsh-tool-fs` at the service-injection boundary; it removes the policy gate and leaves the bare provider behavior. The intended deployment stance is that a config loading the fs tools also loads `dsh-fs-policy`, so the user-facing behavior and prompt discipline are read-before-write/edit (the `coding-agent` and `acp-agent` demos wire the full stack). The bare-provider mode exists because the tool should not be method-coupled to the policy plugin, not because an unconstrained filesystem is the normal product stance.
|
||||
The model is additive: bare `ctx.fs` performs atomic, unconstrained text I/O, while `dsh-fs-policy` adds observed state, read-before-edit, and version guards. Removing the policy therefore leaves the tools usable but unconstrained. Shipped agent configs load the policy; the bare mode exists to keep policy optional at the service boundary, not as the normal deployment stance.
|
||||
|
||||
`dsh-tool-fs` no longer injects `fileContext`. It injects `fs` and `tools`/`systemPrompt`.
|
||||
|
||||
@@ -68,9 +68,7 @@ The events live in `@deepseek-ai/dsh-fs`, not in `dsh-fs-policy`. This is forced
|
||||
|
||||
These events carry existing `dsh-fs` vocabulary (`FsTarget`, `FsVersion`, `FsWriteIntent`) plus an opaque actor — not model-facing concepts (no line windows, numbered lines, or rendered footers leak down).
|
||||
|
||||
**The two `fs/*` decision events are single-slot decision points, NOT a composable interception chain.** A waterfall listener that does not call `next()` short-circuits the rest of the chain (verified in [vendor/cordis/src/events.ts](../../../../vendor/cordis/src/events.ts) — `waterfall` runs listeners around the final `next` thunk, and a listener that returns without calling `next()` reaches neither later listeners nor the tool's default thunk). `dsh-fs-policy` fully decides the write/edit expectation and does not call `next()`, so it occupies that one decision slot in the default deployment. This is deliberate: "what version basis does this mutation guard against" is a single decision, not an accumulation. The names (`fs/write-intent`, `fs/edit-intent`) say "produce the value", not "authorize", so they do not imply a stackable authorization chain. Genuinely composable interception (permission, audit, sandbox) belongs on the existing `tools/execute` waterfall, which every tool call already flows through — not on this fs version-decision slot.
|
||||
|
||||
**The occupant is decided by registration order — first-registered (or `prepend`ed) wins.** cordis dispatches waterfall listeners in registration order (`push`, or `unshift` for `prepend` — [vendor/cordis/src/events.ts](../../../../vendor/cordis/src/events.ts)), and the first non-`next()` decider short-circuits the rest. So the slot is **first-wins**, and `dsh-fs-policy` owning it rests on the default deployment convention: it is the decider registered for these events. The event shape does NOT itself guarantee "an unread edit is rejected" — a plugin that registers a looser `fs/edit-intent` decider BEFORE `dsh-fs-policy` (or with `prepend`) would decide first and bypass the `FS_NOT_OBSERVED` gate. That is the inherent property of a first-wins single slot, stated here so it is not mistaken for an enforced invariant. This RFC does not add a multi-policy composition mechanism; the implementation requirement is that `dsh-tool-fs` dispatches these waterfalls on every write/edit path and that a config wiring the fs tools loads `dsh-fs-policy` as the policy decider.
|
||||
**The two `fs/*` decision events are single-slot, first-wins waterfalls.** `dsh-fs-policy` returns without calling `next()`, so it owns the slot in the default deployment; a listener registered earlier or with `prepend` would replace that policy. Permission, audit, and sandbox concerns remain on the composable `tools/execute` waterfall.
|
||||
|
||||
The actor is typed `object` in `dsh-fs` — a pure opaque carrier the provider seam never reads or narrows. The owner-derivation (`actor.agent?.session`) and the `{ agent?: { session? } }` structural shape stay entirely inside `dsh-fs-policy`, which narrows the `object` actor to that shape in its listeners. `dsh-fs` owns the event names and the fs vocabulary; it does NOT own the policy layer's runtime owner structure.
|
||||
|
||||
@@ -124,7 +122,7 @@ The tool keeps its model-facing schemas (`read`/`write`/`edit`, byte-for-byte un
|
||||
|
||||
The tool passes `exec` (the tool-execution context) as the `actor` argument on every dispatch, so `dsh-fs-policy` can derive its observed-state owner. The tool does not know whether the policy plugin is present: it always provides the bare default behavior in the `next` thunk, and `dsh-fs-policy` short-circuits the thunk before it runs in the default deployment.
|
||||
|
||||
**`fs/observed` fires AFTER the mutation already succeeded**, via a plain `ctx.emit`. The event contract is intentionally narrow: an `fs/observed` listener MUST be synchronous and side-effect-only — `dsh-fs-policy`'s listener is a `WeakMap.set`, which cannot throw under normal operation and returns no promise. The tool does not guard the emit, so a listener that violates the contract by throwing would surface as the tool's `isError` result ([tools/index.ts](../../../../packages/core/tools/src/index.ts) — `ToolRegistry.execute` catches a tool throw into an error result) — reporting failure for a write/edit that actually happened. That is the price of keeping the event a plain fire-and-forget recorder: cordis `emit` does not await listener promises, so async or fallible audit/telemetry/listener work does not belong on this event. If layered or async observation is ever wanted, that is a new event with its own dispatch story.
|
||||
**`fs/observed` fires after a successful operation.** Its listeners must be synchronous, non-throwing recorders; the tool does not guard the plain emit, so a throwing listener would report failure after a mutation already succeeded. Async or fallible observation needs a separate event contract.
|
||||
|
||||
## Policy plugin contract (`dsh-fs-policy`)
|
||||
|
||||
@@ -150,15 +148,15 @@ Both mutations are still atomic (the backend's per-target lock is unconditional)
|
||||
|
||||
## Supersedes
|
||||
|
||||
This amends — does not reverse — [the split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md). The four-layer split, the provider contract, and the freshness *policy* are all kept. What changes is the **coupling between the tool and the policy layer**: a mandatory method service became a plugin-owned event gate, and the fs I/O + read windowing moved from `fileContext` up into `dsh-tool-fs`. The split-fs-seam RFC's description of `dsh-tool-fs` injecting `fileContext` and of `fileContext` owning `read`/`write`/`edit` was updated to match in the same change.
|
||||
This amends — does not reverse — [the split-fs-seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md). The four-layer split, the provider contract, and the freshness *policy* are all kept. What changes is the **coupling between the tool and the policy layer**: a mandatory method service became a plugin-owned event gate, and the fs I/O + read windowing moved from `fileContext` up into `dsh-tool-fs`. The split-fs-seam Agent Note's description of `dsh-tool-fs` injecting `fileContext` and of `fileContext` owning `read`/`write`/`edit` was updated to match in the same change.
|
||||
|
||||
## Verification
|
||||
|
||||
The decoupling and its semantics are pinned by tests: a bare-provider config (no `dsh-fs-policy`) boots the `dsh-tool-fs` root plugin and `read`/`write` (create and overwrite)/`edit` work against the real `dsh-fs-local` — an unread edit and an unread overwrite both succeed, proving the tool carries no `fileContext` dependency, while the same operations with `dsh-fs-policy` present are rejected `FS_NOT_OBSERVED` / gated `createIfAbsent`. A second `fs/edit-intent` listener registered after `dsh-fs-policy` is asserted NOT reached (first-wins short-circuit). A stale-read edit reports `FS_STALE_VERSION` through provider CAS, with `dsh-fs-policy` performing no `stat`; the tool's `stat` budget (read = 1, write = 0, edit = 0, on both paths) is asserted directly. Model-facing schemas stayed byte-for-byte unchanged, so snapshot transcript goldens are unaffected.
|
||||
Tests pin both paths: without `dsh-fs-policy`, the root tool plugin boots against `dsh-fs-local`, and read, create, overwrite, and unread edit succeed; with the policy, unread edit returns `FS_NOT_OBSERVED` and unread overwrite is gated by `createIfAbsent`. A later intent listener is not reached after the policy decides. Stale edits fail through provider CAS while the policy performs no `stat`; the tool budgets remain one `stat` for read and zero for write or edit on either path. Model-facing schemas remain byte-for-byte unchanged, so snapshots do not change.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep `ctx.fileContext` as an in-path method service** — the shape [the split-fs-seam RFC](../simplification/2026-06-26-fsspec-style-fs-seam.md) first landed; rejected because the tool could not run without the policy layer, making policy load-bearing for basic operation instead of an opt-in tightening.
|
||||
- **Keep `ctx.fileContext` as an in-path method service** — the shape [the split-fs-seam Agent Note](../simplification/2026-06-26-fsspec-style-fs-seam.md) first landed; rejected because the tool could not run without the policy layer, making policy load-bearing for basic operation instead of an opt-in tightening.
|
||||
- **Policy-side version checking** (`dsh-fs-policy` stats and compares in its waterfall handler) — rejected for the TOCTOU gap between that check and the tool's actual write; the provider's mutation critical section is the only race-free place, so the policy only chooses the CAS basis and gates on prior observation.
|
||||
- **Per-tool `/read`/`/write`/`/edit` subpath plugins** — dropped on implementation: no consumer needed a single-tool deployment, and subpath publishing forced bespoke `tsdown`/`tsconfig`/`files`/workspace-constraint handling no sibling tool package carries; the per-tool registration helpers remain internal modules the root plugin composes.
|
||||
|
||||
@@ -166,6 +164,6 @@ The decoupling and its semantics are pinned by tests: a bare-provider config (no
|
||||
|
||||
- **Event indirection over a method call.** A waterfall + emit is less direct than `await ctx.fileContext.edit(...)`. The payoff is removing the tool-to-policy method dependency while keeping the default policy plugin; the cost is one more event vocabulary to learn. Mitigated by keeping the three events narrow and documenting the default-thunk semantics on each.
|
||||
- **Policy events in the storage seam.** `dsh-fs` gains two version-decision events plus a recording event though it is "just storage". This is the price of decoupling (the emitter cannot depend on the policy plugin). The events carry only `dsh-fs` vocabulary plus an opaque `object` actor and no model-facing concepts, so the seam stays free of line-window/observation policy types and of the agent/session owner structure.
|
||||
- **Single policy occupant, first-wins by convention.** The `fs/write-intent`/`fs/edit-intent` slots hold exactly one decider; the first-registered (or `prepend`ed) listener wins and the rest are short-circuited. `dsh-fs-policy` owning the slot is a deployment convention, not an event-enforced invariant — a second decider registered first would bypass it. This is acceptable because a second fs-version-policy decider is a misconfiguration, not a feature. If a future need for *layered* fs version policy appears, it is a new RFC (a composable value-passing seam), not a silent second listener on these events. Layered permission/audit/sandbox interception already has its home on `tools/execute`.
|
||||
- **Single policy occupant, first-wins by convention.** The `fs/write-intent`/`fs/edit-intent` slots hold exactly one decider; the first-registered (or `prepend`ed) listener wins and the rest are short-circuited. `dsh-fs-policy` owning the slot is a deployment convention, not an event-enforced invariant — a second decider registered first would bypass it. This is acceptable because a second fs-version-policy decider is a misconfiguration, not a feature. If a future need for *layered* fs version policy appears, it is a new Agent Note (a composable value-passing seam), not a silent second listener on these events. Layered permission/audit/sandbox interception already has its home on `tools/execute`.
|
||||
- **Dropping the post-read confirming stat** makes a follow-up *guarded* edit occasionally fail-closed (`FS_STALE_VERSION` → re-read) under a read/write race. This is a UX nicety lost, never a correctness hole; the provider lock still prevents wrong-version writes.
|
||||
- **The bare provider does no read-before-write/edit and no version check.** A deployment without `dsh-fs-policy` lets the model overwrite or edit any existing file unconditionally. This is the deliberate meaning of keeping the tool independent of a policy service: the safety disciplines live in the `dsh-fs-policy` plugin. A deployment that omits it is opting into an unconstrained filesystem on purpose; that is not the intended stance for a config that ships the fs tools.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Agent Note: stdin + extra env on the bash seam
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The hooks subsystem runs external hook commands the way Claude Code and Codex do: a hook is a shell command that receives its event payload as **JSON on stdin** and reads context from a handful of **environment variables** (`CLAUDE_PROJECT_DIR`, `CLAUDE_PLUGIN_ROOT`, `PLUGIN_ROOT`, …). The harness already has a perfectly good command runner behind the `ctx.bash` capability seam ([dsh-bash](../../../../packages/bash/bash) → [dsh-bash-local](../../../../packages/bash/bash-local)), with process-group kills, output truncation/spill, and a credential scrub. Reusing it for hook execution means a hook bridge does not re-implement subprocess plumbing — but the seam had no way to write stdin or set extra env. This Agent Note adds those two inputs.
|
||||
|
||||
`stdin` and `env` do not create a new model capability because ordinary shell syntax already supplies both. Ambient credentials are protected by `dsh-bash-local`'s child-environment scrub, not by hiding these seam fields; model tool arguments are static JSON and do not expand shell variables. The fields therefore serve trusted in-process callers, such as hook bridges, that need to pass structured input and `CLAUDE_*` variables without embedding them in model-visible shell text. See [defensive-patterns.md](../../../../docs/defensive-patterns.md) for the ambient-environment rule.
|
||||
|
||||
## Decision
|
||||
|
||||
Add `stdin?: string` and `env?: Record<string, string>` to **both** `BashExecRequest` (the model-/plugin-facing request) and `BashExecSpec` (the resolved spec `run`/`start` act on), and thread them through `dsh-bash-local`: `resolve()` carries them verbatim, `run()`/`start()` pass them to `runBash`, which writes the bytes to the child's stdin and merges the extra env.
|
||||
|
||||
Three deliberate choices:
|
||||
|
||||
1. **The model-facing tool omits `stdin` and `env`.** Shell syntax already covers those needs, so duplicate parameters would add surface without authority separation. The tool builds requests only from declared model arguments, signal, and owner; trusted in-process callers may set the seam fields directly. Harness-owned variables use the separate `dshEnv` channel from the [managed environment decision](../feature/2026-07-10-agent-session-identity-and-log-location.md), so ordinary `env` cannot replace them.
|
||||
|
||||
2. **`env` merges AFTER the credential scrub, so an explicit caller entry wins even on a credential-shaped name.** The later managed-namespace decision reserves `DSH_*`: ambient entries are removed, ordinary `env` cannot set them, and trusted `dshEnv` merges last. The complete order is `scrub(process.env, including DSH_*)` → `ENV_OVERRIDES` → ordinary `env` → `dshEnv`.
|
||||
|
||||
3. **`stdin`/`env` are required-absent-OK (plain optional) on the resolved spec, NOT required-but-nullable like `owner`.** `owner` is required-but-nullable because a *silently* missing owner yields an unowned, cross-session-readable task — a security footgun that a visible `undefined` guards against. `stdin`/`env` have no such hazard: a missing one means "no stdin / no extra env", which is the safe, ordinary case (every model-driven call). So they stay plain optionals, matching `signal`.
|
||||
|
||||
`dsh-bash-local` creates a stdin pipe only when bytes are supplied; otherwise fd 0 remains `/dev/null`, preserving prior behavior. It writes the bytes and closes the pipe. `EPIPE` from a child that exits without reading is ignored because command exit and output determine the result.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Configurable ambient-secret scrub.** Rejected as speculative. Trusted callers can explicitly provide required values after the scrub without weakening the default ambient protection.
|
||||
|
||||
## Consequences
|
||||
|
||||
Hook bridges pass JSON payloads and hook-specific variables through the existing bash seam, retaining its process-group, truncation, and spill behavior. The model surface remains unchanged, and the bash tool remains the sole owner of model-call request construction. The vocabulary lives in [the bash data-structure reference](../../../../docs/core-data-structures/bash.md).
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Event-domain semantics — session is the fact log, agent is the live surface
|
||||
# Agent Note: Event-domain semantics — session is the fact log, agent is the live surface
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness extends the agent loop through a Cordis event taxonomy (see [the microkernel event-taxonomy RFC](2026-06-11-microkernel-event-taxonomy.md)). As that taxonomy grew, the line between the three event domains blurred:
|
||||
The harness extends the agent loop through a Cordis event taxonomy (see [the microkernel event-taxonomy Agent Note](2026-06-11-microkernel-event-taxonomy.md)). As that taxonomy grew, the line between the three event domains blurred:
|
||||
|
||||
- `session/*` carries the durable, event-sourced log (`SessionEventMap`).
|
||||
- `agent/*` carries live runtime signals that hand a plugin the `Agent` handle.
|
||||
@@ -12,7 +12,7 @@ The harness extends the agent loop through a Cordis event taxonomy (see [the mic
|
||||
|
||||
Two problems motivated pinning the semantics down. First, several turn/step boundaries existed BOTH as a durable `SessionEvent` (`turn/start`, `turn/end`, `step/start`, `step/end`) AND as a mirrored `agent/*` emit (`agent/turn-start`, `agent/turn-end`, `agent/step-start`, `agent/step-end`). A consumer had two sources of truth for the same fact, and every lifecycle change had to update both. Second, the upcoming Hooks subsystem needs ONE coherent, documented surface to subscribe to — a plugin author (and the Claude Code / Codex hook bridges built on top) must know, without reading the loop, whether to listen on a session event or an agent event, and why.
|
||||
|
||||
This is the foundational change in a stack that adds a Hooks subsystem; it establishes the vocabulary the later PRs (interception-Decision reshape, the `hook/*` durable log, the bridges) build on.
|
||||
This vocabulary is the foundation for interception decisions, the durable `hook/*` log, and the Claude Code and Codex bridges.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -24,14 +24,14 @@ This is the foundational change in a stack that adds a Hooks subsystem; it estab
|
||||
|
||||
**The boundary rule:** a durable, replayable fact is a `SessionEvent`; a live interception or a transient/live-object signal is an `agent`/`tools` Cordis event. A turn or step boundary is a durable fact, so it lives in the session log and is read off the `session/event` feed — it is NOT mirrored as an `agent/*` emit.
|
||||
|
||||
**Applying the rule to the boundary twins:** all four boundary mirrors — `agent/turn-start`, `agent/turn-end`, `agent/step-start`, `agent/step-end` — are **REMOVED**. No production consumer needs the live `Agent` at a boundary: the ACP bridge settles from `session/event` `turn/end` plus `agent/status`, and the only turn-mirror consumer (`dsh-ui-stdio`, a disposable test REPL) was migrated to render boundaries from `session/event`, recovering the short agent label from an `agent/created`→id map. The step mirrors were removed first (they had no consumer at all); the turn mirrors followed once ui-stdio was migrated — see [the remove-boundary-mirror-events RFC](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md), which owns that decision. Removing the emits also simplifies the loop's `closeStep`/`closeTurn` (one append each, no paired emit).
|
||||
**Applying the rule to the boundary twins:** all four boundary mirrors — `agent/turn-start`, `agent/turn-end`, `agent/step-start`, `agent/step-end` — are **REMOVED**. No production consumer needs the live `Agent` at a boundary: the ACP bridge settles from `session/event` `turn/end` plus `agent/status`, and the only turn-mirror consumer (`dsh-ui-stdio`, a disposable test REPL) renders boundaries from `session/event` while retaining its live target object for the fixed `main` label. The step mirrors were removed first (they had no consumer at all); the turn mirrors followed once ui-stdio was migrated — see [the remove-boundary-mirror-events Agent Note](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md), which owns that decision. Removing the emits also simplifies the loop's `closeStep`/`closeTurn` (one append each, no paired emit).
|
||||
|
||||
## Consequences
|
||||
|
||||
- The loop no longer emits any boundary mirror; `closeStep` appends `step/end` only and `closeTurn` appends `turn/end` only. `Session.append` owns post-commit observer containment, so a throwing boundary observer cannot change the turn outcome or starve later consumers; an acceptance or internal validation failure still escapes before the boundary enters the log.
|
||||
- Tests that observed boundaries via the removed emits now observe the durable `turn/start`/`turn/end`/`step/start`/`step/end` session events — the behavior they pin (boundary ordering, step counting) is unchanged; only the feed they read moved to the canonical one. The tests that exercised a *throwing turn-boundary emit listener* were deleted, because that code path no longer exists (there is no emit to throw from). Per [AGENTS.md "tests document behavior, not golden truth"](../../../../AGENTS.md), the behavior and its test moved (or died) together.
|
||||
- The loop marks the step open (`stepOpen = true`) only after `append('step/start')` returns. Internal dispatch validation runs before the log push and may reject without opening a step; post-commit `session/event` observer failures are contained inside `Session.append`. The marker therefore represents exactly the committed boundary that owes a later `step/end`.
|
||||
- The full realization of this is [the simplification RFC "Stop mirroring durable boundaries as agent events"](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md): all four boundary mirrors are removed and every consumer reads boundaries off `session/event`. `agent/steering` (not a boundary mirror) stayed outside that RFC's scope and was removed by its own follow-up, [Remove the `agent/steering` mirror emit](../simplification/2026-07-04-remove-agent-steering-mirror.md) — it mirrored the durable `steering/message`.
|
||||
- The full realization of this is [the simplification Agent Note "Stop mirroring durable boundaries as agent events"](../simplification/2026-06-20-remove-agent-boundary-mirror-events.md): all four boundary mirrors are removed and every consumer reads boundaries off `session/event`. `agent/steering` (not a boundary mirror) stayed outside that Agent Note's scope and was removed by its own follow-up, [Remove the `agent/steering` mirror emit](../simplification/2026-07-04-remove-agent-steering-mirror.md) — it mirrored the durable `steering/message`.
|
||||
- The cordis events catalog (`docs/cordis-catalog/events.md`) is regenerated to drop the mirror events.
|
||||
|
||||
<!-- rfc-format: alternatives-not-recorded (pre-format RFC) -->
|
||||
<!-- agent-note-format: alternatives-not-recorded (pre-format Agent Note) -->
|
||||
@@ -1,18 +1,18 @@
|
||||
# RFC: Resolve filesystem paths against the caller's session cwd
|
||||
# Agent Note: Resolve filesystem paths against the caller's session cwd
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The ACP bridge gives every session its own workspace: `session/new` records the editor's project directory as `SessionHeader.cwd`, and `dsh-tool-bash` defaults each bash call's `workdir` to the calling agent's `session.header.cwd` (see [the per-session cwd RFC work in `packages/ui/acp`](../../../../packages/ui/acp) and `resolveWorkdir` in `dsh-tool-bash`). So a bash command in session A runs in A's project, and in session B runs in B's — one server process, N workspaces.
|
||||
The ACP bridge gives every session its own workspace: `session/new` records the editor's project directory as `SessionHeader.cwd`, and `dsh-tool-bash` defaults each bash call's `workdir` to the calling agent's `session.header.cwd` (see [the per-session cwd Agent Note work in `packages/ui/acp`](../../../../packages/ui/acp) and `resolveWorkdir` in `dsh-tool-bash`). So a bash command in session A runs in A's project, and in session B runs in B's — one server process, N workspaces.
|
||||
|
||||
The filesystem tools did NOT honor this. `ctx.fs.resolve(path)` took no caller context, and `dsh-fs-local` resolved every relative path against a single `config.cwd` fixed at plugin load (`process.cwd()`). In the ACP demo that means `write foo.txt` and `bash cat foo.txt` resolve `foo.txt` against **different** directories — the fs tools against the server's launch dir, bash against the session's project dir. The two tools disagree about what "the current directory" is, which is a correctness bug the moment an editor opens any project other than the server's launch dir. It only appeared to work in the snapshot harness because that harness launches the child process in the same temp dir it passes as the session cwd, so the two coincide.
|
||||
Filesystem resolution used one plugin-load cwd while bash used the session project directory. Relative paths therefore disagreed whenever the editor project differed from the server launch directory; snapshots hid the bug by making those paths identical.
|
||||
|
||||
## Decision
|
||||
|
||||
Thread the caller's session cwd into path resolution, exactly as `dsh-tool-bash` already does for `workdir`. The **caller** (the tool) supplies the cwd; the provider does not read a session or agent.
|
||||
|
||||
- `FileSystem.resolve` widens to `resolve(path: string, opts?: { cwd?: string }): Promise<FsTarget>`. `opts.cwd` is the base a RELATIVE `path` resolves against; an absolute `path` ignores it; omitting `opts.cwd` uses the backend's own default. An options object (not a positional `cwd?`) leaves room for future resolution hints without another signature change.
|
||||
- `FileSystem.resolve` accepts `resolve(path: string, opts?: { cwd?: string; signal?: AbortSignal }): Promise<FsTarget>`. `opts.cwd` is the base a RELATIVE `path` resolves against; an absolute `path` ignores it; omitting `opts.cwd` uses the backend's own default. `opts.signal` cancels resolution when the backend performs I/O. The options object keeps both caller-owned resolution controls together without positional growth.
|
||||
- `dsh-fs-local.resolve` uses `resolveLocalTarget(opts?.cwd ?? this.config.cwd, path)`. `config.cwd` stays the default for a caller that supplies none (non-ACP / no-session use, and the single-session stdio demo where `process.cwd()` IS the workspace).
|
||||
- `dsh-tool-fs`'s `read`/`write`/`edit` derive the session cwd through a shared `sessionCwd(exec)` helper (`exec.agent?.session.header.cwd`, mirroring bash's `resolveWorkdir`) and pass it to `resolve`. A non-agent / headerless caller yields `undefined`, so the backend applies its default.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Result-time applied-hunk diffs for file mutations
|
||||
# Agent Note: Result-time applied-hunk diffs for file mutations
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -22,7 +22,7 @@ Add a **persisted, tool-private presentation channel** so a tool's `execute` can
|
||||
type ToolExecuteReturn = ContentBlock[] | { content: ContentBlock[]; meta?: unknown }
|
||||
```
|
||||
|
||||
`meta` is an opaque payload the core never interprets — typed `unknown` at every seam (the tool that produced it owns and narrows its shape). It MUST be JSON-serializable: the registry threads it onto the `tool/result` **session event**, and `Session.append` runtime-validates all event data with the existing `isJsonValue` predicate, so a non-serializable `meta` is rejected at the source. On replay the same `meta` is read back and handed to `presentResult` via a widened `ToolResult` (`{ content, isError, meta? }`). Because the payload lives in the event log, the diff reproduces on session reload / snapshot replay **for free** — the event-sourcing guarantee, not a re-computation. Typing `meta` as `unknown` (rather than a shared serializable-value type) keeps the tools core free of a dependency it would otherwise take just to name the type, and the runtime `isJsonValue` gate — not the static type — is what actually enforces serializability.
|
||||
`meta` is tool-owned `unknown` that the core persists without interpretation. `Session.append` rejects non-JSON values, and replay passes the stored payload back to `presentResult`; presentations therefore reproduce without I/O or recomputation. Runtime validation avoids adding a shared serializable-value dependency to the tools core.
|
||||
|
||||
This is the general shape ("a tool attaches durable result presentation"), not an fs-specific one — any tool can use it.
|
||||
|
||||
@@ -31,7 +31,7 @@ This is the general shape ("a tool attaches durable result presentation"), not a
|
||||
Per the [capability-seam split](2026-06-13-capability-seams.md), the storage backend returns only **storage facts** and the model-facing tool owns **presentation**:
|
||||
|
||||
- `dsh-fs` widens `FsEditOutcome` with `{ before: string; after: string }` and `FsWriteOutcome` with `{ before: string | null; after: string }` (`before: null` ⇒ a create, or an existing-but-undiffable binary/non-UTF-8 file). The local backend already holds both texts at write time; it returns them as raw LF-normalized text, with **no diff/UI concept** entering the seam.
|
||||
- `dsh-tool-fs` computes the contextual hunk from before/after and attaches it as `meta: { diffs: FileDiff[] }`. A contextual hunk is computed only when a before-version exists — edit always; write on overwrite; a create has no before, matching `claude-agent-acp`'s empty `structuredPatch` on create. But the completed `tool_call_update` is ALWAYS a `diff` card for a successful mutation: an ACP `tool_call_update.content` REPLACES the call's content, so rendering the model-facing result text would clobber the pending diff. So `write`'s result falls back to an args-derived whole-file diff (`oldText: null`) when it has no contextual hunk (a create, or an overwrite whose content is unchanged), and `edit` — which always changes content — always has a hunk. A failed/aborted/policy-rejected mutation applied nothing, so it carries no `meta` and falls through to the generic error rendering (its message must show).
|
||||
- `dsh-tool-fs` stores contextual hunks in `meta: { diffs: FileDiff[] }`. Successful mutations always complete with a diff card because ACP result content replaces the pending card: creates or unchanged overwrites fall back to an args-derived whole-file diff, while edits use applied hunks. Failed mutations carry no diff metadata and render their error normally.
|
||||
|
||||
### 3. The bridge renders a `diff` result card
|
||||
|
||||
@@ -39,7 +39,7 @@ Per the [capability-seam split](2026-06-13-capability-seams.md), the storage bac
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Hand-rolling or vendoring the diff algorithm.** Computing hunks-with-context is a solved problem with sharp edge cases (grouping, context coalescing, the trailing-newline marker). Rather than hand-roll it, `dsh-tool-fs` takes a runtime dependency on the npm [`diff`](https://www.npmjs.com/package/diff) package (a `^9.0.0` range, exact-pinned by the lockfile; it ships its own types) and uses its `structuredPatch`. The repo's default is to vendor Cordis-framework source, but that policy is about the *framework*; a leaf tool package taking a small, well-known, self-typed utility dependency is the same shape as `dsh-acp` depending on `@agentclientprotocol/sdk`. Vendoring a diff algorithm would be re-implementing a battle-tested one for no benefit — the [pre-release "foundation over blast radius"](../../../../AGENTS.md) reasoning does not argue for re-deriving standard algorithms. The dependency's output is normalized in one small module (`packages/fs/tool-fs/src/diff.ts`).
|
||||
**Hand-rolling or vendoring the diff algorithm.** Contextual hunks have established edge cases, so `dsh-tool-fs` uses the typed [`diff`](https://www.npmjs.com/package/diff) package and normalizes `structuredPatch` output in one module. The repository's vendoring policy applies to its framework source, not every leaf utility.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -54,6 +54,6 @@ Per the [capability-seam split](2026-06-13-capability-seams.md), the storage bac
|
||||
|
||||
## Related
|
||||
|
||||
- Completes the one remaining representation difference named as a non-goal in [Tagged render-intent union](2026-07-02-tool-render-intent-union.md) — that RFC's Non-goals section is updated to record that applied-hunk diffs shipped here.
|
||||
- Completes the one remaining representation difference named as a non-goal in [Tagged render-intent union](2026-07-02-tool-render-intent-union.md) — that Agent Note's Non-goals section is updated to record that applied-hunk diffs shipped here.
|
||||
- Builds on the [filesystem capability seam](2026-06-17-filesystem-capability-seam.md) (the before/after are storage facts the backend returns) and [event-sourced sessions](2026-06-11-event-sourced-sessions.md) (the `meta` payload persists on the `tool/result` event, so replay reproduces the card).
|
||||
- The `meta` channel is deliberately generic: a future tool (a structured search, a data-table result) can attach its own durable result presentation without another core change.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Tagged render-intent union for tool-call presentation
|
||||
# Agent Note: Tagged render-intent union for tool-call presentation
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -10,7 +10,7 @@ A tool declares how its calls render in a UI (an editor's tool-call card) throug
|
||||
- Which combinations are *valid* is unwritten: a `terminal` call that also sets `content` means "description above the card"; a generic call that sets `terminal` is meaningless but representable. The type permits nonsense.
|
||||
- There is no way to express the one file-tool affordance an editor most wants — a **diff card** (`{path, oldText, newText}`, which Zed renders as an inline diff / new-file preview). `ToolCallPresentation.content` is the *LLM* `ContentBlock[]` vocabulary (text/image), so a tool literally cannot ask for a diff.
|
||||
|
||||
The existing `FIXME(tool-presentation)` in `packages/core/tools/src/index.ts` named the fix: "redesign the type so a tool declares its render INTENT once (e.g. a tagged union over card kinds) rather than a bag of optional fields the bridge stitches together." The rejected RFC [Collapse tool-owned UI presentation](../../rejected/simplification/2026-06-20-generic-tool-rendering.md) deferred it explicitly: rich rendering "should return later as a tagged render-intent union after there are at least two real tools and two real consumers to validate the vocabulary." That bar is now met — two producer families (`dsh-tool-bash`, `dsh-tool-fs`) and two consumers (the ACP bridge live path + the snapshot-golden replay path).
|
||||
The existing `FIXME(tool-presentation)` in `packages/core/tools/src/index.ts` named the fix: "redesign the type so a tool declares its render INTENT once (e.g. a tagged union over card kinds) rather than a bag of optional fields the bridge stitches together." The rejected Agent Note [Collapse tool-owned UI presentation](../../rejected/simplification/2026-06-20-generic-tool-rendering.md) deferred it explicitly: rich rendering "should return later as a tagged render-intent union after there are at least two real tools and two real consumers to validate the vocabulary." That bar is now met — two producer families (`dsh-tool-bash`, `dsh-tool-fs`) and two consumers (the ACP bridge live path + the snapshot replay path).
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -43,7 +43,7 @@ interface TerminalResultView { card: 'terminal'; title?: string; output?: string
|
||||
### Producer mapping
|
||||
|
||||
- `dsh-tool-fs` read → `generic` (`kind:'read'`, a follow-along `location`); write → `diff` (`oldText:null`); edit → `diff` (`oldText:old_string || null`, `newText:new_string ?? ''`). This mirrors `claude-agent-acp`'s `toolInfoFromToolUse` Read/Write/Edit arms field-for-field.
|
||||
- `dsh-tool-bash` foreground → `terminal` call + `terminal` result; `run_in_background` and `bash_output`/`bash_kill` → `generic`.
|
||||
- `dsh-tool-bash` foreground → `terminal` call + `terminal` result; `run_in_background` → `generic`. The generic `task_*` controls own their own generic cards.
|
||||
- `dsh-tool-todo` → `generic`.
|
||||
|
||||
### Terminal fallback ownership
|
||||
@@ -70,7 +70,7 @@ A new render intent is a compile-breaking change at the bridge switch — delibe
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Live incremental `terminal_output_delta` streaming** and **command classification** — the terminal-rendering RFC's own deferred follow-ups, untouched here.
|
||||
- **Live incremental `terminal_output_delta` streaming** and **command classification** — the terminal-rendering Agent Note's own deferred follow-ups, untouched here.
|
||||
|
||||
## Related
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Add direct directory listing to the filesystem seam
|
||||
# Agent Note: Add direct directory listing to the filesystem seam
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -8,7 +8,7 @@ Status: implemented
|
||||
|
||||
The immediate pressure came from skill loading: reading an individual `SKILL.md` can already go through `ctx.get('fs')`, but discovering which skill roots contain `<name>/SKILL.md` or `<name>.md` still needs directory enumeration. Adding directory listing only in `dsh-skill` would either keep a direct Node dependency there or invent a one-off local helper outside the filesystem provider stack.
|
||||
|
||||
This branch deliberately lands the provider capability first and does not add a model-facing `ls`/`list` tool or change skill discovery. The follow-up consumer can validate UX and prompt shape separately, while this PR establishes the backend seam and local implementation.
|
||||
This decision adds the provider capability without a model-facing `ls`/`list` tool or skill-discovery change. Those consumers require separate UX, prompt, and policy decisions.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -36,7 +36,7 @@ Broken or disappeared children may be represented as `type: 'other'` without `ve
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Add a model-facing list tool now.** Rejected for this PR. The immediate request is the provider seam, and the user explicitly asked not to change skill loading or other upper layers in this branch. A model-facing tool needs prompt/schema/rendering decisions that should be reviewed separately.
|
||||
**Add a model-facing list tool with the seam.** Rejected because its prompt, schema, and rendering contracts are independent of the provider primitive.
|
||||
|
||||
**Keep directory enumeration in each consumer.** Rejected. That would bind product packages such as `dsh-skill` to Node/local filesystem behavior and bypass policy/remote/sandboxed backends.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Prompt variables and tool-guidance ownership
|
||||
# Agent Note: Prompt variables and tool-guidance ownership
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -8,9 +8,9 @@ The assembled system prompt had four defects, all of one family: facts the harne
|
||||
|
||||
**The model could not know its own name.** `AgentOptions.model` drives every request, but no prompt text carried it — and nothing COULD carry it: sections in `dsh-system-prompt` were context-global while the model name is per-agent, and `assemble()` took no per-agent input at all.
|
||||
|
||||
**Tool guidance was hand-written prose in leaf YAML.** The bash/subagent/todo_write usage guidance lived in the `systemPrompt` strings of `examples/coding-agent/cordis.yml` and `examples/acp-agent/cordis.yml` — two drifting copies (the ACP one was already abridged) — while `dsh-tool-fs` and `dsh-tool-web` owned their guidance as `ctx.systemPrompt.section()` contributions. Loading or dropping a tool plugin meant editing every deployment's persona by hand; both YAMLs carried a `FIXME(config-comments)` apologizing for a symptom of the split, and the stdio welcome banner hand-enumerated the tool set too.
|
||||
**Tool guidance was hand-written prose in leaf YAML.** The bash/subagent/todo_write usage guidance lived in the coding-agent and ACP persona strings — two drifting copies (the ACP one was already abridged) — while `dsh-tool-fs` and `dsh-tool-web` owned their guidance as `ctx.systemPrompt.section()` contributions. Loading or dropping a tool plugin meant editing every deployment's persona by hand; both YAMLs carried a `FIXME(config-comments)` apologizing for a symptom of the split, and the old terminal welcome banner hand-enumerated the tool set too.
|
||||
|
||||
**The persona rendered after tool guidance.** The loop string-joined `agent.options.systemPrompt` AFTER the assembled sections, so the model read "Use the read tool…" before "You are coding-agent" — backwards relative to the identity-first convention (Claude Code, Codex) and a second composition path besides the section pipeline.
|
||||
**The persona rendered after tool guidance.** The loop string-joined `agent.options.systemPrompt` AFTER the assembled sections, so the model read "Use the read tool…" before "You are a coding agent" — backwards relative to the identity-first convention (Claude Code, Codex) and a second composition path besides the section pipeline.
|
||||
|
||||
**The fork tool's description was false.** `dsh-tool-subagent` hardcoded one description written for spawn semantics — "a separate agent that works in its own context … it does not see this conversation" — and the `subagent_fork` instance (whose child inherits the parent's completed turns) got the same words; the YAML prose corrected the lie out-of-band. Minor kin: `PromptSection.name` was documented "(diagnostics / dedup)" but duplicates were silently accepted.
|
||||
|
||||
@@ -24,30 +24,30 @@ The assembled system prompt had four defects, all of one family: facts the harne
|
||||
|
||||
### Prompt variables
|
||||
|
||||
Plugins contribute named values via `ctx.systemPrompt.variable(name, provider)`; prompt text references them as `{{name}}`. Providers are functions of the `AssembleContext` and may return `undefined` — "no value for THIS assembly". `assemble()` resolves every registered variable into `PromptAssembly.variables` (waterfall listeners can see, add, or override); `renderPrompt` interpolates. Rendering is STRICT — fail loud beats shipping a malformed prompt: a reference to an unregistered name throws (listing what exists; lookup is `Object.hasOwn`, so a prototype property like `{{constructor}}` is unknown, not a function spliced into the prompt), a registered-but-valueless reference throws, a complete `{{…}}` group that is not a well-formed name (`[a-z][a-z0-9_]*`, e.g. `{{ model }}`) throws, and a `{{` that opens no complete group while a `}}` still follows (`{{{model}}}`, `{{a{b}}`) throws. A lone `{{` with no `}}` anywhere after it is ordinary prose and passes through verbatim; substituted values are never re-scanned. Registration rejects duplicate and unreferenceable names, mirroring the tool registry — and `section()` now rejects duplicate section names, making the documented dedup real.
|
||||
Plugins register `{{name}}` values through `ctx.systemPrompt.variable(name, provider)`. Assembly resolves them into the waterfall-visible variable map. Rendering rejects unknown own-property references, registered providers that return `undefined`, malformed complete references, and unbalanced references that still contain a closing `}}`; a lone unmatched `{{` remains prose, and substituted values are not rescanned. Registration rejects invalid or duplicate variable names, and section names are unique.
|
||||
|
||||
`dsh-agent-loop` registers the two built-ins, both pure projections of the context agent: `model` (= `options.model`) and `cwd` (= `session.header.cwd`). The example personas write `powered by the {{model}} model` — the model name is stated once, in the `model:` config key. `{{cwd}}` is demonstrated in the ACP example only: every ACP session carries the client's cwd, while config-pre-created stdio agents have none (a persona claiming `{{cwd}}` there fails the turn — by design). The variables stay on the loop plugin (unlike the sections below): they are runtime facts of the agents THIS loop drives, and a replacement loop supplies its own.
|
||||
|
||||
### Persona as the order-0 section
|
||||
|
||||
`dsh-system-prompt` itself registers the two harness-owned sections (they must survive a swapped loop plugin, so they do NOT live on `dsh-agent-loop`): the static `harness:identity` at order `-100` — every prompt opens by stating the agent is powered by the DeepSeek Harness SDK — and the global default `deployment:persona` at order 0, whose text is the plugin's own `persona` config. `AgentOptions.systemPrompt` and the loop's special-case join are gone: `fullSystemPrompt ≡ renderPrompt(assembly)`, one ordered pipeline for everything the model sees, and `agent/pre-step` (compaction's token-pressure input) measures exactly the real prompt. An agent-scoped section with the same `deployment:persona` name shadows the default for that agent; programmatic setup may register one directly, and the subagent persona feature installs one before publishing an in-process child when the selected provider supports it. Order bands are convention: harness identity `-100`, persona `0`, tool guidance `100–199`; other negative orders also render before the persona.
|
||||
`dsh-system-prompt` owns `harness:identity` at order `-100` and the configured `deployment:persona` at order 0, so both survive a replacement loop. Prompt rendering has one path, `renderPrompt(assembly)`, and the routed request header therefore records the exact prompt later replayed by `ctx.tokenMeter` for compaction pressure. An agent-scoped `deployment:persona` shadows the global default and lets subagent providers install a persona before publication. The conventional order bands are identity `-100`, persona `0`, and tool guidance `100–199`.
|
||||
|
||||
### Tool guidance ownership
|
||||
|
||||
Per-tool semantics and when-to-use live in tool DESCRIPTIONS, which already ship in every request — the YAML prose was ~fully redundant with them. Sections carry only the cross-call habits a single call's description cannot: `dsh-tool-bash` contributes `tool:bash` (order 105) — check the `[exit code: N]` marker on every result; `dsh-tool-fs`'s read section gains the "not shell commands like cat" contrast. `todo_write` and the subagent tools need NO section — their descriptions already carry the whole contract. The leaf personas shrink to identity + behavior (verify your work; keep answers brief), and the welcome banner stops enumerating tools.
|
||||
Per-tool semantics and selection guidance live in tool descriptions. Prompt sections carry only cross-call habits, such as checking bash exit markers or preferring filesystem tools over shell commands. `todo_write` and subagent tools need no section because their descriptions contain the full contract. Deployment personas contain only role and behavior.
|
||||
|
||||
### The subagent conversation-history descriptor
|
||||
|
||||
`SubagentProvider` gains `readonly inheritsParentContext: boolean` — a DESCRIPTIVE conversation-history fact beside `capabilities`, not in it (capabilities are start-time validation; nothing validates against this flag). Spawn and ACP declare `false`, fork declares `true`. The name refers only to conversation seeding, not Cordis scope, services, tools, or authority. `dsh-tool-subagent` derives both the tool description and the `prompt` parameter description from the flag (`providerWording`): the fork instance now tells the model the child is seeded with the conversation's completed turns (not the in-flight turn) and that its prompt should state only what is new. Deriving the description from a provider that arrives on its own fiber is what forced the provider-lifecycle events and the tool's reactive registration — that mechanism, its Loader-concurrency rationale, and its rejected alternatives are recorded in [the provider-lifecycle-events RFC](2026-07-05-subagent-provider-lifecycle-events.md).
|
||||
`SubagentProvider.inheritsParentContext` describes conversation seeding, not scope, services, tools, or authority. Spawn and ACP set it to `false`; fork sets it to `true`. `dsh-tool-subagent` derives its tool and prompt-parameter descriptions from the flag, including that fork inherits completed turns but not the in-flight turn. Provider lifecycle events keep that wording synchronized with reactive provider registration; their rationale lives in the [provider-lifecycle-events Agent Note](2026-07-05-subagent-provider-lifecycle-events.md).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **The loop composes an identity line itself** — hardcodes model-facing prose in the one package that must stay thin ("plugins, not loop changes"), and outside the section pipeline it would be a second composition path. (The identity DOES ship as a code literal — but as an ordinary section registered by `dsh-system-prompt`, whose `system-prompt/assemble` waterfall remains the escape valve for a deployment that must drop it.)
|
||||
- **Inject the model name via the `agent/request` waterfall** — prompt text composed in two places, and `agent/pre-step`'s `fullSystemPrompt` would omit it, so compaction would measure a prompt that is not what the model sees.
|
||||
- **Hand-write the model name in each persona** — duplicates the `model:` key one line above and silently lies after a config edit; the exact disease this RFC cures.
|
||||
- **Inject the model name via the `agent/request` waterfall** — prompt text would be composed in two places and the earlier rendered persona could disagree with the final routed header. The request plugin that owns late routing must also own any earlier prompt claim about that model.
|
||||
- **Hand-write the model name in each persona** — duplicates the `model:` key one line above and silently lies after a config edit; the exact disease this Agent Note cures.
|
||||
- **Lenient interpolation (leave unknown refs verbatim, or substitute empty)** — a typo ships `{{modle}}` (or a hole) to the model and nobody notices until transcript review.
|
||||
- **Per-instance subagent wording in config** — returns model-facing prose to every deployment × instance, the P2 disease again. **Keying wording off the provider NAME** — `providerName` is itself config, so a renamed provider silently gets the wrong words.
|
||||
- **Resolving the provider at `apply` time (a load-order requirement)** and **section-only subagent wording (lazily resolved at assemble)** — the alternatives to the provider-lifecycle events; both rejected in [the provider-lifecycle-events RFC](2026-07-05-subagent-provider-lifecycle-events.md).
|
||||
- **Resolving the provider at `apply` time (a load-order requirement)** and **section-only subagent wording (lazily resolved at assemble)** — the alternatives to the provider-lifecycle events; both rejected in [the provider-lifecycle-events Agent Note](2026-07-05-subagent-provider-lifecycle-events.md).
|
||||
|
||||
## Out of scope
|
||||
|
||||
@@ -56,10 +56,10 @@ Per-tool semantics and when-to-use live in tool DESCRIPTIONS, which already ship
|
||||
|
||||
## Shipped invariants
|
||||
|
||||
- `renderPrompt(await assemble(assembleContextFor(agent)))` for the coding-agent example renders the harness identity, then the persona (with the agent's model name interpolated), then the fs/bash/web guidance sections; the loop has no other prompt-composition path.
|
||||
- The `subagent_fork` schema description says the child inherits the conversation; the `subagent` one says it does not. The tool follows its provider: absent before the backend activates, present after, gone when the backend unloads, re-worded from the fresh provider on reload.
|
||||
- Unknown/valueless/malformed/unbalanced `{{…}}` references throw with the section name in the message; duplicate section, variable, and tool-name registrations all throw.
|
||||
- Snapshot goldens are prompt-independent by construction: llm-replay keys replay on (turn, step) chunk streams and never re-verifies the outgoing request.
|
||||
- The tui-agent prompt renders identity, persona with the interpolated model, then fs/bash/web guidance through one assembly path.
|
||||
- Fork and fresh subagent descriptions reflect whether the provider inherits completed conversation turns; the tool appears, disappears, and is reworded with provider lifecycle changes.
|
||||
- Unknown, valueless, malformed, or unbalanced variable references name the section and throw; duplicate section, variable, and tool registrations also throw.
|
||||
- Snapshot replay is prompt-independent: it keys recorded chunk streams by turn and step without comparing the outgoing request.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# Agent Note: Every LLM request is reconstructable from the session log
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The request pipeline did not guarantee prefix stability for provider caching, and the session log could not reconstruct what the model saw. It omitted model, system prompt, and tool schemas while allowing per-call request rewrites. Cache behavior and replay equivalence therefore depended on whichever plugins happened to be loaded.
|
||||
|
||||
The reference shape for the happy path is MiniCode's `LLMClient`: a stateful conversation client, appended to — never rebuilt — as the conversation advances, resetting only when the system prompt, tool set, or compaction genuinely changes what the model must see. The design question this Agent Note answers is how to get that discipline without giving up event-sourcing.
|
||||
|
||||
## Decision
|
||||
|
||||
### The principle
|
||||
|
||||
**Model-visible ⟺ logged.** Anything that reaches a model request must be recorded in the session log. The checkable consequence: **every conversation request the loop sends is a pure function of the session log** — anyone holding the log reconstructs it byte-for-byte. Scope, stated precisely: the guarantee covers the loop-built `GenerateOptions`; provider wire bytes follow from it because both adapters' serialization is a pure per-message function at a pinned code version; direct one-shots (compaction's summarize call) log their envelope scalars (`compact/summary.{provider, model, maxTokens}`) and their input is deterministic code over the logged region — reconstructable from log + code, outside the invariant because only the loop marks request ownership.
|
||||
|
||||
Prefix-cache stability is corollary #1, not the headline: an append-only log projected by a per-node pure function yields requests that are append-extensions of their predecessors whenever the header is unchanged — stability is emergent, not managed. Byte-exact audit/replay is corollary #2; resume and fork with *attributable* drift is corollary #3.
|
||||
|
||||
### The mechanism
|
||||
|
||||
**Messages.** `Session.deriveMessages()` is cached: each surface entry is projected exactly once, when first seen, through the public per-event function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree.
|
||||
|
||||
`EpochHeader` records the request's non-history state: call config, rendered system prompt, tool schemas, and session prefix, with empty values canonicalized to absence. `request/header` always writes a full snapshot: the first loop instance uses reason `initial`, later instances use `resume`, and an in-instance change uses `change`. `foldRequestHeader` selects the latest snapshot. Legacy `request/header-delta` events and the removed `fallback` reason are rejected when appended or loaded.
|
||||
|
||||
Each step rebuilds prompt assembly. On the instance's first step, `agent/session-prefix` extends a frozen empty seed with request-only opener messages; the result is frozen and cached for that loop instance before the generic `agent/pre-step` checkpoint and boundary snapshot. The first call config starts from explicit `AgentOptions`, preserving fork overrides and resume reconfiguration; later calls start from the folded header. `agent/request` may replace only that frozen config seed, while model-visible content enters through logged channels. The loop records the owed header event—the prefix's only durable home—builds `GenerateOptions` from prefix, snapshot, and header, and deep-freezes it while leaving `AbortSignal` live. Per-instance state is only the cached prefix and whether its anchoring snapshot has been written.
|
||||
|
||||
**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step(agent, turn, step, signal)` remains the generic seam for content needed by the current request. Header reconstruction selects the step's `request/header`, or carries the prior snapshot when no new header is written.
|
||||
|
||||
**Enforcement.** The `dsh-agent-loop/invariant` companion registers with `ctx.invariants` and, when selected, independently rebuilds each loop request through a fresh `Session`, so the live cache cannot vouch for itself, then compares messages and folded header fields at `llm/stream`. The loop applies an internal non-enumerable identity before freezing each request; the independently built companion recognizes that identity, while direct one-shots remain excluded regardless of their frozen shape or session id. Correctness depends on sequence-bounded reconstruction rather than listener order. A with-key e2e requires positive cache-read tokens after the first request; per-step usage is the production signal, and a header change or compaction appears as a cache-read drop on the next step.
|
||||
|
||||
### The MiniCode shape: adopted, with the provenance arrow inverted
|
||||
|
||||
Like MiniCode, the conversation advances append-only and resets only when model-visible state changes. Unlike MiniCode, the event log remains the source of truth because it also owns persistence, recovery, boundaries, tool pairing, and provenance. `Session` caches message and header folds derived from that log, making every request independently checkable.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Client as source of truth** (literal MiniCode): a second operative truth beside the log — the two drift and nothing notices; see the section above.
|
||||
- **A stateful transmission client mirroring the log** — duplicates conversation state, needs rollback around listeners, leaves an unlogged edit surface, and still cannot reconstruct request headers. Session-owned caches plus logged headers avoid those split truths.
|
||||
- **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records.
|
||||
- **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability.
|
||||
- **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline.
|
||||
- **A custom header-delta codec** (system line edits, name-keyed tool edits, whole config/prefix replacements): reduced repeated bytes but duplicated the representation and its diff/apply/fallback machinery. Full snapshots retain one replay representation.
|
||||
- **Narrative changed-field lists on header snapshots**: derivable by comparing consecutive snapshots. The `reason` remains because an instance boundary is not derivable from the snapshot values.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event.
|
||||
- Choosing between the advisory channels is a change-frequency decision, and the design makes the stable one structural: an `agent/session-prefix` contribution is composed once per loop instance and reused verbatim, so it extends the cacheable prefix at zero marginal cost and CANNOT bust the provider cache mid-session; content that changes mid-session flows through the append-only history channels — `agent.inject()` and tool/prompt-submit `additionalContexts` — each a durable `context/message` paid once and prefix-cached thereafter, at the price of accumulating in history and the log. Route session-frozen openers to the prefix and change notices to the history channels; a per-step request-only tail slot was deliberately dropped (no consumer, and a durable append covers every current update pattern).
|
||||
- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replacement entry), a real prompt, tool, or config change (`request/header` with reason `change`), or a process boundary with drift (a differing `resume` snapshot). The provider's own reasoning-content exclusion is managed server-side.
|
||||
- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam.
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated.
|
||||
- Snapshot expected outputs changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
@@ -1,12 +1,12 @@
|
||||
# RFC: Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`
|
||||
# Agent Note: Subagent provider-lifecycle events — `subagent/provider-added` / `subagent/provider-removed`
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
[The prompt-variables RFC](2026-07-05-prompt-variables-and-tool-guidance-ownership.md) makes `dsh-tool-subagent` DERIVE its model-facing wording from its provider: `SubagentProvider.inheritsParentContext` (spawn/ACP `false`, fork `true`) drives both the tool description and the `prompt` parameter description (`providerWording`), so the fork tool stops lying about context inheritance. That fix created a cross-fiber data dependency: a tool's description is fixed at TOOL REGISTRATION (deliberately — the description is where tool-choice guidance lives), but the provider arrives on its own plugin fiber, on no particular schedule.
|
||||
[The prompt-variables Agent Note](2026-07-05-prompt-variables-and-tool-guidance-ownership.md) makes `dsh-tool-subagent` DERIVE its model-facing wording from its provider: `SubagentProvider.inheritsParentContext` (spawn/ACP `false`, fork `true`) drives both the tool description and the `prompt` parameter description, so the fork tool stops lying about context inheritance. That fix created a cross-fiber data dependency: a tool's description is fixed at TOOL REGISTRATION (deliberately — the description is where tool-choice guidance lives), but the provider arrives on its own plugin fiber, on no particular schedule.
|
||||
|
||||
The first implementation resolved the provider at the tool plugin's `apply` time and threw when it was absent — an implicit load-order requirement ("list the backend before the tool in cordis.yml"). Review reproduced the failure that requirement hides: the cordis Loader starts sibling entries CONCURRENTLY (`Promise.all` over the group) and `Entry.init()` does not await activation, so a backend whose activation is delayed leaves the tool's fiber permanently failed even when "listed first". The ordering the requirement leaned on is not a contract the Loader offers — "async state is not synchronous state" ([defensive patterns](../../../defensive-patterns.md)).
|
||||
Resolving the provider at the tool plugin's `apply` time creates an implicit load-order requirement ("list the backend before the tool in cordis.yml"). That requirement fails because the Cordis Loader starts sibling entries concurrently and `Entry.init()` does not await activation: a delayed backend can leave the tool fiber failed even when listed first. The Loader offers no sibling-order guarantee — "async state is not synchronous state" ([defensive patterns](../../../../docs/defensive-patterns.md)).
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -21,14 +21,14 @@ The events also complete the seam's vocabulary: `ctx.subagents` is a named regis
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Resolving the provider at `apply` time and throwing when absent (a load-order requirement)** — the first implementation, rejected after review reproduced the failure above. Documenting the requirement ("list backends first") would pin a guarantee the Loader does not make.
|
||||
- **Resolve the provider at `apply` time and throw when absent** — rejected because "list backends first" would claim a Loader ordering guarantee that does not exist.
|
||||
- **Retrying the lookup (poll until the provider appears)** — converges eventually but invents a private readiness protocol beside the one the framework already has (effect registration + disposal); it also cannot notice a provider LEAVING, so HMR would strand a tool whose wording describes a disposed backend.
|
||||
- **Section-only subagent wording, lazily resolved at assemble time** — tolerates any load order too, but moves tool-choice guidance out of the DESCRIPTION, contradicting the ownership rule the prompt-variables RFC establishes (per-tool semantics and when-to-use belong in the description). Reactive registration keeps the description authoritative AND order-free.
|
||||
- **Section-only subagent wording, lazily resolved at assemble time** — tolerates any load order too, but moves tool-choice guidance out of the DESCRIPTION, contradicting the ownership rule the prompt-variables Agent Note establishes (per-tool semantics and when-to-use belong in the description). Reactive registration keeps the description authoritative AND order-free.
|
||||
- **Keying wording off the provider NAME instead of the provider object** — `providerName` is itself config, so a renamed provider silently gets the wrong words; deriving from the resolved provider's own `inheritsParentContext` cannot drift.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Consumers deriving state from a named provider react to `subagent/provider-added`/`-removed` instead of reading the registry at `apply` time; `dsh-tool-subagent` is the reference implementation.
|
||||
- **The two emits carry asymmetric failure semantics, deliberately.** `provider-removed` fires inside the registration's disposer and is delivered with PER-LISTENER containment (the service's `emitLifecycle`, not raw `ctx.emit`, which halts dispatch on the first throw): a throwing subscriber is logged, never starves a later mirror into holding a stale tool, and never disrupts the backend fiber's teardown — dispose reaches quiescence. `provider-added` propagates: it fires at registration time, where a throwing listener unwinds the yielded rollback — the same fail-loud register-time semantics as the system-prompt registries. The run-time backstop bounds what a stale mirror could cost anyway: `start()` re-resolves the provider by name per run, so a tool that outlived its provider fails that call cleanly instead of dispatching into a dead backend. The [events catalog](../../../cordis-catalog/events.md) carries the exact signatures, and the [producer/consumer map](../../../event-producer-consumer.md) shows `dsh-subagent` emitting and `dsh-tool-subagent` consuming both events.
|
||||
- **Addition fails loud; removal is contained per listener.** An addition listener may unwind registration. Removal runs during disposal, so one throwing listener is logged without starving later mirrors or disrupting teardown. `start()` still resolves the provider by name for every run, preventing stale tools from calling a removed backend. See the [events catalog](../../../../docs/cordis-catalog/events.md) and [producer/consumer map](../../../../docs/event-producer-consumer.md).
|
||||
- **A window where the tool is absent.** Between backend disposal and re-registration (an HMR reload), the model sees no subagent tool. This is the honest state — the alternative is a tool that dispatches into nothing — and the tool registry's `tools/change` emit keeps prompt assembly current.
|
||||
- **Two waiting fibers sharing a `toolName` is an invalid config caught late.** If two loads of `dsh-tool-subagent` name different providers but the same `toolName`, both wait, and whichever provider arrives first registers; the second registration throws only when ITS provider arrives. `TODO(subagent-dup-toolname)` in the plugin records this blast radius; the tool registry's duplicate-name rejection remains the backstop.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: A shared timeout/deadline primitive, with hard-kill left to each capability
|
||||
# Agent Note: A shared timeout/deadline primitive, with hard-kill left to each capability
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -12,15 +12,13 @@ Timeout handling was drifting apart across the tool-bearing capabilities, and th
|
||||
|
||||
Each new external-process or network tool re-derived the same four things — clamp the requested value, start a timer, fuse the timeout with upstream cancellation, and distinguish "timed out" from "cancelled" on the way out — and the fusion and reason-recovery are exactly the parts that are easy to get subtly wrong (web_fetch's `signal.reason` dance is evidence). At the same time, the *termination* each performs is irreducibly different: bash kills an OS process group (work runs in a child process, outside this runtime, reachable only by signal), while web aborts an in-process `fetch` (undici tears down the socket). There is no single mechanism that can stop all of them.
|
||||
|
||||
The two reference agents surveyed converged on the same split. Codex models "what will end this exec early" as one value (`ExecExpiration`, an enum fusing timeout and a cancellation token) whose `wait_with_outcome()` returns `TimedOut | Cancelled`, while the actual `kill_process_group` lives outside it — and that abstraction is reused *only* across the exec family, with MCP, model-stream, and guardian each keeping their own bespoke `tokio::time::timeout`. Claude Code shares nothing: bash and ripgrep each own a private SIGTERM→SIGKILL kill and distinguish timeout from cancellation by throwing distinct error types, while file I/O has no timeout. Both confirm the boundary drawn here: the timing-and-classification half is worth sharing within a family of like-terminated operations; the termination half is not shareable and stays in each capability.
|
||||
|
||||
## Decision
|
||||
|
||||
`@deepseek-ai/dsh-timeout` lives under `packages/util/` (peer to `dsh-brand`) and owns the *timing and classification* half of timeout; the *termination* half — the hard kill — stays in each capability's implementation. It is a library of pure functions, **not** a cordis service or plugin: it takes no `ctx`, registers nothing, holds no cross-call state, and emits no events. There is deliberately no central "timeout service" that would have to know how to stop every capability's work — that knowledge is exactly what a microkernel keeps out of shared layers, and what Codex's exec-only `ExecExpiration` scope demonstrates.
|
||||
|
||||
### The library surface
|
||||
|
||||
Three functions plus one reason type:
|
||||
Four functions, one watchdog interface, and one reason type:
|
||||
|
||||
```ts ignore-check
|
||||
/** The internal reason attached to a timeout abort, so consumers can classify it after the fact. */
|
||||
@@ -53,19 +51,34 @@ export function deadline(
|
||||
code: string,
|
||||
): { signal: AbortSignal; [Symbol.dispose](): void }
|
||||
|
||||
/** A stable signal plus one-at-a-time, timer-guarded async-iterator demand. */
|
||||
export interface IdleWatchdog {
|
||||
readonly signal: AbortSignal
|
||||
next<T>(iterator: AsyncIterator<T>): Promise<IteratorResult<T>>
|
||||
[Symbol.dispose](): void
|
||||
}
|
||||
|
||||
/** Arm only while one iterator `next()` is outstanding, then rearm on later demand. */
|
||||
export function idleWatchdog(
|
||||
upstream: AbortSignal | undefined,
|
||||
timeoutMs: number,
|
||||
code: string,
|
||||
): IdleWatchdog
|
||||
|
||||
/** Recover the TimeoutReason from an aborted signal (or error); `code` scopes the match to this deadline's timer. */
|
||||
export function timeoutOf(x: AbortSignal | { reason?: unknown }, code?: string): TimeoutReason | undefined
|
||||
```
|
||||
|
||||
`deadline` is `AbortSignal.any([upstream, <timeout controller>])` with three things the standard library does not give: a typed, identifiable `TimeoutReason` on the timeout abort (native `AbortSignal.timeout()` yields a fixed `TimeoutError`, indistinguishable across timeout kinds), an internal `timeoutMs <= 0` "no timeout" sentinel for backend-owned background work, and a `Symbol.dispose` cleanup that works with both `using` and manual disposal. `AbortSignal.any` is a Node ≥ 20 primitive; it is the single mechanism that fuses two abort sources into one, adopting the reason of whichever fires first. External request hints validate as positive finite numbers via `clampTimeout` before they reach `deadline`; `0` is not a model-/plugin-facing "disable timeout" value. When `timeoutMs <= 0` and no upstream signal is present, `deadline()` returns a never-aborting signal plus a no-op disposer so callers keep one call shape. `TimeoutReason` is an internal classification reason: providers translate it into seam-specific public errors or result fields before returning to callers. `timeoutOf`'s optional `code` scopes classification to the caller's own deadline: when the `upstream` is itself a deadline (a future `tools/execute` middleware arming a per-call deadline), `AbortSignal.any` preserves the outer `TimeoutReason` if it fires first, and an unscoped match would misreport the outer timeout as the inner capability's own; scoping to `code` reads a foreign timeout as an ordinary upstream cancel.
|
||||
`deadline` fuses an upstream signal with a one-shot timer through `AbortSignal.any`, adds a typed `TimeoutReason`, and exposes disposable timer cleanup. Non-positive timeouts are an internal no-timeout sentinel for backend-owned background work; external hints pass through `clampTimeout` and must be positive and finite. Without a timer or upstream signal, the function returns a never-aborting signal with the same disposal shape. `idleWatchdog` instead requires a positive finite interval, keeps one stable fused signal for the entire stream, and arms its timer only while one iterator `next()` is outstanding; resolution disarms it, later demand rearms it, concurrent demand fails, and disposal clears the active arm. Providers translate timeout reasons into seam-specific results. `timeoutOf(signal, code)` scopes classification so an outer nested deadline is treated as upstream cancellation rather than the inner capability's timeout.
|
||||
|
||||
### The division of labor
|
||||
|
||||
| Concern | Owner |
|
||||
|---|---|
|
||||
| Validate request hint and clamp default/max | `dsh-timeout` (`clampTimeout`) — pure arithmetic plus the shared positive-finite request contract |
|
||||
| Arm timer, abort on deadline, carry reason, fuse with upstream cancel | `dsh-timeout` (`deadline`) |
|
||||
| Clear the timer | `dsh-timeout` (`[Symbol.dispose]`) |
|
||||
| Arm one-shot timer, abort on deadline, carry reason, fuse with upstream cancel | `dsh-timeout` (`deadline`) |
|
||||
| Arm and rearm only around outstanding iterator demand | `dsh-timeout` (`idleWatchdog`) |
|
||||
| Clear the timer | `dsh-timeout` (`[Symbol.dispose]` on either primitive) |
|
||||
| Classify the first abort reason after abort | `dsh-timeout` (`timeoutOf`) |
|
||||
| **Actually terminate the work** | the capability's implementation |
|
||||
| The default/max *values* | the capability's config |
|
||||
@@ -76,7 +89,8 @@ The signal only *notifies*; termination is always the listener's job, and the li
|
||||
### How each capability consumes it
|
||||
|
||||
- **web_fetch** — the tool stays validate-and-forward; the provider's hand-rolled controller + `setTimeout` + manual listener + `finally` + `signal.reason` recovery is replaced by provider-owned `deadline`/`timeoutOf`. A pre-aborted upstream signal still throws `WEB_ABORTED` up front; otherwise `fetch` runs against the fused `d.signal`, and `translateAbortOrNetwork` classifies a thrown error by the signal (`timeoutOf` → `WEB_FETCH_TIMEOUT`, else aborted → `WEB_ABORTED`, else network → `WEB_PROVIDER_ERROR`). The public error-code contract is unchanged, and `TimeoutReason` never crosses the web seam as the public error.
|
||||
- **bash** — `resolve()` stays a pure request-to-spec step: it clamps with `clampTimeout(request.timeoutMs, config.timeoutMs, config.maxTimeoutMs, 'bash-local: request.timeoutMs')` and carries `request.signal` through unchanged. Foreground `run()` owns the timeout: `using d = deadline(spec.signal, spec.timeoutMs, 'BASH_TIMEOUT')`, then `runBash` receives only `d.signal`. `runBash` no longer owns any timer — it listens for abort and runs its existing SIGTERM→grace→SIGKILL process-group kill, and its `SpawnSpec`/`SpawnOutcome` no longer carry `timeoutMs`/`timedOut`/`aborted` (the executor classifies from the deadline signal instead). `run()` computes `timedOut = timeoutOf(d.signal, 'BASH_TIMEOUT') !== undefined` and `aborted = d.signal.aborted && !timedOut`, so the public seam booleans (`BashRunResult.timedOut`/`aborted`) are mutually exclusive — the shared deadline reports the cause that first cut the command short, and the `code` scope keeps a nested outer deadline from being misread as bash's own timeout. Background `start()` creates no deadline and forwards only the upstream signal, so background tasks stay timeout-free; a task's killed-vs-completed status reads its own `spec.signal.aborted`.
|
||||
- **bash** — `resolve()` clamps the request into an explicit spec. Foreground `run()` creates the deadline and passes its signal to process execution, whose existing abort listener performs the process-group kill. The executor classifies the first abort as timeout or cancellation. Background starts remain timeout-free and forward only upstream cancellation.
|
||||
- **LLM adapters** — `dsh-llm-deepseek` and `dsh-llm-pi-ai` wrap actual transport iteration with `idleWatchdog`. The five-minute configured interval covers only outstanding provider demand, not time the downstream consumer spends between chunks. The stable signal reaches `fetch` or the SDK for the whole call, so timeout closes the underlying request and maps to `TIMEOUT`, while an earlier caller abort maps to `ABORTED`.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -84,6 +98,7 @@ The signal only *notifies*; termination is always the listener's job, and the li
|
||||
- `SpawnSpec.timeoutMs` and `SpawnOutcome.timedOut`/`aborted` were removed rather than kept as always-zero/always-false vestiges: with `runBash` owning no timer and the executor owning classification, they were read nowhere. This is the one deviation from the literal proposal shape (which passed `timeoutMs: 0` into `runBash`); an always-0 field read by nothing is dead weight under the per-file coverage gate.
|
||||
- web_fetch shed its bespoke controller/timer/listener/reason-recovery; the classifier now keys off the deadline signal (`timeoutOf` + `aborted`) rather than the thrown error's shape, which is robust across both the request-phase reject-with-reason and the read-phase bare-`AbortError`.
|
||||
- `AbortSignal.any` and `using`/`Symbol.dispose` enter the repo for the first time here (Node ≥ 24 baseline, already met).
|
||||
- Model streams now share one rearmable timer contract without turning a sliding idle interval into a total-call deadline or charging consumer think time. The primitive still only notifies; adapter tests prove their transports observe its stable signal and terminate.
|
||||
|
||||
Out of scope, named to mark the boundary: `web_search` can gain an optional model-facing `timeout_ms` once its tool-schema/snapshot coverage is planned; future ripgrep-backed fs discovery tools can consume the same provider-owned deadline shape once they exist; a `tools/execute` waterfall middleware could arm a default deadline for every tool call by driving `exec.signal` — that would be a plugin that *consumes* this library and still only notifies, the hard kill remaining each capability's job.
|
||||
|
||||
@@ -95,4 +110,4 @@ Out of scope, named to mark the boundary: `web_search` can gain an optional mode
|
||||
|
||||
**A `withTimeout(promise, ms)` wrapper instead of a signal factory.** Rejected because racing a promise against a timer resolves the *tool-call* promise on deadline without stopping the underlying work — the child process or fetch socket leaks on. Handing out a signal and requiring the capability to listen is what forces a real termination path to exist. This mirrors the "dispose must reach quiescence, not just request it" defensive rule.
|
||||
|
||||
**Keep bash's two independent triggers (`killTimer` + `onAbort`) rather than fusing.** Rejected for the convergence goal: fusing into one `deadline` signal removes bash's bespoke timer and gives every capability one shape. The trade-off is that bash's `timedOut`/`aborted` booleans become first-abort classifications rather than independent facts that can both be true when timeout and user abort race before process close. That is acceptable because the result reports the cause that first cut the command short; the termination action stays the same uniform SIGTERM→grace→SIGKILL kill. Note the deliberate non-alignment with Codex: Codex forks its kill by outcome (timeout → immediate SIGKILL; cancel → SIGTERM + 50 ms grace → SIGKILL), whereas the fused signal drives one uniform `kill()` for both, matching Claude Code's unified bash kill. Splitting the kill by `timeoutOf` is possible later if a need appears; there is none now.
|
||||
**Keep separate bash timeout and cancellation triggers.** Rejected because one deadline signal removes the bespoke timer and standardizes classification. Racing causes report whichever abort arrived first, while the existing SIGTERM-to-SIGKILL termination path remains unchanged.
|
||||
@@ -0,0 +1,155 @@
|
||||
# Agent Note: Tool result retention library
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Several model-facing tools already bound the amount of context they return, but each one owns a different local mechanism and vocabulary: bash keeps a tail plus spill files, web search caps source lists, web fetch caps body content, and `glob` / `grep` discovery needs an inline first page while keeping exact omission metadata for the full result set. A single `truncate(text)` helper cannot cover those cases: item tools need item counts and grouping outside the primitive, while text tools need byte budgets and UTF-8-safe head/tail cuts.
|
||||
|
||||
The shared abstraction the tools need is **retention**, not generic collection. A caller feeds items or text chunks into a bounded object and later receives the retained content plus exact omission metadata. Tool-specific code still owns business semantics: file grouping, line numbering, exit codes, provider error states, spill files, and model-facing prose. The common library owns only the mechanical question "what did we keep, and what did we omit?"
|
||||
|
||||
## Decision
|
||||
|
||||
`@deepseek-ai/dsh-retention` lives under `packages/util/` (peer to `dsh-brand` and `dsh-timeout`) and owns bounded model-facing output. It is a library of pure classes and functions, **not** a Cordis service or plugin: it takes no `ctx`, registers nothing, holds no cross-call state, and emits no events. Tool packages import it directly when they need bounded output.
|
||||
|
||||
The library has two independent retainers:
|
||||
|
||||
- `ItemRetainer<T>` handles ordered logical units such as paths, grep matches, or search sources. It supports `head` retention only in v1, while keeping the retainer shape open to additional retention strategies later.
|
||||
- `TextRetainer` handles byte-oriented text streams such as bash stdout/stderr or web response bodies. It supports `head`, `tail`, and `headTail` retention while preserving UTF-8 boundaries at `finish()`.
|
||||
|
||||
Both retainers return a small `PushDecision` after each `push()` so callers can tell whether that unit/chunk was fully retained and whether the accumulated result is now truncated. Omission counts are exact because callers keep feeding every observed item/chunk.
|
||||
|
||||
```ts ignore-check
|
||||
/**
|
||||
* How much content the retainer omitted.
|
||||
*
|
||||
* `unknown` is reserved for callers that omit without a count; the retainers
|
||||
* themselves return `none` or `exact`.
|
||||
*/
|
||||
type Omitted =
|
||||
| { kind: 'none' }
|
||||
| { kind: 'exact'; count: number }
|
||||
| { kind: 'unknown' }
|
||||
|
||||
interface PushDecision {
|
||||
kept: boolean
|
||||
truncated: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Final result for ordered logical units.
|
||||
*/
|
||||
interface RetainedItems<T> {
|
||||
items: T[]
|
||||
truncated: boolean
|
||||
seen: number
|
||||
kept: number
|
||||
omitted: Omitted
|
||||
}
|
||||
|
||||
/**
|
||||
* Final result for text streams.
|
||||
*
|
||||
* The returned `text` is safe to send to a formatter; the retainer does not add
|
||||
* tool-specific headers, exit markers, XML tags, or recovery instructions.
|
||||
*/
|
||||
interface RetainedText {
|
||||
text: string
|
||||
truncated: boolean
|
||||
omittedBytes: Omitted
|
||||
}
|
||||
```
|
||||
|
||||
### Strategies
|
||||
|
||||
Item retention supports a head window. Text retention supports head, tail, and headTail byte windows.
|
||||
|
||||
```ts ignore-check
|
||||
type ItemRetentionStrategy =
|
||||
| {
|
||||
/** Keep the first `maxItems` units. Use for `glob`, `grep`, and web sources. */
|
||||
kind: 'head'
|
||||
maxItems: number
|
||||
}
|
||||
|
||||
type TextRetentionStrategy =
|
||||
| {
|
||||
/** Keep the first `maxBytes` bytes. */
|
||||
kind: 'head'
|
||||
maxBytes: number
|
||||
}
|
||||
| {
|
||||
/** Keep the final `maxBytes` bytes. Requires reading to the end. */
|
||||
kind: 'tail'
|
||||
maxBytes: number
|
||||
}
|
||||
| {
|
||||
/** Keep a stable prefix and suffix, omitting the middle. Requires reading to the end. */
|
||||
kind: 'headTail'
|
||||
headBytes: number
|
||||
tailBytes: number
|
||||
}
|
||||
```
|
||||
|
||||
### Tool mapping
|
||||
|
||||
`read` is intentionally outside the v1 retention library. Its `read-render` helper owns a file-specific pagination contract: `offset` / `limit`, line numbers, `totalLines`, offset-out-of-range errors, per-line preview truncation, and a selected-output byte cap that can stop scanning mid-window. That is a line-window renderer, not a generic retention primitive. It may share future neutral notice helpers, but it should not pass its already-selected window through `ItemRetainer`.
|
||||
|
||||
`FsGlobEntry` and `FlatGrepMatch` below are the intended discovery-tool item shapes, not existing retention-library exports. `FsGlobEntry` is one backend-derived path, and `FlatGrepMatch` is one ungrouped grep match before the backend groups retained matches by file.
|
||||
|
||||
`glob` uses `ItemRetainer<FsGlobEntry>` with `{ kind: 'head', maxItems: globMaxResults }` after collecting the full sorted path list. The tool keeps the retained first page inline and may save the full list through the spill seam. Path mapping, skipped candidates, and `incomplete` stay outside the retainer.
|
||||
|
||||
`grep` uses `ItemRetainer<FlatGrepMatch>` with `{ kind: 'head', maxItems: grepMaxMatches }` before grouping. The executor parses ripgrep output, maps paths, applies per-line preview truncation, and pushes flat matches. After `finish()`, the tool groups retained matches by file and can save the full match list through the spill seam when the inline result is capped. Grouping is not part of the retainer because the cap is total matches, not files; per-match preview truncation and `incomplete` are also separate from result-level retention.
|
||||
|
||||
`bash` can use `TextRetainer` with `tail` or `headTail` and reads to process completion. The bash executor still owns spill files, exit status, signal, timeout, and background-task behavior; the retention helper only replaces ad hoc in-memory head/tail accounting where that behavior is desired. Long-running task ownership remains orthogonal to the [generic long-running tool runtime](2026-06-20-generic-long-running-tool-runtime.md).
|
||||
|
||||
`web_fetch` can use `TextRetainer` with `head` or `headTail`, or keep provider-owned body caps when the provider must read and decode internally. Either way, the fetch result's `truncated` remains a provider/tool fact, and the library only supplies retained text and omission metadata.
|
||||
|
||||
`web_search` can use `ItemRetainer<WebSearchSource>` with `head`. Current providers often return an array, so this is post-hoc but still standardizes notices.
|
||||
|
||||
### Notices
|
||||
|
||||
The library exposes a neutral notice shape and a tiny formatter hook, but tools provide the user-facing words. A grep footer says "Narrow the pattern, path, or include"; a web fetch footer says "Fetch a more specific URL or section"; bash may point to a spill file. The retainer cannot know those recovery actions.
|
||||
|
||||
```ts ignore-check
|
||||
interface RetentionNotice {
|
||||
scope: string
|
||||
strategy: 'head' | 'tail' | 'headTail'
|
||||
unit: 'items' | 'bytes' | 'chars' | 'lines'
|
||||
limit: number | { head: number; tail: number }
|
||||
kept: number
|
||||
omitted: Omitted
|
||||
}
|
||||
|
||||
const formatGrepNotice = (notice: RetentionNotice): string =>
|
||||
formatRetentionNotice(
|
||||
notice,
|
||||
({ kept }) => `Results capped at ${kept}. Narrow the pattern, path, or include to see more.`,
|
||||
)
|
||||
```
|
||||
|
||||
The formatter hook is deliberately small: a tool turns a `RetentionNotice` into its own footer text. The helper may standardize omission wording, but it does not own recovery guidance.
|
||||
|
||||
`truncated` means the retainer omitted otherwise-available content because of a budget. It does not mean the upstream was incomplete. Tools keep separate fields for permission failures, skipped binary files, provider partial failures, unreadable candidates, invalid UTF-8, and any other "could not inspect" condition.
|
||||
|
||||
## Consequences
|
||||
|
||||
**What shipped.** `@deepseek-ai/dsh-retention` exports `ItemRetainer`, `TextRetainer`, the result types (`RetainedItems`, `RetainedText`), the strategy types (`ItemRetentionStrategy`, `TextRetentionStrategy`), `Omitted`, `PushDecision`, `RetentionNotice`, and the neutral notice helpers `describeOmitted` / `formatRetentionNotice` — with no dependency on Cordis or any tool package. Unit tests cover item-head retention with exact omission counts, text-head retention, text-tail retention, head-tail byte retention, zero budgets, UTF-8 boundary handling (2-, 3-, and 4-byte codepoints and invalid lead bytes at each cut), and unknown omission wording.
|
||||
|
||||
**What is documented but not yet migrated.** `glob`, `grep`, `bash`, `web_fetch`, and `web_search` have their mappings documented in the [package README](../../../../packages/util/retention/README.md), but not every tool has been migrated onto the library in this change; migration is deliberately separate follow-up work. `read` is documented as intentionally out of scope: its `read-render` line-window contract (`offset`/`limit`, `totalLines`, offset-range errors, per-line preview truncation, a byte cap over the selected window) is not generic retention, and one `Omitted` count cannot represent both sides of a line window.
|
||||
|
||||
**Boundaries the library holds.** `truncated` means the retainer omitted otherwise-available content because of a budget; it never means the upstream was incomplete. Tool-specific states — `incomplete`, permission failures, provider partial failures, binary skips, bash spill-path recovery, invalid UTF-8 — stay in tool-domain fields, outside the retainer. When a future change migrates a tool, that package's README and tests must prove the model-facing result text is unchanged except for deliberate notice wording.
|
||||
|
||||
**Tradeoffs accepted.** The v1 surface deliberately supports only item `head` retention and text `head` / `tail` / `headTail`; windows, grouped budgets, sort-aware caps, and upstream-stop control wait until a second consumer proves the need. Text retention counts bytes for process/body safety, leaving character- and line-level preview budgets as separate tool-owned concerns.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Post-hoc `truncate(text)` only.** Rejected: it matches Codex's history/tool-output truncation use case but loses item counts, grouping boundaries, UTF-8-safe byte windows, and exact omission metadata.
|
||||
|
||||
**One generic `Collector<T>` with pluggable callbacks.** Rejected for v1: it hides the two important resource modes. Logical item retention counts items; text retention counts bytes and preserves UTF-8 boundaries. Separate `ItemRetainer` and `TextRetainer` names make that difference explicit while keeping the API small.
|
||||
|
||||
**Put `read` windowing behind `ItemRetainer`.** Rejected for v1: `read` is the only current window consumer, and its semantics are file pagination rather than generic retention. A single `Omitted` count cannot represent both sides of a line window, and `read` also carries `totalLines`, offset-range errors, per-line preview truncation, and a byte cap over selected output. Keeping `read-render` tool-owned avoids growing the shared library around one special case.
|
||||
|
||||
**Make truncation part of `ToolExecutionResult`.** Rejected: the tool registry would have to understand tool-specific recovery guidance, grouping, line numbering, exit status, and provider semantics. Retention is a library used before a tool returns `ContentBlock[]`; the model-facing result remains tool-owned.
|
||||
|
||||
**Expose limits in every model-facing tool schema.** Rejected as the default: Claude Code's grep exposes `head_limit` / `offset`, but this harness keeps routine budgets as deployment config unless the model genuinely needs pagination control. A future read-like continuation field can be added per tool; it does not belong in the shared retention primitive.
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: Tool-call timeout policy as a plugin
|
||||
# Agent Note: Tool-call timeout policy as a plugin
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The [timeout/deadline RFC](2026-07-06-timeout-deadline-library.md) extracted the timing-and-classification primitive into `@deepseek-ai/dsh-timeout`, but timeout policy was still attached to individual capabilities and model-facing schemas. `bash` exposed `timeoutMs`; `web_fetch` exposed `timeout_ms`; `web_search` had no model-facing timeout even though providers already honor `exec.signal`; a future grep/glob tool would either import the timeout library directly or invent its own timeout policy. That is the wrong authoring shape for a plugin SDK: a tool author should normally forward `exec.signal` to the implementation it calls, and deployment policy should decide the budget.
|
||||
The [timeout/deadline Agent Note](2026-07-06-timeout-deadline-library.md) extracted the timing-and-classification primitive into `@deepseek-ai/dsh-timeout`, but timeout policy was still attached to individual capabilities and model-facing schemas. `bash` exposed `timeoutMs`; `web_fetch` exposed `timeout_ms`; `web_search` had no model-facing timeout even though providers already honor `exec.signal`; a future grep/glob tool would either import the timeout library directly or invent its own timeout policy. That is the wrong authoring shape for a plugin SDK: a tool author should normally forward `exec.signal` to the implementation it calls, and deployment policy should decide the budget.
|
||||
|
||||
At the same time, not every timeout in the repo is a model-facing tool-call budget. Hooks execute command hooks by calling `ctx.bash` directly, not through `ctx.tools.execute()`, and the `bash` model tool multiplexes foreground execution, background start, background polling, and hook reuse through the same backend. Moving every timeout into a tool plugin in one step would conflate those paths and risk breaking hook timeout semantics.
|
||||
|
||||
@@ -50,16 +50,15 @@ The plugin is `@deepseek-ai/dsh-timeout-policy`, a zero-config function/namespac
|
||||
searchTimeoutMs: 30000
|
||||
```
|
||||
|
||||
Keeping the tool name out of this plugin's config is deliberate: a budget keyed by a free-text tool name could be mistyped (`web_fech`) and then silently apply to nothing. Declaring `timeoutMs` on the tool makes that failure class structurally impossible — the enforcer reads `ctx.tools.get(exec.name)?.timeoutMs`, and `exec.name` is the tool being dispatched, so the lookup always resolves and there is no unknown-name path to warn or throw about. `timeoutMs` is validated positive-finite by `defineTool` at definition time. For a tool that declares a budget the listener arms `deadline(exec.signal, timeoutMs, 'TOOL_TIMEOUT')`, swaps the derived signal onto `exec` for the downstream dispatch, restores the caller's own signal afterward, and returns a structured `TOOL_TIMEOUT` result when `timeoutOf(d.signal, 'TOOL_TIMEOUT')` matches. A tool with no declared budget delegates unchanged.
|
||||
Timeouts live on tool definitions rather than a free-text name map, eliminating misspelled unused policy. `defineTool` validates a positive finite budget. During dispatch the enforcer derives a deadline signal and assigns it to `exec.signal`; the registry fuses that deadline with the original caller signal before the body under the [tool-cancellation contract](2026-07-19-cooperative-tool-cancellation.md). The enforcer restores the caller signal afterward and converts its own expiry into `TOOL_TIMEOUT`; tools without a budget pass through unchanged.
|
||||
|
||||
Signal replacement is by **in-place mutation of `exec.signal`**, not by passing a new object to `next()`. Cordis's waterfall `next()` ignores any arguments handed to it and re-invokes downstream listeners with the shared payload array (`vendor/cordis/src/events.ts`), so the documented cordis idiom — mutate the shared object, then delegate — is the only mechanism that reaches dispatch. The plugin restores `exec.signal` to the caller's original in a `finally` so `tools/post-execute` never sees this plugin's (possibly already-aborted) deadline signal.
|
||||
Signal replacement is by **in-place mutation of `exec.signal`**, not by passing a new object to `next()`. Cordis's waterfall `next()` ignores any arguments handed to it and re-invokes downstream listeners with the shared payload array (`vendor/cordis/src/events.ts`), so mutation is how the wrapper supplies its deadline to the registry. The registry re-fuses the captured caller signal immediately before the body, and the plugin restores `exec.signal` to the caller's original in a `finally` so `tools/post-execute` never sees the plugin's deadline signal.
|
||||
|
||||
`timeout-policy` owns both uses of the `TOOL_TIMEOUT` code: the internal deadline code passed to `deadline()`/`timeoutOf()` (scoped so a nested outer deadline reads as an ordinary cancel) and the structured tool-result error code. Its replacement result is:
|
||||
|
||||
```ts ignore-check
|
||||
function toolTimeoutResult(callId: CallId, timeoutMs: number): ToolExecutionResult {
|
||||
function toolTimeoutResult(timeoutMs: number): ToolExecutionResult {
|
||||
return {
|
||||
callId,
|
||||
content: [{ type: 'text', text: `Error: tool call timed out after ${timeoutMs}ms` }],
|
||||
isError: true,
|
||||
error: { name: 'ToolTimeoutError', code: 'TOOL_TIMEOUT' },
|
||||
@@ -75,17 +74,17 @@ No new session event is needed for reconstructability: `TOOL_TIMEOUT` is the fin
|
||||
|
||||
`web_fetch` and `web_search` are migrated. `dsh-tool-web` keeps ownership of their model-facing schemas, and those schemas expose no timeout knob: `web_fetch` dropped its `timeout_ms` parameter to match the reference-agent shape, and `web_search` stays query-only. The tool bodies do not import `@deepseek-ai/dsh-timeout`; they forward `exec.signal` to `ctx.web`.
|
||||
|
||||
`dsh-web-fetch-local` keeps a provider-level timeout (`timeoutMs`/`maxTimeoutMs`) as a large resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments; it owns no model-facing timeout. When a `TOOL_TIMEOUT` signal reaches the fetch provider first, provider-scoped classification treats it as upstream `WEB_ABORTED`, and the outer `tools/execute` wrapper replaces the final tool result with `TOOL_TIMEOUT`. A shipped web-tool deployment configures the provider backstop above the `timeout-policy` budget so the tool-call policy normally wins for model calls.
|
||||
`dsh-web-fetch-local` keeps one configured provider-level `timeoutMs` as a large resource backstop for direct `ctx.web.fetch()` callers and misconfigured deployments; it owns no model-facing timeout. When a `TOOL_TIMEOUT` signal reaches the fetch provider first, provider-scoped classification treats it as upstream `WEB_ABORTED`, and the outer `tools/execute` wrapper replaces the final tool result with `TOOL_TIMEOUT`. A shipped web-tool deployment configures the provider backstop above the `timeout-policy` budget so the tool-call policy normally wins for model calls.
|
||||
|
||||
`bash` stays on the current backend timeout path. `dsh-tool-bash` continues to expose `timeoutMs` and `run_in_background`; `dsh-bash-local` continues to use `@deepseek-ai/dsh-timeout` for `BASH_TIMEOUT`; hook bridges continue to call `runHook()` and pass `timeoutMs` through `ctx.bash`. This keeps foreground/background/hook behavior stable.
|
||||
|
||||
`read`, `write`, `edit`, `todo_write`, `bash_output`, and `bash_kill` do not opt into tool-call timeout: they are local filesystem or short registry/session operations where a deadline would be best-effort only or unnecessary.
|
||||
`read`, `write`, `edit`, `todo_write`, `task_list`, and `task_kill` do not opt into tool-call timeout. `task_output` owns its bounded wait because a wait timeout is a successful live-status result, not a tool failure.
|
||||
|
||||
A future model-facing grep/glob tool can be implemented on top of `ctx.bash` without importing `@deepseek-ai/dsh-timeout`: it forwards `exec.signal` to `ctx.bash`, and declares its own `timeoutMs` (from its plugin's config) for the enforcer to apply. If bash-local's backend timeout becomes a problem for such a tool, the bash seam can later add a caller-owned-deadline mode; that is outside this cut.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Name the plugin `tool-timeout`.** The literal RFC name matched the `gen-tool-catalog` completeness guard's `packages/*/tool-*` glob, which requires every match to register a model-facing tool. This plugin registers none — it is a `tools/execute` wrapper — so a `tool-*` name would either fail `verify-tool-catalog` or force a misleading boot entry. The package is `@deepseek-ai/dsh-timeout-policy` in a new `packages/timeout/` group; the cordis.yml `id` can still be `timeout-policy`.
|
||||
**Name the plugin `tool-timeout`.** The literal Agent Note name matched the `gen-tool-catalog` completeness guard's `packages/*/tool-*` glob, which requires every match to register a model-facing tool. This plugin registers none — it is a `tools/execute` wrapper — so a `tool-*` name would either fail `verify-tool-catalog` or force a misleading boot entry. The package is `@deepseek-ai/dsh-timeout-policy` in a new `packages/timeout/` group; the cordis.yml `id` can still be `timeout-policy`.
|
||||
|
||||
**Keep per-tool timeout handling only.** This was the shape for `bash` and `web_fetch`, and it matches Claude Code and Codex for shell commands. It loses for web-style tools because every new timeout-capable tool must choose validation, cap semantics, docs, snapshots, and classification. The plugin centralizes policy and classification while leaving each tool's schema focused on business input.
|
||||
|
||||
@@ -99,12 +98,12 @@ A future model-facing grep/glob tool can be implemented on top of `ctx.bash` wit
|
||||
|
||||
**Use `tools/pre-execute` plus `tools/post-execute` instead of a new around seam.** A pre listener could arm a deadline and mutate `exec.signal`; a post listener could classify and replace. That loses because the deadline lifetime would cross two independent waterfalls: a call-id map, cleanup on every pre-deny/tool-throw/post-throw/dispose path, and ordering rules with every other listener. `tools/pre-execute` is also the allow/deny gate, not an execution wrapper. `tools/execute` gives the timeout one lexical scope: arm, delegate, classify, dispose.
|
||||
|
||||
**Use `Promise.race` to enforce timeouts for non-cooperative tools.** Rejected for the same reason as the timeout-library RFC: it returns control to the caller while the underlying process, fetch, or provider operation may still be running. The plugin only sends a signal; termination remains the implementation's responsibility.
|
||||
**Use `Promise.race` to enforce timeouts for non-cooperative tools.** Rejected for the same reason as the timeout-library Agent Note: it returns control to the caller while the underlying process, fetch, or provider operation may still be running. The plugin only sends a signal; termination remains the implementation's responsibility.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `@deepseek-ai/dsh-tools` gains an around-dispatch surface after the interception seams deliberately split pre/post tool hooks. Its contract is narrow — wrap registry dispatch, not replace the pre-gate or post-result policy — and the base `next()` is dispatch-with-normalization so a wrapper never sees a raw tool throw.
|
||||
- Multiple `tools/execute` listeners compose by ordinary Cordis waterfall order: a listener that calls `next()` wraps downstream listeners plus dispatch; one that returns without `next()` short-circuits them. A deployment combining timeout with a future retry/sandbox/metrics wrapper chooses semantics by registration order ("timeout covers the whole retry" vs "timeout covers each attempt").
|
||||
- Opt-in by declaration is a deliberate misconfiguration risk: a tool can declare a `timeoutMs` without honoring `exec.signal`, and that tool will not stop on timeout. The plugin contract states that declaring a budget means cooperative; the web tools prove the pattern on tools that already forward the signal.
|
||||
- Opt-in by declaration is a deliberate misconfiguration risk: a tool can declare a `timeoutMs` without honoring `exec.signal`, and that tool will not stop on timeout. The registry awaits that non-quiescent body rather than racing it, while the plugin contract states that declaring a budget means cooperative; the web tools prove the pattern on tools that already forward the signal.
|
||||
- During the transition `bash` and the migrated web tools use different timeout paths on purpose: `TOOL_TIMEOUT` is the model-facing tool-call budget, while `BASH_TIMEOUT` remains the bash backend timeout used by bash and hooks.
|
||||
- Deviation from the literal proposal, recorded per the implemented-RFC rule: the plugin package is `@deepseek-ai/dsh-timeout-policy` (not `tool-timeout`), signal replacement is in-place `exec.signal` mutation before `next()` (not `next({ ...exec, signal })`, which cordis ignores), and the per-tool budget is declared on the `ToolDefinition` (`timeoutMs`, set by the owning tool plugin from its config) rather than mapped by tool name in this plugin's config — so the enforcer is zero-config and a mistyped tool name is impossible. All three are described in `## Decision` above.
|
||||
- Deviation from the literal proposal, recorded per the implemented-Agent Note rule: the plugin package is `@deepseek-ai/dsh-timeout-policy` (not `tool-timeout`), signal replacement is in-place `exec.signal` mutation before `next()` (not `next({ ...exec, signal })`, which cordis ignores), and the per-tool budget is declared on the `ToolDefinition` (`timeoutMs`, set by the owning tool plugin from its config) rather than mapped by tool name in this plugin's config — so the enforcer is zero-config and a mistyped tool name is impossible. All three are described in `## Decision` above.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: The agent is a registration scope
|
||||
# Agent Note: The agent is a registration scope
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -14,7 +14,7 @@ The mechanism also needs a publication boundary. An agent must not become visibl
|
||||
|
||||
Every live agent owns one flat registration layer exposed as `agent.ctx`. Code registers through the context that owns a contribution; scope-aware services combine deployment-global registrations with exactly one matching agent layer; operations choose that layer from their real agent; and the layer exists for the agent's complete published lifetime.
|
||||
|
||||
Cordis is the plugin framework underneath the SDK. A Cordis **context** is the object plugins use to access services and register effects whose cleanup follows that context. The [Cordis primer](../../../cordis-primer.md) explains the framework in more detail.
|
||||
Cordis is the plugin framework underneath the SDK. A Cordis **context** is the object plugins use to access services and register effects whose cleanup follows that context. The [Cordis primer](../../../../docs/cordis-primer.md) explains the framework in more detail.
|
||||
|
||||
For most contributors, the complete contract is four rules:
|
||||
|
||||
@@ -43,7 +43,7 @@ flowchart LR
|
||||
|
||||
The missing cross-edges are the isolation rule: Agent A's local registrations do not enter Agent B's view, and a parent's registrations do not enter a child merely because the parent owns the child's lifetime.
|
||||
|
||||
The companion [runtime-design RFC](2026-07-12-agent-scope-runtime-design.md) explains the implementation and correctness reasoning. The [subagent composition-controls RFC](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) owns the separate `persona`, `toolFilter`, and `maxDepth` feature.
|
||||
The companion [runtime-design Agent Note](2026-07-12-agent-scope-runtime-design.md) explains the implementation and correctness reasoning. The [subagent composition-controls Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) owns the separate `persona`, `toolFilter`, and `maxDepth` feature.
|
||||
|
||||
### Registration origin chooses visibility and cleanup
|
||||
|
||||
@@ -60,8 +60,7 @@ The ordinary contributor pattern is to register the complete local world during
|
||||
|
||||
```js
|
||||
const handle = await ctx.agents.create({
|
||||
agentId: AgentId('reviewer'),
|
||||
sessionId: SessionId('reviewer-session'),
|
||||
sessionId: SessionId('reviewer'),
|
||||
agentOptions: { model: 'model-name' },
|
||||
setup(agentCtx) {
|
||||
agentCtx.systemPrompt.section({
|
||||
@@ -103,7 +102,7 @@ An event about Agent A normally reaches unscoped listeners and A-scoped listener
|
||||
|
||||
At the Cordis level, `Scoped<T>` is an opaque routing receiver. It carries the filter used to choose listeners but is not the domain object. Event signatures therefore keep the real `Agent`, tool execution, approval request, or other subject as an explicit argument that listeners can inspect.
|
||||
|
||||
A listener registered with `{ global: true }` deliberately bypasses contextual audience filtering while its cleanup still follows the registering context. Registry-membership notifications remain unfiltered because they describe shared registry state rather than one agent's operation. The generated [event catalog](../../../cordis-catalog/events.md) is the exhaustive event reference.
|
||||
A listener registered with `{ global: true }` deliberately bypasses contextual audience filtering while its cleanup still follows the registering context. Registry-membership notifications remain unfiltered because they describe shared registry state rather than one agent's operation. The generated [event catalog](../../../../docs/cordis-catalog/events.md) is the exhaustive event reference.
|
||||
|
||||
### Creation publishes last and disposal revokes last
|
||||
|
||||
@@ -0,0 +1,189 @@
|
||||
# Agent Note: Tool output spill policy
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Tool outputs need bounded model-facing previews, but some oversized results are still useful later. A fetched page body or a verbose tool response should not consume the next model request in full, but the model should be able to inspect the complete formatted result later with existing file-reading tools.
|
||||
|
||||
Before this change the behavior was uneven. `dsh-bash-local` already writes complete stdout/stderr streams to private temp spill files when its in-memory tail overflows, but ordinary text tool results were returned inline unless the tool hand-rolled its own cap. The [tool result retention library](2026-07-06-tool-result-retention-library.md) owns preview mechanics, but it does not own storage or an execution-pipeline policy that applies those mechanics to final tool results.
|
||||
|
||||
The shape matches the timeout policy design: a tool author normally returns the text result, and a policy plugin enforces the deployment's default context budget. Tool-specific early spill remains possible later for outputs that do not survive to the final `ToolExecutionResult`; the first cut proves the default final-result path.
|
||||
|
||||
## Decision
|
||||
|
||||
A thin spill storage seam plus a default spill policy plugin, in a new `packages/spill/` group:
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@deepseek-ai/dsh-spill` | Interface: `ctx.spillStore`, vocabulary types, no storage implementation. |
|
||||
| `@deepseek-ai/dsh-spill-local` | Local backend: private, session-scoped file storage on the host filesystem. |
|
||||
| `@deepseek-ai/dsh-spill-policy` | Tool-result policy plugin: wraps final text results after dispatch and replaces oversized results with a retained preview plus a spill locator. |
|
||||
|
||||
There is no dedicated model-facing consumer package. The consumer is the existing `ctx.tools` execution pipeline: `dsh-spill-policy` consumes final tool results through the `tools/post-execute` waterfall, and the model follows the backend-supplied retrieval hint for the returned locator.
|
||||
|
||||
### Spill seam
|
||||
|
||||
The storage seam is minimal: save text and return a locator plus retrieval hint.
|
||||
|
||||
```ts ignore-check
|
||||
interface SpillStore {
|
||||
saveText(input: SaveTextSpill): Promise<SpillRef>
|
||||
}
|
||||
|
||||
interface SpillSource {
|
||||
toolName: string
|
||||
callId: CallId
|
||||
label: string
|
||||
}
|
||||
|
||||
interface SaveTextSpill {
|
||||
owner: { sessionId: SessionId }
|
||||
source: SpillSource
|
||||
suggestedName: string
|
||||
content: string
|
||||
}
|
||||
|
||||
type SpillLocator = Branded<'SpillLocator'>
|
||||
|
||||
interface SpillRef {
|
||||
locator: SpillLocator
|
||||
bytes: number
|
||||
retrievalHint: string
|
||||
}
|
||||
```
|
||||
|
||||
`SpillLocator` is a [branded](../../../../packages/util/brand) model-facing handle returned by the backend. The local backend renders it as a filesystem path; a remote or database backend can render a URI, key, or command token. Consumers treat it as opaque and render it with `retrievalHint` instead of assuming `read` is always the right retrieval mechanism. `SpillOwner.sessionId` is the save-time storage namespace: forked sessions inherit existing spill locators from the seeded log without copying or re-owning them, and new spills after the fork use the child session id. A retention-period cleanup may expire old locators with other old session artifacts; the spill seam does not define a per-session cleanup policy.
|
||||
|
||||
`dsh-spill-local` owns only storage details: session-scoped directory selection, safe names, path-traversal protection, the write, and returning `{ locator, bytes, retrievalHint }`. It does not own retention policy, tool-result replacement, search, or file inspection. Files land at `<root>/session-<hash>/<random>-<safeName>`, where `root` is a configured path or a lazily-created private (0700) per-process temp dir, the session subdir is a short `sha256(sessionId)` prefix, and the leaf is a random hex prefix plus the caller's `suggestedName` sanitized to one path segment (mirrors the JSONL backend's `encodeSegment`). The write is `open(path, 'wx', 0o600)` — exclusive and owner-only, so a planted symlink cannot redirect it. The locator is the path, and the retrieval hint tells the model it can use `read` or `grep` on that path.
|
||||
|
||||
### Spill policy
|
||||
|
||||
`dsh-spill-policy` is a `tools/post-execute` result transformer with one configuration knob:
|
||||
|
||||
```ts ignore-check
|
||||
interface Config {
|
||||
/** Omitted means no automatic spill policy. Present means apply to oversized plain text tool results. */
|
||||
maxInlineBytes?: number
|
||||
}
|
||||
```
|
||||
|
||||
When `maxInlineBytes` is omitted the plugin registers nothing (a true no-op). When set, it applies a default policy to final plain-text tool results:
|
||||
|
||||
1. Let the tool run normally, delegating via `next()` so a downstream listener settles the result first.
|
||||
2. Flatten the accepted final `ContentBlock[]` only when it is entirely plain text; a result with any non-text block is left untouched.
|
||||
3. If its UTF-8 byte size is at or below `maxInlineBytes`, leave it unchanged.
|
||||
4. If it is larger, call `ctx.spillStore.saveText()` with the full final text.
|
||||
5. Replace the model-facing result with a retained head/tail preview plus the spill reference.
|
||||
|
||||
The preview is an implementation default owned by the policy: a head/tail split of `maxInlineBytes` via the retention library's `TextRetainer`. Future config can expose preview sizing only after a second deployment needs it.
|
||||
|
||||
The replacement text is intentionally generic because the policy only knows the final formatted tool result, not the tool's internal resource:
|
||||
|
||||
```text
|
||||
<retained preview>
|
||||
|
||||
(Omitted N bytes. Full formatted result stored at: /.../session-.../....txt. Use read with offset/limit, or grep this path to search within it.)
|
||||
```
|
||||
|
||||
If `ctx.spillStore.saveText()` fails (permissions, ENOSPC, backend unavailable), or the call has no session owner, or no backend is loaded, the plugin logs the reason and returns the original result unchanged. Spill failure never turns a successful tool call into an `isError` result or hides the inline result.
|
||||
|
||||
The policy skips `read` to avoid a circular `read -> spill file -> read again` loop. Additional opt-out configuration is deferred until a real second tool needs it.
|
||||
|
||||
## Showcase: web_fetch
|
||||
|
||||
`web_fetch` is the first showcase because it returns a naturally large text result and needs no tool-specific spill code. The tool is ordinary:
|
||||
|
||||
```ts ignore-check
|
||||
ctx.tools.register(defineTool({
|
||||
name: 'web_fetch',
|
||||
async execute(args, exec) {
|
||||
const result = await ctx.web.fetch({ url: args.url }, exec.signal ? { signal: exec.signal } : undefined)
|
||||
return [{ type: 'text', text: formatFetchOutput(result) }]
|
||||
},
|
||||
}))
|
||||
```
|
||||
|
||||
With `dsh-spill-policy` configured, a large formatted fetch result is automatically retained and spilled. A deployment demonstrates the behavior by setting the provider resource cap higher than the policy cap:
|
||||
|
||||
```yaml
|
||||
- id: web-fetch-local
|
||||
name: '@deepseek-ai/dsh-web-fetch-local'
|
||||
config:
|
||||
maxBodyChars: 500000
|
||||
|
||||
- id: spill-local
|
||||
name: '@deepseek-ai/dsh-spill-local'
|
||||
|
||||
- id: spill-policy
|
||||
name: '@deepseek-ai/dsh-spill-policy'
|
||||
config:
|
||||
maxInlineBytes: 50000
|
||||
```
|
||||
|
||||
This separation is important. `web-fetch-local` still owns resource caps (`maxResponseBytes`, `maxBodyChars`) to protect network, memory, and decoding work. `spill-policy` owns only the model-facing context cap after the result already exists. If the provider already returned `truncated: true`, the spill file contains the full formatted result the tool returned, not the full original webpage; the policy does not claim otherwise.
|
||||
|
||||
## Relationship to retention and early spill
|
||||
|
||||
Retention is separate from spill storage:
|
||||
|
||||
- `@deepseek-ai/dsh-retention` owns preview mechanics (`TextRetainer`, `ItemRetainer`, and omitted metadata).
|
||||
- `@deepseek-ai/dsh-spill` owns saving final text and returning a locator plus retrieval hint.
|
||||
- `@deepseek-ai/dsh-spill-policy` applies the default final-result policy in the tool pipeline, composing the two.
|
||||
|
||||
The final-result policy cannot replace tool-owned early spill. Some useful content is not present in final `ToolExecutionResult.content`:
|
||||
|
||||
- `bash` final output is already a tail plus a temp spill path; the complete stdout/stderr streams live in executor files.
|
||||
- `subagent` final output is the child final answer, not the child rollout.
|
||||
- Future tools may produce runtime artifacts that are never represented by their final `ToolExecutionResult.content`.
|
||||
|
||||
Those cases can consume `ctx.spillStore` directly in later work. They are not part of the first showcase.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No new model-facing `artifact_read` or `artifact_search` tool in v1.
|
||||
- No per-tool retention configuration in v1.
|
||||
- No model-facing timeout/truncation arguments.
|
||||
- No migration of `read` output into spill files.
|
||||
- No replacement for provider/resource caps such as `web-fetch-local.maxBodyChars`.
|
||||
- No bash temp-file normalization or subagent rollout capture in the first cut.
|
||||
|
||||
## Deferred
|
||||
|
||||
- `saveFile()` / `linkOrCopy` for existing executor spill files, needed for bash normalization.
|
||||
- Tool-owned spill for subagent rollouts (`await run.result`, read in-process child session before `run.dispose()`, save JSONL).
|
||||
- Per-tool opt-out or per-tool policy declarations if the built-in `read` skip is insufficient.
|
||||
- Remote or database storage backends for ACP or remote environments where a local path is not meaningful.
|
||||
- Cleanup and retention policy for old spill files, likely tied to session cleanup.
|
||||
|
||||
## Testing
|
||||
|
||||
- `dsh-spill` unit tests pin the seam contract: registration as `ctx.spillStore`, one-implementation-per-context, and disposal release.
|
||||
- `dsh-spill-local` unit tests cover `saveText`, `encodeSegment` sanitization (separators/tilde/whole-segment dots/empty), the session-hash directory, owner-only permissions, distinct paths per save, the configured/private root, and a storage-failure rejection.
|
||||
- `dsh-spill-policy` unit tests drive real tools through `ctx.tools.execute`: disabled-mode no-op, oversized-text replacement, small/non-text passthrough, `read` skip, best-effort fallback (save failure / no backend / no owner), and downstream-composition (bounding a replaced result, preserving `additionalContexts`).
|
||||
- `dsh-tool-web` integration drives `web_fetch` through `ctx.tools.execute` with the real `spill-local` backend + policy, proving the model-facing text changes only by the deliberate spill notice while the spill file holds the full formatted result.
|
||||
- The `tui-agent` example loads `spill-local` + `spill-policy`, so its keyless Loader/PTY smoke exercises the real load path (the namespace-plugin export shape + `inject`).
|
||||
|
||||
## Consequences
|
||||
|
||||
The default policy only sees final formatted text. It cannot preserve provider-internal content that was already capped or runtime artifacts that were never part of the result. This is acceptable for the first cut because the showcase is final-result spill, not early spill; tool-owned early spill remains deferred work.
|
||||
|
||||
Returning real paths from the local backend keeps v1 simple and matches proven agent-tool behavior, while the seam itself only promises an opaque locator plus retrieval hint so remote backends can return non-file locators.
|
||||
|
||||
The local-backend value proposition depends on the existing `read`/`grep` tools being able to inspect the returned local path, even when the spill directory is outside the session cwd. That holds today because the filesystem policy records observations and write guards but does not confine reads to the workspace. A future workspace-confinement policy must either allow local spill paths explicitly or use a non-file spill backend whose retrieval hint points at a supported reader.
|
||||
|
||||
**Snapshot gap.** No ACP snapshot scenario covers the transcript-visible `web_fetch` spill notice yet. The ACP snapshot harness replays keyless and cannot hit the live web, and a `web_fetch` spill requires a real over-cap HTTP body; a deterministic scenario would need a seeded loopback fetch target the replay tree does not currently wire (the examples do not load `tool-web` at all). The behavior is covered instead by the `dsh-tool-web` integration test against a loopback server. Closing the gap is follow-up work: wire `tool-web` + a seeded fetch target into the ACP example, then record a `web-fetch-spill` scenario.
|
||||
|
||||
The policy can become too large if it starts owning tool-specific semantics. It stays narrow: plain-text final results only. Tool-owned early spill remains future work.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Require each tool to opt in with a retention declaration.** Rejected for v1: the goal is a default behavior similar to Claude Code's generic tool-result persistence. A single `maxInlineBytes` deployment knob is enough to prove the shape.
|
||||
|
||||
**Make `tool-results` a broad tool-result platform.** Rejected: a broad package name invites retention policy, result replacement, preview wording, search, and early spill into one seam. The shared storage part is smaller: save text and return a locator plus retrieval hint.
|
||||
|
||||
**Use `ctx.fs.writeText` or the model-facing `write` tool.** Rejected: workspace filesystem writes carry project-file semantics, write/edit policy, observation state, and user-facing side effects. Spill files are runtime artifacts, not model-authored workspace edits. The existing `read` tool may inspect them later, but creation belongs to the runtime spill seam.
|
||||
|
||||
**Let `web-fetch-local` fetch without caps and rely on spill-policy.** Rejected: spill-policy runs after the final tool result exists and cannot protect network, memory, or decoding resources. Provider resource caps stay mandatory.
|
||||
|
||||
**Merge retention into spill.** Rejected: retention and spill have different responsibilities. `TextRetainer`/`ItemRetainer` decide what preview is kept and what was omitted; spill storage only saves the final text the policy asks it to save.
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md: b934f7fd7087006be4f7eb3659e44e78b8ede367
|
||||
2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md: 3b5b60a95bef0695a446cdd3d45d299550f449f6
|
||||
@@ -0,0 +1,61 @@
|
||||
# Agent Note: After-call compaction pressure and context-overflow recovery
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-10-after-call-compaction-pressure-and-overflow-recovery.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`agent/pre-step` runs before final request routing and before assistant output, tool results, buffered context, and steering exist. Even with the assembled prompt and session prefix, its pressure view is provisional because `agent/request` can still change routing or call configuration and tool schemas are not frozen with those inputs. Adding fields cannot make pre-call state describe a completed call and couples the generic seam to compaction.
|
||||
|
||||
Successful calls are not the only pressure signal. A provider can reject a request for exceeding its context window before it returns usage, and some successful calls omit usage. The system therefore needs replayable post-call pressure plus a narrow failure-recovery path that preserves the provider error whenever compaction cannot prove useful progress.
|
||||
|
||||
## Decision
|
||||
|
||||
### Successful pressure moves to a durable post-step checkpoint
|
||||
|
||||
`agent/pre-step` is narrowed to `(agent, turn, step, signal)`. It remains a generic serial checkpoint before `step/start`, but it carries no compaction-only prompt or prefix fields.
|
||||
|
||||
The loop fires awaited serial `agent/post-step(agent, turn, step, signal)` after assistant output, every dispatched or synthetic tool result, post-tool context, and steering are durable, but before `step/end`. This placement gives pressure policy the complete successful-call state without splitting an assistant tool call from its result. A propagated listener failure is an ordinary turn failure; it never enters model-request recovery. Compact-basic contains its expected operational failures as described below.
|
||||
|
||||
`dsh-compact-basic` reads the exact latest routed model from the durable request header only to establish that a completed route exists, then asks the singleton `ctx.tokenMeter` to measure the canonical logged envelope and current surface. It does not fall back to `AgentOptions.model` for automatic pressure. A headerless session has no completed routed request to assess and produces no work; any durable non-empty model name uses the same estimator. Operational measurement or summarization failures warn and continue from the latest durable surface: full history before any replacement, or the pruned surface if pruning already landed.
|
||||
|
||||
### Request recovery is limited to the final model boundary
|
||||
|
||||
`RequestError`, `RequestErrorDecision`, and the `agent/request-error` waterfall represent failures after the final adapter has been selected. Each returned stream handle owns a private failure set that preserves the original thrown error identity across dispatch, iterator construction, and iteration without leaking nested-call provenance into an outer call. Terminal in-band `error` or `aborted` finishes enter the same path. Prompt assembly, request middleware, request logging, result processing, tools, post-step listeners, and cleanup remain ordinary failures.
|
||||
|
||||
The failed step closes before recovery runs. A retry opens the next numbered step and rebuilds the request from the durable log; consecutive recovery attempts reset only after a successful provider request. Both DeepSeek adapters normalize recognized provider context-limit failures to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
|
||||
If cancellation lands after assistant tool calls are durable but before all calls dispatch, the loop records a synthetic `tool/call` and aborted `tool/result` pair for every undispatched call before following the normal abort path. The surface therefore never retains orphaned durable tool calls merely because cancellation won the race.
|
||||
|
||||
### CompactService exposes intent, not token accounting
|
||||
|
||||
`CompactService.compactIfNeeded(agent, trigger, signal)` accepts `trigger: 'pressure' | 'context-overflow'`. The interface gains no estimation methods or token types; `ctx.tokenMeter` remains the reusable accounting owner.
|
||||
|
||||
For `pressure`, compact-basic resolves the durable provider/model target's adapter-owned capacity and exact-target policy, then applies the resulting threshold and retained-tail budgets to one unified `ctx.tokenMeter.measure()` result. Below pressure it returns without pruning. Once pressure qualifies, optional `ctx.toolResultPrune` rewrites oversized current results and compact-basic remeasures through the same meter; safe pressure skips the model call, while remaining pressure selects and summarizes from the pruned surface. The same singleton meter owns range pricing, provenance, shadowed token counts, and non-shrinking-summary rejection. Common defaults remain threshold ratio `0.8`, retained-history ratio `0.16`, summarization provider/model `''`, `maxTokens: 8192`, `compactionRetries: 1`, and `auto: true`; optional `modelPolicies` entries override them for an exact provider/model pair.
|
||||
|
||||
For canonical overflow, compact-basic requires no capacity metadata and bypasses scalar pressure and the normal retained-token budget. It prunes first, then chooses the maximal tool-balanced head range while leaving the newest indivisible unit and attempts one shrinking summary compaction under the same signal when a range exists. The automatic listener snapshots `session.surface.replaceGeneration` and returns `{ action: 'retry' }` whenever pruning or summarization increases it. This remains true when pruning lands before later summary work throws; cancellation still wins. A backend returning a result without replacement cannot authorize retry, while pruning-only progress can authorize a retry without a `CompactionResult`.
|
||||
|
||||
`maxOverflowRetries` is optional and defaults to `1`; `0` disables overflow recovery without disabling pressure. `auto: false` registers neither automatic listener. Noncanonical errors, exhausted attempts, an already-aborted signal, a missing routed model, no safe range, no generation change, and recovery throws before any replacement all delegate to the next listener. With no later recovery, the loop reports the original provider error object and code. A recovery throw after generation advances authorizes retry from durable progress; cancellation or disposal remains authoritative even if recovery work completes concurrently.
|
||||
|
||||
The default summarizer resolves explicit configuration, then the latest logged route, then agent options. Because direct `llm/stream` middleware may reroute that auxiliary call, `compact/summary.{provider, model}` records the final mutable `GenerateOptions` target observed after dispatch rather than the pre-waterfall candidate.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests cover final-adapter failure provenance and identity, closed-step retry numbering and reset, cancellation and disposal, post-step ordering, routed-envelope pressure, pressure-gated pruning, pruning-only relief, pruned-input summarization, balanced overflow reduction, durable prune progress before later failure, generation proof, caps, delegation, and auxiliary-call routing. Real-loop tests cover thrown and in-band overflow through pruning or summary compaction to a reconstructed retry request.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep provisional pre-step pressure and add more arguments** — rejected because later routing and request mutation remain outside any earlier snapshot, while generic lifecycle becomes coupled to one plugin.
|
||||
- **Retry the same numbered step** — rejected because recovery appends durable events after the failed boundary. A new step preserves balanced nesting and reconstructability.
|
||||
- **Retry whenever `compactIfNeeded` returns a result** — rejected because a custom backend can report success without changing model-visible state. `replaceGeneration` is the authoritative proof.
|
||||
- **Let compact-basic parse provider wording** — rejected because classification belongs at adapters and must cover both thrown and in-band delivery.
|
||||
- **Fall back to `AgentOptions.model` when no durable route exists** — rejected because automatic policy must describe a completed logged request. Headerless pressure and recovery delegate unchanged.
|
||||
|
||||
## Consequences
|
||||
|
||||
Post-step pressure describes the completed routed request, including durable tool results and request-only prefix fields. Optional model-free pruning removes predictable tool-output bulk before summary selection and can independently create retry-worthy progress. Canonical overflow supplies the backstop when no successful usage anchor exists. Recovery is bounded, cancellation-owned, and monotonic: it retries only after a visible surface generation change.
|
||||
|
||||
The cost is one additional serial checkpoint on successful steps and adapter-maintained overflow classification. Provider wording and heuristic character density remain maintenance risks. Surface compaction still cannot repair an envelope that alone exceeds the window, split an indivisible non-tool node, or repair a tool unit whose non-prunable remainder remains oversized. The optional pruner can repair an otherwise indivisible tool pair when removable text-bearing tool-result content is the bulk.
|
||||
|
||||
This Agent Note supersedes only the pre-step automatic-trigger portion of the [compaction capability-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md). The service split, standalone token meter, balanced range contract, log-recorded lock, summary replacement, and sole `summarize()` subclass hook remain unchanged.
|
||||
@@ -0,0 +1,61 @@
|
||||
# Agent Note:调用后压缩压力与上下文溢出恢复
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`agent/pre-step` 运行在最终请求路由之前,也早于 assistant 输出、工具结果、缓冲上下文与 steering 的产生。即使它接收已装配提示词与会话前缀,压力视图仍是临时的,因为 `agent/request` 还可以改变路由或调用配置,工具 schema 也没有与这些输入一同冻结。增加字段无法让调用前状态描述已完成调用,还会把通用 seam 与压缩耦合。
|
||||
|
||||
成功调用也不是唯一的压力信号。提供方可能在返回 usage 之前就因上下文窗口超限拒绝请求,一些成功调用也不提供 usage。因此,系统需要可回放的调用后压力,以及一条狭窄的失败恢复路径;当压缩无法证明取得有效进展时,必须保留原始提供方错误。
|
||||
|
||||
## 决策
|
||||
|
||||
### 成功压力移动到持久 post-step 检查点
|
||||
|
||||
`agent/pre-step` 收窄为 `(agent, turn, step, signal)`。它仍是 `step/start` 之前的通用串行检查点,但不再携带压缩专用的提示词或前缀字段。
|
||||
|
||||
循环在 assistant 输出、所有已分发或合成的工具结果、工具后上下文与 steering 都持久化之后、`step/end` 之前,触发等待式串行 `agent/post-step(agent, turn, step, signal)`。该位置让压力策略看到完整的成功调用状态,同时不会拆开 assistant 工具调用与其结果。向外传播的监听器失败属于普通 turn 失败,绝不会进入模型请求恢复;compact-basic 会按下文所述在内部处理其预期的操作性失败。
|
||||
|
||||
`dsh-compact-basic` 从持久请求头读取精确的最新实际路由模型,只用它确认已经存在完整路由,随后让单例 `ctx.tokenMeter` 计量规范日志信封与当前表层。自动压力不会回退到 `AgentOptions.model`。没有请求头的会话尚无已完成路由请求可供判断,因此不执行工作;任意持久记录的非空模型名都使用同一个估算器。操作性的计量或摘要失败会发出警告,并从最新持久表层继续:任何替换发生前使用完整历史;若剪枝已经落盘,则使用已剪枝表层。
|
||||
|
||||
### 请求恢复只覆盖最终模型边界
|
||||
|
||||
`RequestError`、`RequestErrorDecision` 与 `agent/request-error` waterfall 表示最终适配器已经选定之后的失败。每个返回的流句柄都绑定一个私有失败集合;该集合在分发、异步迭代器构造与迭代过程中保留原始抛出错误的身份,同时防止把嵌套调用的错误来源误归到外层调用。终止性的带内 `error` 或 `aborted` finish 进入同一路径。提示词装配、请求中间件、请求日志、结果处理、工具、post-step 监听器与清理仍属于普通失败。
|
||||
|
||||
恢复运行前,失败 step 已经关闭。重试会打开下一个编号 step,并从持久日志重建请求;连续恢复尝试计数只在提供方请求成功后重置。两个 DeepSeek 适配器都把识别出的提供方上下文限制错误规范化为 `CONTEXT_WINDOW_EXCEEDED`。
|
||||
|
||||
如果取消发生在 assistant 工具调用已经持久化之后、所有调用完成分发之前,循环会为每个尚未分发的调用记录一对合成的 `tool/call` 与 aborted `tool/result`,随后进入正常中止路径。因此,表层不会仅因取消赢得竞态而留下孤立的持久工具调用。
|
||||
|
||||
### CompactService 暴露意图,而不拥有 token 核算
|
||||
|
||||
`CompactService.compactIfNeeded(agent, trigger, signal)` 接收 `trigger: 'pressure' | 'context-overflow'`。接口不增加估算方法或 token 类型;`ctx.tokenMeter` 继续作为可复用的核算所有者。
|
||||
|
||||
对于 `pressure`,compact-basic 先解析持久提供方/模型目标的适配器所属容量与精确目标策略,再把得到的阈值与保留尾部预算应用到一次统一的 `ctx.tokenMeter.measure()` 结果。低于压力时直接返回,不执行剪枝。压力达到条件后,可选的 `ctx.toolResultPrune` 会改写当前表层中过大的工具结果,compact-basic 再通过同一个 meter 重新计量;若压力恢复安全则跳过模型调用,否则从已剪枝表层选择范围并生成摘要。范围定价、来源、被遮蔽 token 数与非缩小摘要拒绝也由同一个单例 meter 完成。通用默认值保持为阈值比例 `0.8`、保留历史比例 `0.16`、摘要提供方/模型 `''`、`maxTokens: 8192`、`compactionRetries: 1` 与 `auto: true`;可选 `modelPolicies` 项可以按精确提供方/模型组合覆盖这些值。
|
||||
|
||||
对于规范化溢出,compact-basic 不要求容量元数据,并绕过标量压力与普通保留 token 预算。它先执行剪枝,再在保留最新不可分割单元的同时选择最大的工具配对平衡头部范围;存在范围时,才在同一 signal 下尝试一次缩小摘要压缩。自动监听器先记录 `session.surface.replaceGeneration`,剪枝或摘要让 generation 增加时就返回 `{ action: 'retry' }`。即使剪枝先落盘而后续摘要工作抛错,这条规则仍然成立;取消依然优先。后端若只返回结果但没有替换表层,不能授权重试;只有剪枝取得进展时,即使没有 `CompactionResult` 也可以授权重试。
|
||||
|
||||
`maxOverflowRetries` 可选且默认为 `1`;`0` 只禁用溢出恢复,不会禁用压力检查。`auto: false` 不注册任何自动监听器。非规范化错误、尝试耗尽、已经中止的 signal、缺失路由模型、没有安全范围、generation 未变化,以及在任何替换之前恢复抛错,都会委托给下一个监听器。若没有后续恢复,循环报告原始提供方错误对象与代码。generation 增加后的恢复抛错会基于持久进展授权重试;即使恢复工作并发完成,取消或销毁仍具有最终优先级。
|
||||
|
||||
默认摘要器依次解析显式配置、最近记录的路由与 agent options。因为直接 `llm/stream` 中间件可以重新路由该辅助调用,`compact/summary.{provider, model}` 记录分发后最终可变的 `GenerateOptions` 目标,而不是 waterfall 之前的候选值。
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试覆盖最终适配器失败的来源与身份、已关闭 step 的重试编号与重置、取消与销毁、post-step 顺序、已路由信封压力、压力门控剪枝、剪枝独立解除压力、从已剪枝输入生成摘要、平衡溢出缩减、后续失败前已落盘的剪枝进展、generation 证明、上限、委托与辅助调用路由。真实循环测试覆盖抛出式和带内溢出,并验证剪枝或摘要压缩后的重试请求从替换表层重建。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **保留临时 pre-step 压力并增加更多参数**——不予采纳,因为后续路由与请求变换仍在更早快照之外,同时通用生命周期会耦合到单个插件。
|
||||
- **重试相同编号的 step**——不予采纳,因为恢复会在失败边界之后追加持久事件。新 step 保持边界配对与可重建性。
|
||||
- **只要 `compactIfNeeded` 返回结果就重试**——不予采纳,因为自定义后端可能报告成功却没有改变模型可见状态。`replaceGeneration` 才是权威证明。
|
||||
- **让 compact-basic 解析提供方措辞**——不予采纳,因为分类属于适配器,而且必须同时覆盖抛出式与带内交付。
|
||||
- **没有持久路由时回退到 `AgentOptions.model`**——不予采纳,因为自动策略必须描述已完成且已记录的请求。没有请求头的压力检查与恢复会原样委托。
|
||||
|
||||
## 后果
|
||||
|
||||
Post-step 压力描述已完成的路由请求,包括持久工具结果与仅请求前缀字段。可选的无模型剪枝会在选择摘要前移除可预测的工具输出体积,也能独立产生足以重试的进展。当成功 usage 锚点不存在时,规范化溢出提供兜底路径。恢复有明确上限、以取消为准,并保持单调:只有模型可见的表层 generation 变化后才重试。
|
||||
|
||||
代价是成功 step 增加一个串行检查点,并需要适配器持续维护溢出分类。提供方措辞与启发式字符密度仍是维护风险。表层压缩依然无法修复仅信封本身就超出窗口的情况,也不能拆分不可分割的非工具节点,或修复非可剪枝剩余部分仍然过大的工具单元。若可移除的文本工具结果是主要体积,可选剪枝器仍可修复原本不可分割的工具配对。
|
||||
|
||||
本 Agent Note 只取代[压缩能力接缝 Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 中的 pre-step 自动触发部分。服务拆分、独立 token meter、平衡范围契约、日志记录锁、摘要替换与唯一 `summarize()` 子类 hook 均保持不变。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.md: 0d4686a5a233785ca4832ef068a118b484a872fe
|
||||
2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: dcc9213c6b3a088b8b8bce2a442c5232ed5b7d0b
|
||||
@@ -0,0 +1,85 @@
|
||||
# Agent Note: Single-file executable SDK runtime distribution (single-exe)
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
DeepSeek Harness needs a dedicated SDK distribution form for the Python library — no Node installation, runs directly on the target platform: a single-file executable (hereafter "the exe") that exposes a stdio JSON-RPC serving surface (`HarnessSdkServer`, the Python SDK's peer), where the plugins and configuration actually booted are decided entirely by a `cordis.yml` supplied from outside the exe.
|
||||
|
||||
- The JSONRPC protocol for talking to the Python SDK is already validated
|
||||
- A standardized way for cordis.yml to load every plugin (ESModule) is needed
|
||||
- The distribution must carry the Node runtime, and support a locally linked source debugging mode
|
||||
|
||||
## Decision
|
||||
|
||||
### Packaging route: @yao-pkg/pkg's `--sea` mode
|
||||
|
||||
The exe is packaged with the **`--sea` (enhanced SEA) mode** of [@yao-pkg/pkg](https://github.com/yao-pkg/pkg) (the actively maintained fork after vercel/pkg was archived). Relative to Node's native SEA, pkg adds a `/snapshot` VFS and runtime module hooks on top, hands the ESM entry to Node's default ESM loader unchanged, and depends on no ESM→CJS transpilation.
|
||||
> Measured (macos-arm64, node24 target, pkg 6.21.0): bare-specifier ESM dynamic import inside the VFS (including top-level await), CJS interop, `node:sqlite`, fail-loud on package names outside the set, and on-disk ESM import outside the VFS all pass; `import.meta.url` comes through unchanged as `file:///snapshot/...`.
|
||||
|
||||
`--sea` requires target ≥ node22; the exe uniformly targets node24. One pkg invocation packages exactly one target; multi-platform builds invoke it once per platform.
|
||||
|
||||
Terminology reminder: pkg's `/snapshot` VFS has nothing to do with this repo's testing-system "snapshot" (ACP replay expected outputs, `$DSH_SNAPSHOT`); this document says "VFS" for the former.
|
||||
|
||||
### The serving surface is a plugin: the two packages ui/jsonrpc + examples/jsonrpc-demo
|
||||
|
||||
The deterministic protocol implementation (`server.ts` / `transport.ts`) lands as two packages on the existing `ui/acp` + `examples/acp-demo` pattern — the serving surface is itself a plugin:
|
||||
|
||||
- [`packages/ui/jsonrpc`](../../../../packages/ui/jsonrpc/README.md) (`@deepseek-ai/dsh-jsonrpc`): the pure protocol plugin; on apply it mounts `HarnessSdkServer` plus a line-delimited JSON-RPC transport on the process stdio, with disposal through `ctx.effect()`. Whether to serve is decided by `cordis.yml`; a yml that does not mount it is a legitimate process that does not serve. Protocol-level exit belongs to the plugin (after answering the `shutdown` request it disposes its own fiber, then `exit(0)`; an HMR-style unload only stops the service without exiting the process).
|
||||
- [`packages/examples/jsonrpc-demo`](../../../../packages/examples/jsonrpc-demo/README.md) (`@deepseek-ai/dsh-jsonrpc-demo`): a thin app bin — `installFailLoud` + `loadEnv` + config discovery + `boot()` from [`dsh-app-boot`](../../../../packages/ui/app-boot/src/index.ts), done once boot completes; the server is brought up by the `dsh-jsonrpc` entry in the yml. Its only dependency is app-boot. Process-level exit belongs to the bin (stdin EOF/SIGTERM → dispose then 0, SIGINT → 130).
|
||||
|
||||
Config discovery has two channels and fails loudly when both are missing: the `DSH_CORDIS_CONFIG` environment variable first (the SDK client convention), then an argv positional argument; no default path and no built-in fallback whatsoever — "the plugins actually booted are decided by an external cordis.yml" is a hard semantic.
|
||||
|
||||
### Plugin resolution: the VFS holds a real package tree, the closure manifest IS the deploy root
|
||||
|
||||
Inside the exe's VFS sits a **real package tree in build-artifact form** (each package's `lib/` plus a real `node_modules`); the Loader resolves plugin names through standard dynamic `import()`: bare specifiers resolve upward along `node_modules` from the Loader's position inside the VFS, and land inside the VFS naturally. The closed set needs no allowlist code — the set is whatever the VFS has installed, and importing a name outside the set fails.
|
||||
|
||||
The deploy root is [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json) (`dsh-jsonrpc-agent-pkg`, a pnpm workspace member and a zero-code pure dependency manifest) — the unified source of truth for "which plugins the exe ships" and "what the Python runtime distributes". Adding a plugin to the exe = adding one dependency line to the manifest and repackaging. [`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) traverses every workspace package covered by that manifest and requires every non-optional workspace peer at the runtime root, reporting the complete referencing-package → missing-peer chain; CI static, pre-push, and the single-exe build run it before packaging. Deploy also packs by each package's `files`, so the shared chunks tsdown splits out must be covered by `files`.
|
||||
|
||||
### Build pipeline and artifacts
|
||||
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → inject the pkg configuration (`bin` points at `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js` inside the closure, `assets` is a full glob — dynamic import is invisible to pkg's static analysis, so everything must be packed in explicitly) → one `pkg --sea` per target → the executables `dsh-jsonrpc-agent-pkg-<platform>-<arch>` land in `dist-exe/` and are copied back into the runtime directory. CI treats them as intermediate test inputs and retains their platform wheels. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted yields a zero-symlink file tree (most stable for the pkg VFS, physically guaranteeing a single cordis instance); disabling automatic peer installation keeps unpublished package names from triggering registry resolution; link-workspace-packages points the closure at workspace/vendor sources.
|
||||
|
||||
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml), triggered explicitly only — `workflow_dispatch`, or the `build-exe` label on a pull request; native builds on the three platforms linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64, with `~/.pkg-cache` cached; macOS ad-hoc signing is handled by pkg. Each leg drives a mock SSE model through the SDK with the default config and a custom `cordis.yml`, drives the exe directly over NDJSON JSON-RPC, verifies the JSONL and final response, and installs release-shaped wheels into a clean venv without `runtime_bin`; Linux additionally inspects GLIBC requirements and runs in a manylinux 2.28 container. A full three-target run retains four artifacts, each containing one release file: the platform-independent SDK wheel and three native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts only `python-vX.Y.Z` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and three native runtime wheels, then a single serialized job checks and publishes all four to the project PyPI registry. Windows is a non-goal.
|
||||
|
||||
### Python SDK distribution: two carriers, exe for production, node for development
|
||||
|
||||
The Python SDK lives at [`python/`](../../../../python/README.md): `python/sdk` (the client) + `python/sdk-runtime` (the runtime carrier package). The runtime package's data directory holds three kinds of content: the checked-in default `runtime/cordis.yml`, the build-injected platform exe, and the build-injected `runtime/node/` closure tree. `resolve_bundled_launch_args()` automatic resolution **finds the exe only**; the node carrier is enabled only by an explicit `DSH_RUNTIME_MODE=node` (running `runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js`, requiring a system node ≥22.19), positioned as the development-verification channel for members of this repo, and does not enter wheel distributions.
|
||||
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative stable `X.Y.Z` from the repository root `package.json` and stages both packages at that version, with the SDK depending exactly on `deepseek-harness-runtime-bin==X.Y.Z`. An optional `python-vX.Y.Z` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. The SDK is a `py3-none-any` wheel; the wheel-only runtime package contains exactly one exe and uses one of `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, or `py3-none-macosx_11_0_arm64`. Its Hatch hook rejects sdists, universal tags, mixed executable payloads, and unsupported platforms.
|
||||
|
||||
The exe's "must be explicitly configured" hard semantic is unchanged; the zero-config experience is restored by the wrapper: when the caller gave no `cordis`, named no explicit runtime, and the environment has no `DSH_CORDIS_CONFIG`, the client explicitly injects the checked-in default `cordis.yml` (agent-core + preloaded llm-deepseek + JSONL persistence + bash-local + the `dsh-jsonrpc` serving entry, with `!!js` environment-variable fallbacks) via `DSH_CORDIS_CONFIG`.
|
||||
|
||||
### Naming lineage
|
||||
|
||||
`@deepseek-ai/dsh-jsonrpc-demo` (the package) → `dsh-jsonrpc-agent` (the bin) → `dsh-jsonrpc-agent-pkg` (the closure manifest; no scope prefix, deliberately sidestepping the constraints' package-shape rules for `@deepseek-ai/dsh-*`) → `dsh-jsonrpc-agent-pkg-<platform>-<arch>` (the exe artifacts). The wire `serverInfo.name` stays `deepseek-harness-sdk-runtime` (a protocol-stable value); the Python dist names are `deepseek-harness` / `deepseek-harness-runtime-bin`.
|
||||
|
||||
## Disposition of worker-style plugins
|
||||
|
||||
`dsh-workflow-workerthread` and `dsh-code-runtime-worker` are supported inside the exe. Their built hosts convert the sibling `lib/worker.cjs` URL with `fileURLToPath()` and pass the resulting filesystem string to `Worker`, which is the form pkg's Worker hook resolves inside the VFS. The worker entries are CommonJS because that hook compiles VFS worker files as CommonJS. The workflow engine keeps its data-URL bootstrap for unbuilt source execution; only its built sibling entry uses the filesystem string. The custom-config executable smoke loads both backends, invokes a real `run_code` call and a zero-agent `workflow` call, and requires each worker to return `42` from inside pkg's VFS.
|
||||
|
||||
## Testing
|
||||
|
||||
The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build completes a turn against a mock endpoint through the default SDK path, a custom config, and the direct binary protocol, with final text and JSONL checked. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. The same build leg runs a committed executable-specific snapshot through the Python SDK: a keyless scripted model mounts a Cordis plugin that registers a tool, invokes that tool from `run_code`, runs a direct spawn subagent and a workflow that starts a second spawn child, then unmounts the plugin. The comparison normalizes the SDK result and notification stream plus the parent and two child JSONL logs. This harness stays separate from ACP's `pnpm run test:snapshot` because the protocols and build artifacts differ. The platform wheel is then installed in a clean venv and run without `runtime_bin`.
|
||||
|
||||
Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disposes immediately, so a short-lived pipe aborts an in-flight turn — pipe-driven runs must keep stdin open until the turn ends.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Bare Node native SEA.** The injected main script must be a single CJS file, and the blob carries no filesystem and no module resolution, so a dynamic import of a bare specifier has nothing to resolve against; the only option is compiling plugins statically into the main script and registering them by hand — bypassing standard module resolution and hardcoding the plugin set, contrary to "configuration decides everything". The final route is in fact "the official SEA foundation + pkg's VFS/module-hook layer"; what was rejected is the bare use, not SEA itself.
|
||||
|
||||
**pkg standard mode.** Killed by the PoC, not a trade-off: it turns ESM into CJS + V8 bytecode via esbuild, the runtime vm compilation wires up no dynamic-import callback, every `import()` throws `ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`, and `--options experimental-require-module` has no effect; it also depends on community-patched Node binaries (no macos-arm64 prebuilt; compiling from source on the spot takes about 10 minutes). Zero viability for this repo's architecture.
|
||||
|
||||
**Pre-bundling each package ESM→CJS into the VFS.** The compromise that keeps real resolution semantics and only downgrades the module format; `--sea` passed measurement outright, so this layer of build complexity never needed introducing.
|
||||
|
||||
**jsonrpc-agent carrying the full closure dependencies.** The app bin would declare 53+ dependencies it never imports — a "packaging manifest" masquerading as real dependency relationships — and would force constraints to open two exceptions for it, cordis-in-dependencies and a files wildcard. With the closure manifest landing on the python-side manifest package, constraints needs no exception at all and the bin keeps the normal package shape isomorphic to acp-agent.
|
||||
|
||||
**An open plugin set (loading user plugins from disk).** This round ships a closed set; the PoC incidentally confirmed that on-disk ESM import outside the VFS works (through the `ctx.baseUrl` relative-path channel). It is listed as a future evolution, which must separately solve sharing the cordis instance inside the exe with external plugins.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Bought**: zero-dependency single-file distribution on target platforms; plugin semantics strictly identical to running from source (the same real package tree, no transpilation, no registry); the serving surface, the plugin set, and the configuration all converge on two sources of truth — `cordis.yml` plus one dependency manifest; the exe and node carriers share one tree and one semantics, so development verification never waits for packaging; official Node binaries remove the patched-binary supply-chain concern.
|
||||
|
||||
**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial).
|
||||
@@ -0,0 +1,85 @@
|
||||
# Agent Note: 单文件可执行的 SDK 运行时分发(single-exe)
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-10-single-file-executable-sdk-runtime-distribution.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
DeepSeek Harness 需要为 Python 库专门提供一种无需安装 Node、可直接在目标平台运行的 SDK 分发形态:一个单文件可执行程序(下称 exe),通过 stdio 提供 JSON-RPC 对外服务接口(`HarnessSdkServer`,Python SDK 的对端),且实际启动的插件与配置完全由 exe 外部输入的 `cordis.yml` 决定。
|
||||
|
||||
- 与 Python SDK 通信的 JSON-RPC 协议已经过验证
|
||||
- 需要提供通过标准化 `cordis.yml` 加载所有插件(ES 模块)的能力
|
||||
- 分发物要自带 Node 运行时,并支持本地源码链接的调试模式
|
||||
|
||||
## 决策
|
||||
|
||||
### 打包路线:@yao-pkg/pkg 的 `--sea` 模式
|
||||
|
||||
exe 使用 [@yao-pkg/pkg](https://github.com/yao-pkg/pkg)(vercel/pkg 归档后的活跃维护 fork)的 **`--sea`(enhanced SEA)模式**打包。相比 Node 原生 SEA,pkg 在其上增加 `/snapshot` 虚拟文件系统(VFS)与运行时模块钩子,将 ESM 入口原样交给 Node 默认的 ESM loader,不依赖任何 ESM→CJS 转译。
|
||||
> 实测(macos-arm64、node24 构建目标、pkg 6.21.0):VFS 内裸包名 ESM 动态 `import()`(含顶层 `await`)、CJS 互操作、`node:sqlite`、集合外包名明确报错、VFS 外磁盘 ESM `import()` 全部通过,`import.meta.url` 原样为 `file:///snapshot/...`。
|
||||
|
||||
`--sea` 要求构建目标 ≥ node22,exe 统一以 node24 为构建目标;每次 pkg 调用只打包一个构建目标,多平台各调用一次。
|
||||
|
||||
术语提醒:pkg 的 `/snapshot` VFS 与本仓库测试体系的“快照”(ACP 回放预期输出、`$DSH_SNAPSHOT`)无关,本文用“VFS”指前者。
|
||||
|
||||
### 对外服务接口也是插件:ui/jsonrpc + examples/jsonrpc-demo 两包
|
||||
|
||||
确定性协议实现(`server.ts` / `transport.ts`)按 `ui/acp` + `examples/acp-demo` 的既有模式落为两包——对外服务接口本身也是插件:
|
||||
|
||||
- [`packages/ui/jsonrpc`](../../../../packages/ui/jsonrpc/README.md)(`@deepseek-ai/dsh-jsonrpc`):纯协议插件;执行 `apply` 时,在进程 stdio 上挂载 `HarnessSdkServer` 与按行传输的 JSON-RPC 层,资源释放走 `ctx.effect()`。是否提供服务由 `cordis.yml` 决定;未挂载该插件的配置会启动一个不提供此服务的合法进程。协议级退出归插件所有(应答 `shutdown` 请求后 dispose 自身 fiber,再调用 `exit(0)`;HMR 式卸载只停止服务,不退出进程)。
|
||||
- [`packages/examples/jsonrpc-demo`](../../../../packages/examples/jsonrpc-demo/README.md)(`@deepseek-ai/dsh-jsonrpc-demo`):轻量应用入口——`installFailLoud` + `loadEnv` + 配置发现 + [`dsh-app-boot`](../../../../packages/ui/app-boot/src/index.ts) 的 `boot()`;`boot()` 完成后入口即完成,服务器由 `cordis.yml` 中的 `dsh-jsonrpc` 条目启动。它只依赖 `app-boot`。进程级退出归 `bin` 所有(stdin EOF/SIGTERM → dispose 后返回 0,SIGINT → 130)。
|
||||
|
||||
配置发现有两个通道,均缺失时立即报错:优先使用 `DSH_CORDIS_CONFIG` 环境变量(SDK 客户端约定),其次使用 argv 位置参数;没有默认路径或内置回退——“实际启动的插件由外部 `cordis.yml` 决定”是硬语义。
|
||||
|
||||
### 插件解析:VFS 装载真实包树,闭包清单就是部署根目录
|
||||
|
||||
exe 的 VFS 内是**构建产物形态的真实包树**(各包的 `lib/` + 真实 `node_modules`)。loader 通过标准动态 `import()` 解析插件名:裸包名从 VFS 内 loader 所在位置沿 `node_modules` 向上解析,自然落在 VFS 内。封闭集不需要白名单代码——VFS 中安装了什么,集合中就有什么;`import()` 集合外的名称会失败。
|
||||
|
||||
部署根目录是 [`python/sdk-runtime/package.json`](../../../../python/sdk-runtime/package.json)(`dsh-jsonrpc-agent-pkg`,pnpm 工作区成员、零代码纯依赖清单),也是“exe 安装哪些插件”与“Python 运行时分发什么”的统一事实源。向 exe 添加插件,就是在清单中增加一行依赖后重新打包。[`scripts/verify-runtime-closure.ts`](../../../../scripts/verify-runtime-closure.ts) 遍历该清单覆盖的全部工作区包,要求每个非可选的工作区对等依赖(peer dependency)都显式列在运行时根目录,并报告“引用包 → 缺失对等依赖”的完整链路;CI 静态检查、pre-push 与 single-exe 构建都会在打包前运行该门禁。部署还会依据各包的 `files` 字段打包,因此 tsdown 拆出的共享分片必须被 `files` 覆盖。
|
||||
|
||||
### 构建管线与产物
|
||||
|
||||
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts):运行时闭包校验 → `pnpm run build` →(清空后)`pnpm --filter dsh-jsonrpc-agent-pkg deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 注入 pkg 配置(`bin` 指向闭包内的 `node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js`;`assets` 使用全量 glob,因为动态 `import()` 对 pkg 静态分析不可见,必须显式打入全部内容)→ 每个构建目标调用一次 `pkg --sea` → 可执行文件 `dsh-jsonrpc-agent-pkg-<platform>-<arch>` 写入 `dist-exe/`,并拷回运行时目录。CI 将这些文件作为测试中间输入,只保留对应平台的 wheel 包。四个部署标志都有实测依据:未启用 `inject-workspace-packages` 时必须使用 `--legacy`;`hoisted` 产出无符号链接的文件树(对 pkg VFS 最稳定,并从物理上保证只有一个 Cordis 实例);关闭对等依赖自动安装可避免未发布包名触发注册表解析;`link-workspace-packages` 让闭包指向工作区/vendor 源码。
|
||||
|
||||
CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml),且只允许显式触发:手动派发 `workflow_dispatch`,或给 PR 添加 `build-exe` 标签。linux-x64、linux-arm64(`ubuntu-24.04-arm`)和 macos-arm64 三个平台分别进行原生构建,并缓存 `~/.pkg-cache`;macOS 的 ad-hoc 签名由 pkg 处理。每个平台都使用模拟 SSE 模型,分别通过默认配置和自定义 `cordis.yml` 驱动 SDK,再通过 NDJSON JSON-RPC 直接驱动 exe,校验 JSONL 与最终响应;最后把发布形态的 wheel 包安装到干净的 venv 中,并在不传 `runtime_bin` 的情况下运行。Linux 还会检查 GLIBC 依赖,并在 manylinux 2.28 容器中运行。完整构建三个目标时保留 4 个产物,每个产物只含一个发布文件:平台无关的 SDK wheel 包与 3 个原生运行时 wheel 包;手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-vX.Y.Z` 标签流水线,构建一个 SDK wheel 包和 3 个原生运行时 wheel 包,再由单个串行任务校验并将这 4 个文件发布到项目的 PyPI 注册表。Windows 不在目标范围内。
|
||||
|
||||
### Python SDK 分发:双载体,exe 用于生产,`node` 用于开发
|
||||
|
||||
Python SDK 位于 [`python/`](../../../../python/README.md):`python/sdk` 是客户端,`python/sdk-runtime` 是运行时载体包。运行时包的数据目录包含三类内容:检入的默认 `runtime/cordis.yml`、构建注入的平台 exe,以及构建注入的 `runtime/node/` 闭包树。`resolve_bundled_launch_args()` 的自动解析**只查找 exe**;`node` 载体仅在显式设置 `DSH_RUNTIME_MODE=node` 时启用(运行 `runtime/node/node_modules/@deepseek-ai/dsh-jsonrpc-demo/lib/bin.js`,需要系统 Node ≥22.19),定位为本仓库成员的开发验证通道,不随 wheel 包分发。
|
||||
|
||||
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) 从仓库根目录的 `package.json` 读取权威的稳定版本 `X.Y.Z`,以该版本暂存两个包,并让 SDK 精确依赖 `deepseek-harness-runtime-bin==X.Y.Z`。可选的 `python-vX.Y.Z` 发布标签只是一项一致性断言,与仓库版本不同时会被拒绝;源码 `pyproject.toml` 中的开发占位版本从不决定发布版本。SDK 是 `py3-none-any` wheel 包;只提供 wheel 包的运行时包恰好包含一个 exe,标签为 `py3-none-manylinux_2_28_x86_64`、`py3-none-manylinux_2_28_aarch64` 或 `py3-none-macosx_11_0_arm64`。其 Hatch 钩子拒绝 sdist、通用标签、混合可执行载荷以及不支持的平台。
|
||||
|
||||
exe“必须显式配置”的硬语义不变;零配置体验由包装层恢复:调用方没有提供 `cordis`、没有显式指定运行时,且环境中没有 `DSH_CORDIS_CONFIG` 时,客户端将检入的默认 `cordis.yml`(`agent-core` + 预载的 `llm-deepseek` + JSONL 持久化 + `bash-local` + `dsh-jsonrpc` 对外服务条目,并通过 `!!js` 使用环境变量兜底)显式注入 `DSH_CORDIS_CONFIG`。
|
||||
|
||||
### 命名血统
|
||||
|
||||
`@deepseek-ai/dsh-jsonrpc-demo`(包)→ `dsh-jsonrpc-agent`(`bin`)→ `dsh-jsonrpc-agent-pkg`(闭包清单;没有作用域前缀,刻意避开 `constraints` 对 `@deepseek-ai/dsh-*` 的包形状规则)→ `dsh-jsonrpc-agent-pkg-<platform>-<arch>`(exe 产物)。协议字段 `serverInfo.name` 保持为 `deepseek-harness-sdk-runtime`(协议稳定值);Python 分发名为 `deepseek-harness` / `deepseek-harness-runtime-bin`。
|
||||
|
||||
## 工作线程插件
|
||||
|
||||
exe 内支持 `dsh-workflow-workerthread` 与 `dsh-code-runtime-worker`。两个后端构建后的宿主都通过 `fileURLToPath()` 转换相邻 `lib/worker.cjs` 的 URL,再将所得文件系统字符串传给 `Worker`;pkg 的 Worker 钩子可以用这种形式解析 VFS 内文件。该钩子会把 VFS 内的工作线程文件作为 CommonJS 编译,所以工作线程入口采用 CommonJS。工作流引擎在未构建的源码执行中仍保留 `data:` URL 引导程序,只有构建后的相邻入口使用文件系统字符串。自定义配置的可执行文件冒烟测试会加载两个后端,实际调用 `run_code` 与不启动 agent 的 `workflow`,并要求两个工作线程都从 pkg 的 VFS 内返回 `42`。
|
||||
|
||||
## 测试
|
||||
|
||||
验证面分三层。机制层:`--sea` 链路的实测结论内嵌在“决策”各节(VFS 内 ESM 动态 `import()`、单一 Cordis 实例、明确报错的配置链路、`node:sqlite`、macOS ad-hoc 签名可运行)。SDK 层:完整的无密钥 pytest 套件以假运行时对端覆盖客户端协议、子进程清理、绝对 `cwd` 传递、双载体启动与载体解析;根 CI 在 Python 3.10 上运行全部用例。端到端层:每个平台构建都通过默认 SDK 路径、自定义配置和直接二进制协议,对模拟端点完成一个轮次,并校验最终文本与 JSONL。自定义配置还会通过打包进 VFS 的真实工作线程文件执行 `run_code` 和不启动 agent 的 `workflow`。同一构建任务还会经 Python SDK 运行一组检入的 exe 专用快照:无密钥脚本化模型挂载一个会注册工具的 Cordis 插件,从 `run_code` 调用该工具,运行一个由 spawn 提供方直接启动的 subagent(子 agent)和一个会通过 spawn 启动第二个子 agent 的工作流,随后卸载该插件。比较时会规范化 SDK 结果与通知流,以及父会话和两个子会话的 JSONL 日志。该 harness 与 ACP 的 `pnpm run test:snapshot` 保持独立,因为二者的协议和构建产物不同。随后把平台 wheel 包安装进干净的 venv,并在不传 `runtime_bin` 的情况下运行。
|
||||
|
||||
手工驱动注意:`bin` 将 stdin EOF 视为“客户端已离开”并立即 dispose,短命管道会中止进行中的轮次——管道驱动必须保持 stdin 打开,直到轮次结束。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**裸用 Node 原生 SEA。** 注入的主脚本必须是 CJS 单文件,blob 内没有文件系统与模块解析,因此动态 `import()` 无法解析裸包名;只能把插件静态编译进主脚本并手工注册。这会绕过标准模块解析并硬编码插件集合,与“配置决定一切”相悖。最终路线实际是“官方 SEA 基础 + pkg 的 VFS/模块钩子层”;否决的是裸用方式,而不是 SEA 本身。
|
||||
|
||||
**pkg 标准模式。** PoC 证明该模式不可行,而非权衡后放弃:它通过 esbuild 将 ESM 转为 CJS + V8 字节码,但运行时 VM 编译没有接入动态 `import()` 回调,任何 `import()` 都会抛出 `ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`,`--options experimental-require-module` 也无效;此外,它依赖社区补丁版 Node 二进制(macos-arm64 没有预编译版本,现场从源码编译约需 10 分钟)。该模式不适用于本仓库架构。
|
||||
|
||||
**每包 ESM→CJS 预打包进 VFS。** 保持真实解析语义、只降级模块格式的折中;`--sea` 直接通过实测,这层构建复杂度无需引入。
|
||||
|
||||
**让 jsonrpc-agent 承担完整闭包依赖。** 应用入口将声明 53 个以上自身并不 `import()` 的依赖,使“打包清单”伪装成真实依赖关系,还会迫使 `constraints` 为其增加 `cordis-in-dependencies` 与 `files` 通配符两个例外。将闭包清单放在 Python 侧的清单包后,`constraints` 不需要任何例外,`bin` 也能保持与 acp-agent 同构的正常包形状。
|
||||
|
||||
**开放插件集(从磁盘加载用户插件)。** 本期采用封闭集;PoC 同时证实,可以通过 `ctx.baseUrl` 相对路径通道从 VFS 外的磁盘 `import()` ESM。该能力列为后续演进,届时还需解决外部插件与 exe 内 Cordis 实例的共享问题。
|
||||
|
||||
## 后果
|
||||
|
||||
**买到的**:目标平台零依赖的单文件分发;插件语义与源码运行严格一致(同一棵真实包树,无转译、无注册表);对外服务接口、插件集与配置全部收敛到 `cordis.yml` 和一份依赖清单这两个事实源;exe 与 `node` 双载体使用同一棵树和相同语义,开发验证无需等待打包;官方 Node 二进制消除了补丁版二进制的供应链顾虑。
|
||||
|
||||
**付出的**:产物约 174MB,且源码原样进入 blob(没有字节码混淆;闭源分发诉求需要另行评估);pkg 的 VFS/模块钩子层仍由社区维护(构建脚本钉死 `@yao-pkg/pkg@6.21.0`,升级需要显式改动);`--sea` 每个构建目标调用一次(与 CI 每个平台一个任务相匹配,本地多平台构建串行执行)。
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Agent-scope runtime design and correctness
|
||||
# Agent Note: Agent-scope runtime design and correctness
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -26,9 +26,9 @@ The design can be skimmed as seven choices:
|
||||
| Compose the model-visible prompt and tool surface | One shared tool view plus the authoritative assembly-waterfall result |
|
||||
| Coordinate subagent, worker, and process shutdown | One cancellation signal plus the independent terminal/quiescence facts of that boundary |
|
||||
|
||||
The rest of this RFC expands those choices in dependency order. It first explains the Cordis mechanics, then scope routing, creation and session commit, tools and prompts, subagents and workflows, and finally the checks that make the reasoning executable.
|
||||
The rest of this Agent Note expands those choices in dependency order: Cordis mechanics, scope routing, creation and session commit, tools and prompts, subagents and workflows, then executable checks.
|
||||
|
||||
The [July 8 RFC](2026-07-08-agent-scope-contexts.md) remains the contributor contract. The separate [subagent composition-controls RFC](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) owns `persona`, `toolFilter`, and `maxDepth`; this document discusses only how their setup fits the lifecycle.
|
||||
The [July 8 Agent Note](2026-07-08-agent-scope-contexts.md) remains the contributor contract. The separate [subagent composition-controls Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md) owns `persona`, `toolFilter`, and `maxDepth`; this document discusses only how their setup fits the lifecycle.
|
||||
|
||||
## Cordis model: context, fiber, effect, receiver, and waterfall
|
||||
|
||||
@@ -206,7 +206,7 @@ Tool presentation and execution share one private resolver. Prompt assembly rema
|
||||
|
||||
The private resolver applies the current presentation mode, live global restrictions, exact local overlay, and local shadowing. Schemas, lookup, execution, Code Mode SDK generation, and restriction validation all use that resolver or its pre-restriction global-name view.
|
||||
|
||||
The [subagent composition-controls RFC](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md#tool-filtering-is-one-live-global-view-rule) owns the user-visible allow/deny semantics. The implementation requirement is agreement: a filtered-away global cannot remain executable through a different lookup path, and a locally shadowed definition is the same definition presented and executed.
|
||||
The [subagent composition-controls Agent Note](../feature/2026-07-12-subagent-persona-tool-filter-and-depth.md#tool-filtering-is-one-live-global-view-rule) owns the user-visible allow/deny semantics. The implementation requirement is agreement: a filtered-away global cannot remain executable through a different lookup path, and a locally shadowed definition is the same definition presented and executed.
|
||||
|
||||
`ToolRestriction` accepts readonly allow/deny names and compiles them into internal sets. Multiple restrictions intersect. Public `visible()` and `knownNames()` methods are unnecessary because only the registry needs the intermediate views.
|
||||
|
||||
@@ -322,19 +322,19 @@ TypeScript cannot govern JavaScript casts, direct Cordis dispatch, process messa
|
||||
|
||||
### Runtime invariants cover cross-service facts
|
||||
|
||||
The invariants plugin verifies that every declared scoped event uses a marked carrier and that event families exposing a subject use the matching key. Session trace validation stages before append commit and advances after the same event commits.
|
||||
The `dsh-scope/invariant` companion verifies, when selected, that every declared scoped event uses a marked carrier and that event families exposing a subject use the matching key. The separate `dsh-session/invariant` contribution stages trace validation before append commit and advances after the same event commits; both register through `ctx.invariants`.
|
||||
|
||||
The plugin does not police trusted setup by scanning registries or reject prompt assembly objects fabricated through casts. Those checks would turn composition contracts into speculative runtime machinery without protecting a real external boundary.
|
||||
|
||||
### Generated artifacts keep public contracts aligned
|
||||
|
||||
The event catalog, service catalog, producer/consumer matrix, configuration catalog, module graph, tool catalog, and type-equivalence blocks are generated or freshness-gated from source. `verify-scoped-dispatch` keeps the declared scoped-event set aligned with runtime invariant coverage.
|
||||
The event catalog, service catalog, producer/consumer matrix, configuration catalog, module graph, tool catalog, type-equivalence blocks, and scoped-event resolver map are generated or freshness-gated from source. The [TypeScript semantic-gates Agent Note](../process/2026-07-14-typescript-program-backed-semantic-gates.md) owns Program construction, semantic event discovery, and resolver-generation rules.
|
||||
|
||||
Behavioral tests pin scoped routing and disposal, final-entry collision cleanup, publication rollback, ordered quiescence, durable pre/post-commit behavior, live tool filtering across presentation and execution, cooperative prompt assembly, structured-output commit in native and Code Mode, async subagent startup and signal cancellation, worker terminal arbitration, ACP settlement, and process teardown.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
The [July 8 RFC](2026-07-08-agent-scope-contexts.md#alternatives-considered) owns alternatives to the public flat-scope contract. The alternatives here concern implementation shape.
|
||||
The [July 8 Agent Note](2026-07-08-agent-scope-contexts.md#alternatives-considered) owns alternatives to the public flat-scope contract. The alternatives here concern implementation shape.
|
||||
|
||||
### Use a transparent proxy as the scope carrier
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-14-provider-routed-llm-adapters.md: 98205d18d07752e0cdba86d7cba80368d45fd816
|
||||
2026-07-14-provider-routed-llm-adapters.zh.md: c35225a86baf4c2d09732b5940abbc8046d365fb
|
||||
@@ -0,0 +1,91 @@
|
||||
# Agent Note: Provider-routed LLM adapters and a generic pi-ai backend
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-14-provider-routed-llm-adapters.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`dsh-llm` registered adapters by exact model name. A plugin supplied a model list at Cordis startup, `LlmService` stored one adapter per listed string, and `GenerateOptions.model` selected the adapter and the provider model at once. This worked while both shipping adapters targeted the same two DeepSeek models, but it conflated two independent decisions: which upstream provider owns a request, and which model that provider should run.
|
||||
|
||||
The conflation prevents a provider gateway from serving an open-ended model catalog. OpenRouter, for example, is one provider with many model ids, while a private OpenAI-compatible endpoint may add models without changing the Harness plugin tree. Every newly selected model currently needs to have been registered during plugin startup. The same model id can also exist at multiple providers, so model-only registration cannot state which provider the caller intended.
|
||||
|
||||
`dsh-llm-pi-ai` exposed none of pi-ai's provider abstraction. It constructed an inline DeepSeek `openai-completions` model, applied DeepSeek-specific payload patches, and stamped every replayed assistant message as DeepSeek. pi-ai itself has a provider/model catalog, selects APIs such as `openai-responses`, `anthropic-messages`, and `google-generative-ai`, and preserves provider-specific response ids and reasoning/tool signatures for later turns. The Harness conversion dropped that provenance, so simply replacing the inline model with a catalog lookup would have made same-model replay and cross-provider handoff incomplete.
|
||||
|
||||
The adapter configuration also assumes one DeepSeek API key and endpoint. A generic backend needs independent credentials and endpoint overrides per provider while leaving AWS, Google ADC, OAuth, and other ambient authentication mechanisms to pi-ai.
|
||||
|
||||
## Decision
|
||||
|
||||
### Provider is the adapter registration key
|
||||
|
||||
`GenerateOptions` and `LlmCallConfig` carry `provider: string` beside `model: string`; `AgentOptions` carries the corresponding optional creation field. A loop request is valid only after both values are non-empty, and both values are part of the logged request header. `agent/request` may return a replacement pair on any step, so a session can switch providers and models without changing the Cordis plugin lifecycle.
|
||||
|
||||
`LlmService` registers and resolves adapters by provider. `registerAdapter(providers, adapter)` checks the entire provider list before mutating the registry, rejects a duplicate with `DUPLICATE_ADAPTER`, and disposes the whole registration as one effect. Model ids are not registration keys; the selected adapter still validates or forwards them. The later [LLM catalog and ACP selection Agent Note](2026-07-15-llm-model-catalog-and-acp-selection.md) added advisory `listProviders()` / `listModels()` discovery without turning model membership into request validation.
|
||||
|
||||
A provider has exactly one adapter owner in a Cordis context. `dsh-llm-deepseek` registers `deepseek`; `dsh-llm-pi-ai` may also register `deepseek`, but loading both owners is a configuration error rather than an ordering rule or fallback. A deployment that wants the hand-rolled DeepSeek implementation excludes `deepseek` from the pi-ai profiles. A deployment that wants pi-ai's DeepSeek implementation does not mount `dsh-llm-deepseek`.
|
||||
|
||||
`dsh-llm-deepseek` removes its model registration list and accepts any model string routed through provider `deepseek`. Its request serialization, `/chat/completions` endpoint, thinking options, SSE parsing, and error behavior remain unchanged; `options.model` is still sent verbatim.
|
||||
|
||||
### Explicit pi-ai provider profiles
|
||||
|
||||
`dsh-llm-pi-ai` takes one non-empty list of provider profiles. Provider names must be unique within the list and present in pi-ai's `getProviders()` result. Each profile contains the provider name plus optional `apiKey`, `baseURL`, headers, reasoning level and budgets, cache retention, transport, SDK timeouts, and a Harness stream-idle timeout. Provider retry fields are deliberately absent: the adapter forces pi-ai's `maxRetries` to zero so one `stream()` call makes one visible provider attempt, while `dsh-llm-retry` owns bounded agent-level recovery. Credentials are never global: an explicit key applies only to its profile, while an absent key lets pi-ai resolve its standard environment variable, OAuth token, AWS credential chain, Google ADC, or other provider-native ambient authentication. An explicitly empty key is invalid configuration rather than an environment fallback.
|
||||
|
||||
The plugin registers all configured provider names against one `PiAiAdapter` in one all-or-nothing call. A request uses its provider to select the matching profile and finds its model in `getModels(provider)` to obtain the catalog descriptor. An unknown provider fails at plugin load; an unknown model fails before network I/O with `UNKNOWN_MODEL`. The catalog object is never mutated. When a profile supplies `baseURL`, the adapter clones the selected descriptor and overrides only `baseUrl`, so a private endpoint can retain pi-ai's API, capabilities, compatibility flags, context limits, and reasoning map. The private endpoint must implement the selected provider's protocol, and the model id must still exist in the installed pi-ai catalog.
|
||||
|
||||
The adapter calls pi-ai's `streamSimple()` so each catalog model chooses its registered API implementation, including OpenAI Responses instead of Chat Completions where the descriptor says `openai-responses`. Harness temperature, maximum tokens, signal, session id, and the profile's common stream options flow through directly. Profile headers merge with the mandatory Harness attribution headers, with Harness attribution winning its reserved names. The adapter no longer maintains DeepSeek-specific payload rewrites or a provider-protocol matrix.
|
||||
|
||||
pi-ai's common stream options do not expose stop sequences. `dsh-llm-pi-ai` rejects a defined Harness `stop` option with `UNSUPPORTED_OPTION` rather than silently ignoring it or growing a second provider-specific payload implementation. `dsh-llm-deepseek` continues to support `stop` through its native request serializer.
|
||||
|
||||
### Durable assistant provenance and replay state
|
||||
|
||||
Assistant messages carry provider-neutral provenance containing the request's `provider` and `model`, plus an optional JSON-serializable adapter replay state. A successful `assistant/message` session event records this provenance and `deriveMessages()` returns it with the assistant message. User, system, context, and tool-result messages carry no assistant provenance. The provider/model fields are authoritative loop data; an adapter owns only its opaque replay-state payload.
|
||||
|
||||
A terminal successful `finish` chunk may carry replay state, and `BlockAssembler` retains it alongside usage and finish reason. The loop attaches it to the assistant provenance only when the post-`agent/step-result` content is structurally equal to the assembled provider output. A listener that rewrites content keeps the provider/model provenance but loses the now-stale replay state. Error and aborted responses do not produce a normal assistant message and therefore do not enter future model history.
|
||||
|
||||
The pi-ai replay state is a versioned, minimal projection of its successful `AssistantMessage`: source API/provider/model, response id/model, stop reason, and index-aligned text, thinking, and tool-call signatures. It does not duplicate text or tool arguments already carried by Harness content blocks, and it omits diagnostics, timestamps, usage, and errors. On a later request, `LlmService` gives replay state to the target adapter only when the historical provider and target provider are currently owned by the same adapter instance. That adapter combines the logged Harness content with replay state when it can restore the historical response, and owns any required cross-model or cross-provider conversion. An adapter receiving replay state with an unknown version or mismatched block shape fails explicitly; a different adapter receives only provider-neutral content and provenance.
|
||||
|
||||
This state is model-visible replay input and therefore follows the existing [reconstructable-request rule](2026-07-05-reconstructable-requests.md): it is present in both the terminal `finish` chunk and the assembled `assistant/message` provenance that drives derivation. Resume and fork preserve it verbatim. Compaction that shadows the assistant message also removes its replay state from the active surface; the summary is ordinary provider-neutral content.
|
||||
|
||||
### Propagate the target through every request producer
|
||||
|
||||
Every model-selection surface carries provider and model together: declarative agents, ACP and stdio app config, the JSON-RPC initialize request, subagent overrides and inheritance, workflow child overrides, and direct compaction summarization. Subagents inherit both fields from their parent before applying request overrides. The system-prompt variable set gains `provider` beside `model`.
|
||||
|
||||
Compaction configuration gains `summarizationProvider` beside `summarizationModel`. Both are empty to inherit, or both are non-empty to select an explicit target; a half-configured pair fails load. Inheritance uses the last logged request target when one exists and falls back to the agent's creation options. `compact/summary` records both fields with the existing model-call envelope.
|
||||
|
||||
The JSON-RPC runtime receives provider and model explicitly. Its convenience fallback mounts `dsh-llm-deepseek` only for provider `deepseek` when that provider has no registered owner; other missing providers fail without guessing an adapter.
|
||||
|
||||
The on-disk session format remains the pre-release pinned version `0`, with no compatibility promise. Seed/load validation rejects request headers lacking provider and assistant messages lacking required provenance instead of accepting an old shape that can no longer reconstruct the request.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep model names as registry keys and add wildcard adapters.** A wildcard introduces fallback ordering between exact registrations and catch-all plugins, makes duplicate ownership dependent on listener order, and still cannot distinguish the same model id at two providers without another convention.
|
||||
|
||||
**Encode provider and model into one string.** Values such as OpenRouter's `openai/gpt-*` already contain provider-like prefixes and slashes. A delimiter convention would leak routing syntax into every model selector and require escaping rules; two explicit fields are unambiguous and independently loggable.
|
||||
|
||||
**Add `backend + provider + model`.** A backend key would allow `dsh-llm-deepseek` and pi-ai's DeepSeek implementation to coexist and switch per request. The accepted deployment rule is instead one adapter owner per provider: implementations of the same upstream are alternatives selected by plugin composition. A third routing dimension would burden every request and configuration for a capability with no current consumer.
|
||||
|
||||
**Let `dsh-llm-pi-ai` automatically register every pi-ai provider.** This would claim ambient credentials and provider names the deployment never intended to expose, and would conflict with native adapters such as `dsh-llm-deepseek`. Explicit profiles make capability and credential scope reviewable.
|
||||
|
||||
**Mount one pi-ai plugin instance per provider.** Separate instances isolate config but repeat plugin declarations and cannot make profile registration atomic. One adapter already receives provider on every request, so a validated profile map is the smaller lifecycle surface.
|
||||
|
||||
**Accept arbitrary inline pi-ai model descriptors.** This would support catalog-external private model ids, but it exposes pi-ai's model and compatibility schema as Harness configuration and makes the adapter responsible for validating protocol-specific combinations. The first version supports custom endpoints by overriding `baseURL` on catalog models; custom descriptors require a separate decision after a real catalog-external deployment is identified.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Provider names are deployment-wide route ownership keys: two providers may use the same model string, but mounting two adapters for one provider fails at load instead of creating fallback order.
|
||||
- Model selection no longer changes the Cordis plugin graph. Catalog-backed adapters can accept any installed catalog model selected after startup, while the native DeepSeek adapter forwards arbitrary DeepSeek model ids.
|
||||
- A custom `baseURL` preserves the selected catalog model's protocol and capabilities; it does not make catalog-external model ids valid. Private endpoints must implement that catalog entry's protocol.
|
||||
- pi-ai credentials, transport knobs, SDK timeouts, and the five-minute-default `streamIdleTimeoutMs` watchdog are scoped per provider profile. Hidden provider retries are disabled; bounded retries belong to the separately composed agent recovery policy.
|
||||
- `dsh-llm-pi-ai` rejects stop sequences because pi-ai's common stream API cannot express them; the native DeepSeek adapter retains its stop support.
|
||||
- Replay state is portable only within the adapter instance that owns both the historical and target providers. Cross-provider and cross-model restoration is an adapter responsibility, and another adapter receives provider-neutral history without the opaque state.
|
||||
- Current pre-release session JSONL requires provider/model request headers and assistant provenance. Older shapes remain version `0` but are rejected rather than migrated.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit coverage exercises registry conflicts, request reconstruction, session validation, profile resolution, single-attempt option forwarding, native API selection including OpenAI Responses, conversion, replay validation, error mapping, caller cancellation, idle-timeout transport termination, content rewrites, and same-instance versus different-instance replay dispatch.
|
||||
- Keyless loop/session tests and ACP snapshots exercise durable provider/model metadata, resume and fork propagation, workflow/subagent overrides, and unchanged user-visible transcripts; the key-gated DeepSeek e2e retains real provider streaming and tool follow-up coverage.
|
||||
- Public JSDoc, package READMEs, architecture and core-data-structure docs, generated catalogs, examples, session fixtures, and Python SDK pairs use provider/model targets consistently and are checked by the repository documentation and type-equivalence gates.
|
||||
|
||||
## Risks
|
||||
|
||||
This is a repo-wide pre-release API break: model-only request construction, adapter registration, app protocols, fixtures, and persisted version-0 event shapes all change together, with no compatibility aliases. The provider exclusivity rule deliberately prevents two implementations of the same upstream from coexisting in one context. A pi-ai dependency update can change the accepted provider/model catalog, so the lockfile and adapter e2e matrix define the tested set. Custom `baseURL` endpoints inherit the chosen catalog model's protocol assumptions and cannot repair an incompatible proxy. Catalog-external model descriptors and multimodal content remain unsupported. pi-ai replay state may contain opaque encrypted reasoning signatures; it is persisted because the provider requires it for continuity, but it is never rendered or logged outside the existing session record.
|
||||
@@ -0,0 +1,91 @@
|
||||
# Agent Note: 基于提供方路由的 LLM 适配器与通用 pi-ai 后端
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-14-provider-routed-llm-adapters.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`dsh-llm` 按精确模型名称注册适配器。插件在 Cordis 启动时提供模型列表,`LlmService` 为列表中的每个字符串保存一个适配器,`GenerateOptions.model` 同时选择适配器与提供方模型。两个正式适配器都只面向相同的两个 DeepSeek 模型时,这种方式可以工作,但它混淆了两个独立决策:由哪个上游提供方承接请求,以及该提供方应运行哪个模型。
|
||||
|
||||
这种混淆使提供方网关无法提供开放的模型目录。例如,OpenRouter 是一个包含大量模型 ID 的提供方,私有 OpenAI 兼容端点也可能在不修改 Harness 插件树的情况下增加模型。目前,每个新选择的模型都必须在插件启动期间完成注册。同一个模型 ID 还可能存在于多个提供方中,因此仅按模型注册无法表达调用方预期使用的提供方。
|
||||
|
||||
`dsh-llm-pi-ai` 没有暴露 pi-ai 的提供方抽象。它以内联方式构造 DeepSeek `openai-completions` 模型,应用 DeepSeek 专用的 payload 补丁,并将每条回放的助手消息标记为 DeepSeek。pi-ai 自身提供提供方/模型目录,能够选择 `openai-responses`、`anthropic-messages`、`google-generative-ai` 等 API,并保留提供方专用的响应 ID,以及后续轮次所需的推理和工具签名。Harness 转换丢弃了这些来源信息,因此仅将内联模型替换为目录查询,会导致同模型回放与跨提供方移交不完整。
|
||||
|
||||
适配器配置同样假定只存在一个 DeepSeek API 密钥和端点。通用后端需要为各提供方分别配置凭据和端点覆盖,同时继续由 pi-ai 处理 AWS、Google ADC、OAuth 等环境认证机制。
|
||||
|
||||
## 决策
|
||||
|
||||
### 提供方作为适配器注册键
|
||||
|
||||
`GenerateOptions` 与 `LlmCallConfig` 在 `model: string` 之外携带 `provider: string`,`AgentOptions` 则携带对应的可选创建字段。只有两个值都非空时,agent loop(智能体循环)请求才有效;两个值也都会写入请求头日志。`agent/request` 可以在任意步骤返回替换后的字段组合,因此会话可以切换提供方与模型,无需改变 Cordis 插件生命周期。
|
||||
|
||||
`LlmService` 按提供方注册和解析适配器。`registerAdapter(providers, adapter)` 在修改注册表前检查整个提供方列表,遇到重复项时返回 `DUPLICATE_ADAPTER`,并将整组注册作为一个 effect 释放。模型 ID 不作为注册键;仍由选中的适配器负责验证或转发。后续的 [LLM 目录与 ACP 模型选择 Agent Note](2026-07-15-llm-model-catalog-and-acp-selection.md) 增加了建议性的 `listProviders()` / `listModels()` 发现接口,但不会把目录成员关系变成请求校验规则。
|
||||
|
||||
在一个 Cordis 上下文中,一个提供方只能有一个适配器所有者。`dsh-llm-deepseek` 注册 `deepseek`;`dsh-llm-pi-ai` 也可以注册 `deepseek`,但同时加载两个所有者属于配置错误,不采用顺序规则或回退行为。若部署选择手写的 DeepSeek 实现,需从 pi-ai 配置中排除 `deepseek`;若部署选择 pi-ai 的 DeepSeek 实现,则不挂载 `dsh-llm-deepseek`。
|
||||
|
||||
`dsh-llm-deepseek` 移除模型注册列表,接受通过 `deepseek` 提供方路由的任意模型字符串。其请求序列化、`/chat/completions` 端点、thinking 选项、SSE(Server-Sent Events)解析和错误行为保持不变;`options.model` 仍会原样发送。
|
||||
|
||||
### 显式 pi-ai 提供方配置
|
||||
|
||||
`dsh-llm-pi-ai` 接受一个非空的提供方配置列表。列表内的提供方名称必须唯一,并且存在于 pi-ai 的 `getProviders()` 结果中。每项配置包含提供方名称,以及可选的 `apiKey`、`baseURL`、headers、推理级别和预算、缓存保留设置、传输方式、SDK 超时和 Harness 流空闲超时。配置中有意不提供重试字段:适配器强制将 pi-ai 的 `maxRetries` 设为零,使一次 `stream()` 调用只发起一次可见的提供方请求;有界的 agent 层恢复由 `dsh-llm-retry` 负责。凭据不设全局值:显式密钥仅对所属配置生效;未提供密钥时,pi-ai 使用标准环境变量、OAuth token、AWS 凭据链、Google ADC 或其他提供方原生环境认证。显式空密钥属于无效配置,不会回退到环境认证。
|
||||
|
||||
插件通过一次全有或全无调用,将所有已配置的提供方名称注册到同一个 `PiAiAdapter`。请求按 provider 选择对应配置,并在 `getModels(provider)` 中查找模型以取得目录描述符。未知提供方会在插件加载时失败;未知模型会在网络 I/O 前以 `UNKNOWN_MODEL` 失败。适配器不会修改目录对象。当配置提供 `baseURL` 时,适配器复制选中的描述符,仅覆盖 `baseUrl`,使私有端点保留 pi-ai 的 API、能力、兼容标志、上下文限制与推理映射。私有端点必须实现所选提供方的协议,模型 ID 也仍须存在于已安装的 pi-ai 目录中。
|
||||
|
||||
适配器调用 pi-ai 的 `streamSimple()`,因此每个目录模型会选择其注册的 API 实现;描述符为 `openai-responses` 时使用 OpenAI Responses,而非 Chat Completions。Harness 的 temperature、最大 token 数、signal、session ID,以及提供方配置中的通用流选项均直接传递。配置 headers 与 Harness 强制归因 headers 合并;发生保留名称冲突时,以 Harness 归因为准。适配器不再维护 DeepSeek 专用 payload 重写或提供方协议矩阵。
|
||||
|
||||
pi-ai 的通用流选项不支持停止序列。若 Harness `stop` 选项已定义,`dsh-llm-pi-ai` 会以 `UNSUPPORTED_OPTION` 拒绝请求,不会静默忽略,也不会增加第二套提供方专用 payload 实现。`dsh-llm-deepseek` 继续通过原生请求序列化器支持 `stop`。
|
||||
|
||||
### 持久化助手来源信息与回放状态
|
||||
|
||||
助手消息携带提供方无关的来源信息,其中包含请求的 `provider` 和 `model`,以及可选的 JSON 可序列化适配器回放状态。成功的 `assistant/message` 会话事件记录这些来源信息,`deriveMessages()` 返回助手消息时也会包含这些信息。用户、system、context 与工具结果消息不携带助手来源信息。provider/model 字段是 agent loop 的权威数据;适配器仅拥有其不透明回放状态 payload。
|
||||
|
||||
成功的终止 `finish` 分片可以携带回放状态,`BlockAssembler` 会将其与 token 用量和结束原因一起保留。只有当 `agent/step-result` 处理后的内容与提供方组装输出在结构上相等时,agent loop 才会把回放状态附加到助手来源信息。监听器重写内容后,provider/model 来源信息仍会保留,但已经陈旧的回放状态会被移除。错误或中止响应不会生成正常助手消息,因此不会进入后续模型历史。
|
||||
|
||||
pi-ai 回放状态是其成功 `AssistantMessage` 的带版本最小投影,包含源 API/provider/model、响应 ID/model、停止原因,以及按索引对齐的文本、thinking 和工具调用签名。它不会重复 Harness 内容块中已有的文本或工具参数,也不包含诊断信息、时间戳、用量或错误。后续请求中,只有历史提供方和目标提供方当前归同一个适配器实例所有时,`LlmService` 才会把回放状态交给目标适配器。适配器在能够恢复历史响应时,将 Harness 记录的内容与回放状态组合,并负责所需的跨模型或跨提供方转换。适配器收到未知版本或块形状不匹配的回放状态时会显式失败;其他适配器只能收到提供方无关的内容与来源信息。
|
||||
|
||||
该状态属于模型可见的回放输入,因此遵循现有的[请求可重建规则](2026-07-05-reconstructable-requests.md):它同时存在于终止 `finish` 分片和驱动派生的已组装 `assistant/message` 来源信息中。恢复和 fork 会原样保留该状态。压缩(compaction)遮蔽助手消息时,也会从活动 surface 中移除其回放状态;摘要属于普通的提供方无关内容。
|
||||
|
||||
### 在所有请求生产方中传播目标
|
||||
|
||||
每个模型选择接口都同时携带 provider 与 model:声明式 agent、ACP(Agent Client Protocol)和 stdio 应用配置、JSON-RPC initialize 请求、subagent 覆盖与继承、工作流子 agent 覆盖,以及直接压缩摘要。subagent 先从父 agent 继承两个字段,再应用请求覆盖。系统提示词变量集合在 `model` 之外增加 `provider`。
|
||||
|
||||
压缩配置在 `summarizationModel` 之外增加 `summarizationProvider`。两个值均为空时继承,均非空时选择显式目标;只配置其中一个会导致加载失败。继承优先使用最近一次记录的请求目标,没有时回退到 agent 创建选项。`compact/summary` 使用现有模型调用 envelope 记录两个字段。
|
||||
|
||||
JSON-RPC 运行时显式接收 provider 与 model。仅当 `deepseek` 提供方没有注册所有者时,其便利回退才会挂载 `dsh-llm-deepseek`;其他缺失的提供方会直接失败,不会猜测适配器。
|
||||
|
||||
磁盘会话格式仍使用预发布阶段固定的版本 `0`,且不承诺兼容性。seed/load 验证会拒绝缺少 provider 的请求头,以及缺少必需来源信息的助手消息,不会接受已无法重建请求的旧格式。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**继续以模型名称作为注册表键,并增加通配适配器。** 通配机制会在精确注册与兜底插件之间引入回退顺序,使重复所有权取决于监听器顺序;若不再增加其他约定,仍无法区分不同提供方中相同的模型 ID。
|
||||
|
||||
**将提供方与模型编码到一个字符串中。** OpenRouter 的 `openai/gpt-*` 等值已经包含类似提供方的前缀和斜杠。分隔符约定会把路由语法泄漏到每个模型选择接口,并需要转义规则;两个显式字段更清晰,也可以分别记录日志。
|
||||
|
||||
**增加 `backend + provider + model`。** backend 键可以让 `dsh-llm-deepseek` 与 pi-ai 的 DeepSeek 实现共存,并按请求切换。最终采用的部署规则是一个提供方对应一个适配器所有者:同一上游的不同实现属于由插件组合选定的替代项。第三个路由维度会增加每个请求与配置的负担,却没有当前消费方。
|
||||
|
||||
**让 `dsh-llm-pi-ai` 自动注册所有 pi-ai 提供方。** 这种方式会占用部署无意暴露的环境凭据和提供方名称,并与 `dsh-llm-deepseek` 等原生适配器冲突。显式配置可以审查能力和凭据范围。
|
||||
|
||||
**每个提供方挂载一个 pi-ai 插件实例。** 独立实例可以隔离配置,但会重复插件声明,也无法实现配置注册的原子性。每个请求本就向同一个适配器提供 provider,因此经过验证的配置映射具有更小的生命周期接口。
|
||||
|
||||
**接受任意内联 pi-ai 模型描述符。** 这种方式可支持目录外的私有模型 ID,但会将 pi-ai 的模型与兼容性 schema 暴露为 Harness 配置,并要求适配器验证协议专用组合。当前版本通过覆盖目录模型的 `baseURL` 支持自定义端点;只有实际出现目录外部署需求后,才会另行决策是否支持自定义描述符。
|
||||
|
||||
## 影响
|
||||
|
||||
- 提供方名称是部署范围内的路由所有权键:两个提供方可以使用相同的模型字符串,但为同一个提供方挂载两个适配器会在加载时失败,不会形成回退顺序。
|
||||
- 模型选择不再改变 Cordis 插件图。目录型适配器可以接受启动后选择的任意已安装目录模型,原生 DeepSeek 适配器则会转发任意 DeepSeek 模型 ID。
|
||||
- 自定义 `baseURL` 会保留所选目录模型的协议与能力,但不会让目录外模型 ID 变为有效。私有端点必须实现该目录项对应的协议。
|
||||
- pi-ai 凭据、传输选项、SDK 超时,以及默认五分钟的 `streamIdleTimeoutMs` 空闲超时机制均按提供方配置隔离。系统禁用隐藏的提供方重试;有界重试由单独组合的 agent 恢复策略负责。
|
||||
- pi-ai 的通用流 API 无法表达停止序列,因此 `dsh-llm-pi-ai` 会拒绝停止序列;原生 DeepSeek 适配器仍支持停止序列。
|
||||
- 仅当历史提供方与目标提供方归同一个适配器实例所有时,回放状态才可移植。适配器负责跨提供方和跨模型恢复;其他适配器只接收不含不透明状态的提供方无关历史。
|
||||
- 当前预发布会话 JSONL 要求请求头包含 provider/model,助手消息包含来源信息。旧格式仍使用版本 `0`,但会被拒绝,不执行迁移。
|
||||
|
||||
## 测试
|
||||
|
||||
- 单元测试覆盖注册表冲突、请求重建、会话验证、配置解析、单次请求的选项转发、包括 OpenAI Responses 在内的原生 API 选择、转换、回放验证、错误映射、调用方取消、空闲超时导致的传输终止、内容重写,以及同一实例与不同实例间的回放分发。
|
||||
- 无密钥的 agent loop/会话测试和 ACP 快照覆盖持久化 provider/model 元数据、恢复与 fork 传播、工作流/subagent 覆盖,以及不变的用户可见 transcript(文本记录);密钥门控的 DeepSeek e2e 测试保留真实提供方的流式输出与工具后续调用覆盖率。
|
||||
- 公共 JSDoc、package README、架构与核心数据结构文档、生成目录、示例、会话 fixture(测试前置数据)和 Python SDK 配对文档统一使用 provider/model 目标,并由仓库文档与类型等价门禁校验。
|
||||
|
||||
## 风险
|
||||
|
||||
这是一次覆盖全仓库的预发布 API 破坏性变更:仅模型的请求构造、适配器注册、应用协议、fixture,以及持久化版本 0 事件格式会同时变化,不提供兼容别名。提供方排他规则有意禁止同一上游的两个实现共存于同一上下文。pi-ai 依赖升级可能改变可接受的提供方/模型目录,因此锁文件与适配器 e2e 矩阵定义已验证集合。自定义 `baseURL` 端点会继承所选目录模型的协议假设,无法修复不兼容的代理。目录外模型描述符与多模态内容仍不受支持。pi-ai 回放状态可能包含不透明的加密推理签名;提供方需要该信息维持连续性,因此系统会持久化该状态,但不会在现有会话记录之外渲染或记录它。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-agent-initiator-scope.md: 69648100e76cfc212469854188d664357fec22f1
|
||||
2026-07-15-agent-initiator-scope.zh.md: 835d7a5b2ab6d2d6fce7971de4fd9d6c69e50d77
|
||||
@@ -0,0 +1,65 @@
|
||||
# Agent Note: Initiating Agent scope over AsyncLocalStorage
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-15-agent-initiator-scope.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The harness has two useful but different notions of context. A Cordis `Context` selects services, registration ownership, and lifetime; `agent.ctx` is the flat registration scope owned by one live Agent. Agent and Session identity instead describe the subject of an asynchronous operation. Changing a root `ctx.agent` to mean “whichever Agent is running” would conflate those meanings and fail when one process drives Agents concurrently.
|
||||
|
||||
Deep process-local infrastructure sometimes needs a trusted initiating Agent below explicit loop, tool, and request parameters—for example, a host-aware transport, tracing helper, logger, or gateway client. Requiring every private helper to forward `agent` adds repetition, while a process-global mutable slot is incorrect across `await`. Model-visible arguments are unsuitable because a model must not choose a trusted Session or routing header. The carrier belongs to the Agent service rather than optional model-visible context.
|
||||
|
||||
## Decision
|
||||
|
||||
The mandatory `ctx.agents` service uses Node `AsyncLocalStorage` to carry the initiating Agent. It stores the exact `Agent` directly rather than introducing a one-field frame; a separate private run token records nested boundary lineage only for teardown bookkeeping and carries no identity. The [core-data catalog](../../../../docs/core-data-structures/core.md#initiating-agent) identifies the carried type.
|
||||
|
||||
`currentInitiator()` reads optionally, `requireInitiator()` throws `no initiating agent is active`, and `withInitiator(agent, operation)` preserves the operation's exact synchronous value or Promise. `withoutInitiator(operation)` establishes a clearing boundary for work that must not inherit an Agent. Session remains derived as `agent.session`; turn, step, tool call, `signal`, model, `cwd`, sandbox, and authorization stay with their existing owners.
|
||||
|
||||
`AgentLoop` already injects `ctx.agents` and wraps each concrete driver's complete `runLoop` lifetime in `agents.withInitiator(agent, ...)`. Its package-private loop, turn, step, and tool-call orchestration entries recover the exact Agent from `ctx.agents`, derive `agent.session` once, and let operation-local helpers capture it instead of forwarding the concrete driver or `Session` through shallow interfaces. A leaf helper keeps a narrow `Session` parameter when that is its actual interface rather than accepting a broader `Context` only for an ambient lookup.
|
||||
|
||||
Concurrent drivers receive independent stores. A child driver's continuations carry the child, while the caller resumes in its prior store as soon as `withInitiator()` returns; active-run tracking keeps the returned Promise in the teardown drain until it settles. Creation, persistence load, and unpublished `setup(agentCtx)` remain outside the child's driver boundary: creation initiated by a parent runs under the parent identity, while `agentCtx.agent` explicitly identifies the child.
|
||||
|
||||
Ambient identity does not replace explicit contracts. `ToolExecution.agent`, `AssembleContext.agent`, `GenerateOptions.sessionId`, task ownership, parent/child requests, `ctx.agent`, `agentCtx.agent`, approval and hook subjects, `cwd` selection, cancellation, worker/process messages, persistence records, and wire identity remain explicit. A remote boundary materializes the identity it needs into its typed request because ALS is process-local.
|
||||
|
||||
`AgentRegistry` owns an ordered initiator lifecycle. Teardown first rejects new boundaries; removing `ctx.agents` then drains injected dependents such as AgentLoop, and the registry waits for active returned-Promise boundaries before calling `AsyncLocalStorage.disable()`. If a boundary's inherited async chain starts an owning Cordis fiber's unload, the private run-token lineage releases that nested boundary chain from the drain, which prevents teardown from waiting on itself while unrelated boundaries still drain. `currentInitiator()` and `requireInitiator()` remain usable through a retained in-flight service reference while the ordinary drain runs; after disposal, initiator methods throw `agent initiator scope is disposed`. Root Context disposal may start sibling fiber teardown concurrently, so active-boundary counting remains necessary in addition to Cordis dependency ordering.
|
||||
|
||||
Initiator scope does not own detached work: registry drain tracks only the Promise returned by `withInitiator()` or `withoutInitiator()`. Asynchronous resources created inside a boundary inherit its store until they settle or ALS is disabled, so their owning seam must stop unreturned work explicitly. Agent-owned foreground work returns its lifetime and keeps its cancellation contract. Unrelated timers, queues, and deployment infrastructure start under `withoutInitiator(operation)`; queue, worker, process, and wire boundaries serialize identity rather than expecting ALS propagation.
|
||||
|
||||
A host-aware transport may derive a deployment-owned header such as `X-Harness-Session-Id` from `ctx.agents.requireInitiator().session.id`; the header is absent from model-visible schema and arguments. No production MCP or Web transport adopts such a header in this decision. A test-double transport proves the trusted boundary without assigning host routing policy to an existing provider-neutral seam.
|
||||
|
||||
This decision extends the [Agent registration-scope contract](2026-07-08-agent-scope-contexts.md) and its [runtime design](2026-07-12-agent-scope-runtime-design.md); it does not change their static `agent.ctx` meaning.
|
||||
|
||||
## Verification
|
||||
|
||||
Agent service tests pin optional and required reads, exact synchronous and cross-realm Promise identity, intrinsic Promise settlement observation, overlapping, nested, and cleared boundaries, restoration after throws or rejection, ordinary and reentrant drain ordering, and retained-reference errors. AgentLoop integration pins concurrent and nested drivers, agentless calls, AgentRegistry restart, root teardown, and package-private loop and tool scheduling through the ambient lookup. Composition, module-graph, build, and runtime-closure checks keep `ctx.agents` wired through the default bundle, SDK spine, Python runtime closure, and direct AgentLoop harnesses without another provider.
|
||||
|
||||
A test-double host-aware transport derives `X-Harness-Session-Id` internally and verifies that tool schema and logged arguments contain no identity field. The service deliberately does not drain async work omitted from the Promise returned by the boundary operation; that work remains subject to its owner's explicit stop contract.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Pass Agent through every function.** Public, worker, process, persistence, and wire boundaries continue to do this, but requiring every process-local private helper to carry Agent adds repetitive forwarding without improving trust. ALS is confined to the asynchronous chain inside those explicit boundaries.
|
||||
|
||||
**Make `ctx.agent` dynamic.** `ctx.agent` already means the static Agent associated with an Agent-scoped Cordis context. Changing the root meaning would mix registration and execution scopes and make concurrent behavior surprising.
|
||||
|
||||
**Add a separate `ctx.agentExecution` service.** The carrier has no independent backend, configuration, or identity type: it stores the same `Agent` that `ctx.agents` already owns, and AgentLoop already depends on that service. A second mandatory provider would add package, composition, lifecycle, generated-catalog, and test-harness wiring without separating a real capability.
|
||||
|
||||
**Store a named or complete runtime frame.** A one-field `{ agent }` frame only wraps the value, while Agent, Session, inbox, cancellation, turn, step, tool execution, and persistence already have authoritative owners. Adding more fields would create stale snapshots and another lifecycle; carrying `Agent` directly keeps the boundary named by its methods without duplicating state.
|
||||
|
||||
**Include a step `AbortSignal`, `cwd`, sandbox, or authorization.** Their lifetimes and authority do not match the driver boundary, and their existing seams already pass them explicitly. Adding a control capability requires a separate decision and nested lifecycle contract.
|
||||
|
||||
**Use a process-global `currentAgent`.** Concurrent Agents and subagents overwrite one another across awaited continuations, so a mutable global is correct only under a serialization guarantee the harness does not make.
|
||||
|
||||
**Derive identity from model-visible arguments.** Model or user input cannot be trusted to select Session, tenant, or sandbox routing.
|
||||
|
||||
**Add routing identity to every capability seam.** That spreads hosting concerns through provider-neutral APIs. A host-aware implementation owns its transport header while public boundaries remain explicit.
|
||||
|
||||
## Consequences
|
||||
|
||||
Deep infrastructure gains one trusted process-local initiating Agent without widening existing tool and capability requests. Concurrent and nested drivers isolate automatically, AgentLoop gains no additional mandatory service, and HMR/root disposal reaches quiescence before ALS is disabled.
|
||||
|
||||
The dependency is implicit in function signatures and carries a capability-bearing Agent object. Consumers must restrict it to cross-cutting infrastructure, treat ambient presence as neither liveness nor authorization, and retain explicit cancellation and ownership checks. ALS also has an always-on propagation cost and does not cross worker, process, HTTP, or durable queue boundaries.
|
||||
|
||||
The teardown design deliberately accepts Node's [Stability 1 (Experimental)](https://nodejs.org/api/async_context.html#asynclocalstoragedisable) `AsyncLocalStorage.disable()` dependency. Node requires `disable()` before an ALS instance can be garbage-collected, which matters when HMR replaces AgentRegistry-owned instances; the service state guard prevents a later boundary from re-entering the instance after disposal.
|
||||
|
||||
The scope deliberately carries only the Agent, omitting turn, step, `signal`, `cwd`, sandbox, and authorization. A real consumer that cannot use existing explicit fields must justify any refinement separately; a stale copied field may at most mislabel telemetry, never grant control.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Agent Note: 基于 AsyncLocalStorage 的发起 Agent 作用域
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-15-agent-initiator-scope.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
Harness 中存在两种有用但不同的上下文概念。Cordis `Context` 负责选择服务、注册归属和生命周期;`agent.ctx` 是一个存活 Agent 所拥有的扁平注册作用域。Agent 与会话身份描述的则是异步操作主体。若把根 `ctx.agent` 改成「当前正在运行的 Agent」,就会混淆这两种含义,并在单进程并发驱动多个 Agent 时失效。
|
||||
|
||||
进程内深层基础设施有时需要在显式传递的循环、工具及请求参数之下获取可信的发起 Agent,例如宿主感知传输层、追踪辅助函数、日志器或网关客户端。要求每个私有辅助函数都转发 `agent` 会造成重复,而进程级可变槽会在跨 `await` 时发生并发错误。模型可见参数也不适用,因为模型不得选择可信的会话或路由请求头。该载体归 Agent 服务所有,而非模型可见的可选上下文。
|
||||
|
||||
## 决策
|
||||
|
||||
必需的 `ctx.agents` 服务使用 Node `AsyncLocalStorage` 携带发起 Agent。它直接存储同一个 `Agent`,不引入只有一个字段的帧;另一个私有运行标记只记录嵌套边界的谱系,供 teardown 记账使用,不携带身份。[核心数据目录](../../../../docs/core-data-structures/core.md#initiating-agent)标明了所携带的类型。
|
||||
|
||||
`currentInitiator()` 用于可选读取,`requireInitiator()` 抛出 `no initiating agent is active`,`withInitiator(agent, operation)` 保留操作返回的同步值或 Promise 本身。`withoutInitiator(operation)` 会建立清空边界,供不得继承 Agent 的工作使用。会话仍通过 `agent.session` 推导;轮次、步骤、工具调用、`signal`、模型、`cwd`、沙箱和授权继续由现有归属方管理。
|
||||
|
||||
`AgentLoop` 已经注入 `ctx.agents`,并用 `agents.withInitiator(agent, ...)` 包裹每个具体驱动的完整 `runLoop` 生命周期。循环、轮次、步骤和工具调用的包内私有入口从 `ctx.agents` 恢复同一个 Agent,一次推导 `agent.session`,再由操作内辅助函数捕获该值,避免在浅层接口中转发具体驱动或 `Session`。若 `Session` 本身就是底层辅助函数的实际接口,该函数会保留狭窄的 `Session` 参数,而不会只为隐式查找而接收更宽泛的 `Context`。
|
||||
|
||||
因此,并发驱动使用彼此独立的存储。子驱动的异步延续携带子 Agent;`withInitiator()` 返回后,调用方立即恢复之前的存储,而活动运行计数仍持续跟踪返回的 Promise,直到其结束。创建、持久化加载和尚未发布的 `setup(agentCtx)` 位于子驱动边界之外:由父 Agent 发起的创建使用父身份,而 `agentCtx.agent` 显式标识子 Agent。
|
||||
|
||||
隐式身份不会取代显式契约。`ToolExecution.agent`、`AssembleContext.agent`、`GenerateOptions.sessionId`、任务归属、父子请求、`ctx.agent`、`agentCtx.agent`、审批与 hook 主体、`cwd` 选择、取消、worker 和进程消息、持久化记录及协议身份都保持显式传递。远程边界会把所需身份写入类型化请求,因为 ALS 只在进程内有效。
|
||||
|
||||
`AgentRegistry` 管理一个有序的发起方生命周期。teardown 会先拒绝新边界;移除 `ctx.agents` 后,AgentLoop 等注入方开始排空,注册表随后等待活动的返回 Promise 边界,最后调用 `AsyncLocalStorage.disable()`。如果某个边界继承的异步调用链启动所属 Cordis fiber 的卸载,私有运行标记谱系会从排空范围中释放该嵌套边界链,从而避免 teardown 等待自身完成,同时继续排空无关边界。在普通排空期间,进行中代码可通过保留的服务引用继续调用 `currentInitiator()` 和 `requireInitiator()`;dispose 后,发起方方法会抛出 `agent initiator scope is disposed`。根 Context dispose 可能并发启动同级 fiber 的 teardown,因此除 Cordis 依赖顺序外仍必须统计活动边界。
|
||||
|
||||
发起方作用域不负责管理脱离返回链的工作:注册表排空只跟踪 `withInitiator()` 或 `withoutInitiator()` 返回的 Promise。边界内创建的异步资源会继承其存储,直到自身结束或 ALS 被禁用;所属 seam 必须显式停止未纳入返回 Promise 的工作。Agent 所有前台工作会把完整生命周期纳入返回值,并保留显式取消契约。无关的定时器、队列和部署基础设施在 `withoutInitiator(operation)` 下启动;队列、worker、进程和协议边界必须序列化身份,不能期待 ALS 传播。
|
||||
|
||||
宿主感知的传输层可以从 `ctx.agents.requireInitiator().session.id` 推导由部署方拥有的 `X-Harness-Session-Id` 等请求头;模型可见 schema 和参数中不包含该请求头。本决策不让现有生产 MCP 或 Web 传输层采用此请求头。测试替身传输层用于证明可信边界,而不会把宿主路由策略分配给现有的提供方无关 seam。
|
||||
|
||||
本决策扩展 [Agent 注册作用域契约](2026-07-08-agent-scope-contexts.md)及其[运行时设计](2026-07-12-agent-scope-runtime-design.md),不会改变其中 `agent.ctx` 的静态含义。
|
||||
|
||||
## 验证
|
||||
|
||||
Agent 服务测试锁定可选与必需读取、同步值和跨 realm Promise 的引用身份、内建 Promise 结束状态观察、并发、嵌套及清空边界、同步抛错或 Promise 拒绝后的恢复、普通与重入排空顺序及保留引用的错误。AgentLoop 集成测试锁定并发与嵌套驱动、无 Agent 调用、AgentRegistry 重启、根 Context 销毁,以及包内私有的循环和工具调度通过隐式查找完成。组合、模块图、构建及运行时闭包检查确保默认组合包、SDK 主干、Python 运行时闭包及直接 AgentLoop harness 通过 `ctx.agents` 完成接线,无需其他提供方。
|
||||
|
||||
测试替身形式的宿主感知传输层在内部推导 `X-Harness-Session-Id`,并验证工具 schema 与记录参数都不包含身份字段。服务有意不排空边界操作所返回 Promise 之外的异步工作;这类工作仍由所属方的显式停止契约管理。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**在每个函数中传递 Agent。** 公开、worker、进程、持久化和协议边界继续显式传递,但要求每个进程内私有辅助函数都携带 Agent 只会造成重复转发,不会提高可信度。ALS 仅限于这些显式边界内部的异步调用链。
|
||||
|
||||
**让 `ctx.agent` 变成动态值。** `ctx.agent` 已经表示与 Agent 作用域 Cordis 上下文静态关联的 Agent。改变根上下文的含义会混合注册作用域与执行作用域,并让并发行为变得意外。
|
||||
|
||||
**新增独立的 `ctx.agentExecution` 服务。** 该载体没有独立后端、配置或身份类型:它存储的是 `ctx.agents` 已经管理的同一个 `Agent`,而 AgentLoop 本就依赖该服务。第二个必需提供方会增加包、组合、生命周期、生成目录及测试 harness 接线,却没有拆出真实能力。
|
||||
|
||||
**保存命名帧或完整运行时帧。** 只有一个字段的 `{ agent }` 帧只是包装该值,而 Agent、会话、inbox、取消、轮次、步骤、工具执行和持久化已经有各自的真源。增加更多字段会产生陈旧快照和另一套生命周期;直接携带 `Agent`,由方法名标识边界,无需重复保存状态。
|
||||
|
||||
**包含步骤级 `AbortSignal`、`cwd`、沙箱或授权。** 它们的生命周期及权限范围与驱动边界不一致,而且现有 seam 已经显式传递这些值。新增控制能力需要独立决策和嵌套生命周期契约。
|
||||
|
||||
**使用进程级 `currentAgent`。** 并发 Agent 和 subagent 会在异步延续执行之间相互覆盖,因此可变全局值只在 Harness 不具备的串行保证下才正确。
|
||||
|
||||
**从模型可见参数推导身份。** 不能信任模型或用户输入来选择会话、租户或沙箱路由。
|
||||
|
||||
**给每个能力 seam 增加路由身份。** 这会把宿主关注点扩散到提供方无关 API。宿主感知实现拥有其传输请求头,而公开边界继续显式传递身份。
|
||||
|
||||
## 后果
|
||||
|
||||
深层基础设施可以获得一个可信的进程内发起 Agent,而无需加宽现有工具和能力请求。并发及嵌套驱动会自动隔离,AgentLoop 不增加新的必需服务,HMR 或根 Context dispose 会在禁用 ALS 前完成排空。
|
||||
|
||||
该依赖不会出现在函数签名中,并且携带一个具有控制能力的 Agent 对象。消费方必须将其限制在横切基础设施中,把隐式存在视为既不证明存活、也不授予权限,并保留显式取消和归属检查。ALS 还有常驻传播成本,也无法跨越 worker、进程、HTTP 或持久化队列边界。
|
||||
|
||||
该销毁设计有意依赖 Node 的 [Stability 1(实验性)](https://nodejs.org/api/async_context.html#asynclocalstoragedisable) API `AsyncLocalStorage.disable()`。Node 要求在 ALS 实例可被垃圾回收前调用 `disable()`,这对 HMR 替换 AgentRegistry 所拥有的实例尤为重要;服务状态守卫会阻止 dispose 后通过后续边界重新进入该实例。
|
||||
|
||||
该作用域有意只携带 Agent,省略轮次、步骤、`signal`、`cwd`、沙箱和授权。若真实消费方无法使用现有显式字段,必须另行论证扩展;陈旧字段最多只能误标遥测数据,绝不能授予控制权。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.md: 6cc8afc6c7431fbf3eb29fc358b432db4f72b529
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.zh.md: 1cce7a58d0ec83dc01feaf72ccb61d294a78ddd5
|
||||
@@ -0,0 +1,66 @@
|
||||
# Agent Note: Advisory LLM catalogs and per-session ACP model selection
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-15-llm-model-catalog-and-acp-selection.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Provider-routed adapters let every request choose `provider + model`, but `LlmService` exposed only routing and streaming. A UI could not discover which providers were registered or which models an adapter was prepared to recommend. ACP clients therefore received no `model` session config option, so Zed, JetBrains, and VS Code integrations had no model list even though the request seam already supported runtime switching.
|
||||
|
||||
Model discovery cannot become request validation. The hand-written DeepSeek adapter deliberately forwards arbitrary model ids to a public or private endpoint, while pi-ai has a finite installed catalog that is authoritative for its own request resolution. Treating one shared catalog as a whitelist would remove the private-endpoint behavior that provider routing was designed to preserve.
|
||||
|
||||
ACP selection must also preserve the provider dimension. The same model id may appear under multiple routes, and switching a global adapter or agent template would leak one editor session's choice into every other session. Prompt variables and request routing must change together; a selection that lands during asynchronous prompt assembly cannot make `{{model}}` name one model while the request reaches another.
|
||||
|
||||
## Decision
|
||||
|
||||
### Provider-neutral advisory discovery
|
||||
|
||||
`LlmAdapter` gains `providerInfo(provider)` and asynchronous `listModels(provider)` methods. Their provider-neutral results are `LlmProviderInfo { id, name }` and `LlmModelInfo { provider, id, name, description? }`. The defaults preserve existing adapter behavior by naming a provider after its route and advertising no models.
|
||||
|
||||
`LlmService.listProviders()` returns detached metadata in registration order. `LlmService.listModels(provider)` delegates to the route owner, validates non-empty ids and names, rejects a mismatched provider or duplicate model id with `INVALID_CATALOG`, and returns detached values. Unknown providers still fail with `NO_ADAPTER`. Provider metadata is validated atomically during `registerAdapter()` so a malformed display record cannot leave a partial registration.
|
||||
|
||||
Catalog membership is advisory. It drives selectors and diagnostics but never changes `stream()` routing and never rejects an otherwise valid request. Provider ownership remains exclusive and lifecycle-bound; model ids remain request-time adapter input.
|
||||
|
||||
`dsh-llm-pi-ai` maps the configured provider's installed `getModels(provider)` entries into the neutral catalog. Its existing request-time catalog lookup remains authoritative and still rejects unknown models with `UNKNOWN_MODEL`. `dsh-llm-deepseek` accepts an optional `models` config containing display entries, defaulting to `deepseek-v4-flash` and `deepseek-v4-pro`. An explicit list replaces those defaults and an empty list disables discovery. The entries improve selector UX for known public or private models, while every unlisted model id continues to pass through unchanged.
|
||||
|
||||
### ACP session config option
|
||||
|
||||
The ACP bridge advertises one select with `id: model` and `category: model` in `session/new` and `session/load` when the session has a complete target whose provider is registered. Each opaque option value encodes the full provider/model pair. Models are grouped by provider when multiple non-empty provider groups exist; a single group is flattened for clients that render simple selects better.
|
||||
|
||||
The session's current target is added to the displayed options when its adapter omits it. This preserves custom DeepSeek and private-endpoint models while keeping the adapter catalog advisory. A target with an unregistered provider is not advertised, and a model-less agent remains available to another `agent/request` supplier.
|
||||
|
||||
`session/set_config_option` accepts only values from the current catalog snapshot and updates a target reference owned by that ACP session. No global `LlmService` or `AgentOptions` state changes, so concurrent sessions may select different providers and models. The existing permission select remains independent, and every response returns the complete refreshed option state.
|
||||
|
||||
### Prompt/request consistency and durability
|
||||
|
||||
Agent setup installs scoped `system-prompt/assemble` and `agent/request` listeners. Prompt assembly snapshots the selected pair once per step, overwrites the assembled `provider` and `model` variables after downstream prompt listeners, and the request listener applies that same snapshot after downstream request listeners. A selection during asynchronous assembly therefore starts on the next step rather than splitting prompt text from routing. Other call-config fields remain untouched.
|
||||
|
||||
The request header remains the durable source of truth. When a selected target is actually used, the existing full `request/header` snapshot records it. `session/load` initializes the ACP selection from the folded last request header before falling back to bridge config. A selection that is never used by a request is intentionally in-memory only because it never became model-visible state.
|
||||
|
||||
ACP's experimental `providers/*` capability is not used. That draft surface configures provider base URLs, protocols, and headers, including secrets; it does not enumerate models and would give the UI authority to rewrite deployment-owned adapter configuration.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Return model strings only.** A model-only value loses the provider route and becomes ambiguous as soon as two providers expose the same id.
|
||||
|
||||
**Make catalogs mandatory whitelists.** This conflicts with the hand-written adapter's arbitrary model pass-through and private deployments. The selected adapter already owns authoritative request validation.
|
||||
|
||||
**Store selection in `AgentOptions` or `LlmService`.** Those are creation-wide or deployment-wide objects. Mutating them would couple concurrent ACP sessions and bypass the logged `agent/request` replacement path.
|
||||
|
||||
**Persist a new model-selection session event immediately.** An unused UI selection has not affected a model request. Recording the existing request header when the target is consumed preserves the model-visible-if-and-only-if-logged rule without adding a second source of truth.
|
||||
|
||||
**Use ACP `providers/*`.** That unstable API changes endpoint and authentication configuration rather than selecting a model for one session, and its lifecycle and secret-handling semantics do not match this feature.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Any adapter can expose a dynamic model list without leaking provider-library types into the core seam.
|
||||
- Catalog consumers must treat absence as “not advertised,” never “invalid request.”
|
||||
- pi-ai-backed ACP deployments automatically inherit the installed pi-ai provider catalogs; hand-written DeepSeek deployments list known choices explicitly and retain arbitrary model support.
|
||||
- ACP clients receive a standard stable model config option, with provider-aware values and per-session isolation.
|
||||
- Request headers remain compatible with the provider-routed session shape; no new JSONL event or format version is required.
|
||||
- A catalog read can be asynchronous. ACP reads a detached snapshot before creating or resuming an agent, so discovery failure cannot leave a partially published session.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit coverage validates catalog detachment and malformed metadata, pi-ai and DeepSeek catalog projection, ACP provider grouping, custom-current insertion, invalid values, provider/model request routing, prompt-variable alignment, concurrent-session isolation, model-less fallback, and load restoration from the request header. The existing ACP transport suites verify that the additional config option does not change prompt, cancellation, replay, approval, or tool-rendering behavior.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Agent Note: 建议性 LLM 目录与 ACP 会话级模型选择
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-15-llm-model-catalog-and-acp-selection.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
基于提供方路由的适配器允许每次请求选择 `provider + model`,但 `LlmService` 只暴露路由和流式调用。UI 无法发现已注册的提供方,也无法知道适配器愿意推荐哪些模型。因此,ACP 客户端收不到 `model` 会话配置项;即使请求接缝已经支持运行时切换,Zed、JetBrains 和 VS Code 集成仍没有模型列表。
|
||||
|
||||
模型发现不能变成请求校验。手写 DeepSeek 适配器会把任意模型 ID 原样转发给公开或私有端点,而 pi-ai 的有限安装目录则是其自身请求解析的权威依据。将共享目录视为白名单,会破坏提供方路由需要保留的私有端点能力。
|
||||
|
||||
ACP 选择还必须保留提供方维度。同一个模型 ID 可能存在于多个路由下;切换全局适配器或 agent 模板会让一个编辑器会话的选择泄漏到其他会话。Prompt 变量与请求路由必须同时变化;如果选择发生在异步 prompt 组装期间,不能让 `{{model}}` 表示一个模型、实际请求却到达另一个模型。
|
||||
|
||||
## 决策
|
||||
|
||||
### 提供方中立的建议性发现
|
||||
|
||||
`LlmAdapter` 增加 `providerInfo(provider)` 与异步 `listModels(provider)` 方法。其提供方中立结果分别为 `LlmProviderInfo { id, name }` 和 `LlmModelInfo { provider, id, name, description? }`。默认实现以路由名称作为提供方名称,并且不展示模型,从而保持现有适配器行为。
|
||||
|
||||
`LlmService.listProviders()` 按注册顺序返回分离后的元数据。`LlmService.listModels(provider)` 委托给路由所有者,校验非空 ID 和名称,并在提供方不匹配或模型 ID 重复时以 `INVALID_CATALOG` 失败,最后返回分离后的值。未知提供方仍以 `NO_ADAPTER` 失败。提供方元数据在 `registerAdapter()` 期间进行原子校验,错误展示记录不会留下部分注册。
|
||||
|
||||
目录成员关系仅提供建议。它驱动选择器与诊断,但不会改变 `stream()` 路由,也不会拒绝原本有效的请求。提供方所有权仍然具有排他性并绑定生命周期;模型 ID 仍是请求时传给适配器的输入。
|
||||
|
||||
`dsh-llm-pi-ai` 将已配置提供方的安装目录 `getModels(provider)` 映射为中立目录。其现有请求时目录查询仍是权威依据,未知模型仍以 `UNKNOWN_MODEL` 失败。`dsh-llm-deepseek` 接受可选的 `models` 配置作为展示条目,默认包含 `deepseek-v4-flash` 和 `deepseek-v4-pro`。显式列表会替换这些默认值,空列表则关闭发现。这些条目改善已知公开或私有模型的选择体验,而所有未列出的模型 ID 仍会原样透传。
|
||||
|
||||
### ACP 会话配置项
|
||||
|
||||
当会话具有完整目标且目标提供方已注册时,ACP bridge 会在 `session/new` 与 `session/load` 中展示一个 `id: model`、`category: model` 的选择项。每个不透明选项值都编码完整的提供方/模型字段组合。存在多个非空提供方分组时按提供方分组;只有一个分组时将其展开,以便对简单选择器支持更好的客户端展示。
|
||||
|
||||
如果适配器目录未包含会话当前目标,该目标仍会加入展示选项。这能保留自定义 DeepSeek 与私有端点模型,同时维持目录的建议性。提供方未注册的目标不会展示;缺少模型的 agent 仍可由其他 `agent/request` 提供者补齐。
|
||||
|
||||
`session/set_config_option` 只接受当前目录快照中的值,并更新该 ACP 会话独占的目标引用。它不会修改全局 `LlmService` 或 `AgentOptions` 状态,因此并发会话可以选择不同的提供方和模型。现有权限选择项保持独立,每次响应都返回完整的刷新后配置项状态。
|
||||
|
||||
### Prompt/请求一致性与持久化
|
||||
|
||||
Agent setup 会安装作用域内的 `system-prompt/assemble` 与 `agent/request` 监听器。Prompt 组装为每个 step 只快照一次选中的字段组合,在下游 prompt 监听器完成后覆盖组装结果中的 `provider` 与 `model` 变量;请求监听器则在下游请求监听器完成后应用同一个快照。因此,异步组装期间发生的选择会从下一个 step 生效,不会导致 prompt 文本与路由分裂。其他调用配置字段保持不变。
|
||||
|
||||
请求头仍是持久化事实来源。当选中目标被实际使用时,现有的完整 `request/header` 快照会记录它。`session/load` 先从折叠后的最后请求头初始化 ACP 选择,再回退到 bridge 配置。一个从未被请求使用的选择只保留在内存中,因为它从未成为模型可见状态。
|
||||
|
||||
本功能不使用 ACP 的实验性 `providers/*` 能力。该草案接口配置提供方 base URL、协议和 headers,其中可能包含密钥;它不枚举模型,并且会赋予 UI 改写部署所有的适配器配置的权力。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**只返回模型字符串。** 仅模型值会丢失提供方路由;两个提供方暴露相同 ID 时立刻产生歧义。
|
||||
|
||||
**将目录设为强制白名单。** 这与手写适配器的任意模型透传和私有部署冲突。请求的权威校验本就属于被选中的适配器。
|
||||
|
||||
**将选择存入 `AgentOptions` 或 `LlmService`。** 这些对象分别面向创建过程或整个部署。修改它们会耦合并发 ACP 会话,并绕开带日志归因的 `agent/request` 替换路径。
|
||||
|
||||
**立即写入新的模型选择会话事件。** 尚未使用的 UI 选择没有影响模型请求。目标被消费时记录现有请求头,既满足“模型可见当且仅当已记录”的规则,也不会引入第二个事实来源。
|
||||
|
||||
**使用 ACP `providers/*`。** 该不稳定 API 用于修改端点与认证配置,而不是为单个会话选择模型;其生命周期和密钥处理语义都不适合本功能。
|
||||
|
||||
## 结果
|
||||
|
||||
- 任意适配器都能暴露动态模型列表,无需把提供方库类型泄漏到核心接缝。
|
||||
- 目录消费者必须把缺失理解为“未展示”,而不是“请求无效”。
|
||||
- 基于 pi-ai 的 ACP 部署会自动继承已安装的 pi-ai 提供方目录;手写 DeepSeek 部署显式列出已知选项,同时保留任意模型能力。
|
||||
- ACP 客户端会收到稳定标准的模型配置项,其中的值保留提供方信息,并按会话隔离。
|
||||
- 请求头继续使用基于提供方路由的会话结构;不需要增加 JSONL 事件或格式版本。
|
||||
- 目录读取可以是异步的。ACP 在创建或恢复 agent 前读取分离后的快照,因此发现失败不会留下部分发布的会话。
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试覆盖目录分离与错误元数据、pi-ai 和 DeepSeek 目录投影、ACP 提供方分组、自定义当前模型补入、无效值、提供方/模型请求路由、prompt 变量一致性、并发会话隔离、无模型回退,以及从请求头恢复选择。现有 ACP 传输测试验证新增配置项不会改变 prompt、取消、回放、审批或工具展示行为。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-lsp-capability-seam.md: 7265b04ac9b2f83764bdd13f07b2d3404c4c1708
|
||||
2026-07-15-lsp-capability-seam.zh.md: 10e8956005045d0934dd9dada5718b85a34cda3f
|
||||
@@ -0,0 +1,198 @@
|
||||
# Agent Note: LSP capability seam and model-facing query tool
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-15-lsp-capability-seam.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The harness has text search and file reads, but neither identifies a program symbol. A textual match cannot reliably distinguish two same-named functions, follow an import alias, connect an interface to its implementations, or report an inferred type. Before changing code, an agent therefore lacks the semantic navigation that a human gets from an editor's language server.
|
||||
|
||||
LSP support has three owners: the model needs a stable query schema, the harness needs provider selection and normalized results, and the local implementation needs process, JSON-RPC, workspace, synchronization, and filesystem behavior. Combining them would bind the model contract to local subprocesses and obstruct remote or sandbox-native providers.
|
||||
|
||||
Many language servers behave best when the queried document is opened with current text. A compatible agent client must bound that state, define whether its source read is a model observation, and keep the document snapshot in the same filesystem namespace as the server's workspace index.
|
||||
|
||||
## Decision
|
||||
|
||||
Add LSP as a three-package capability seam with one read-only model tool and one generic local provider implementation:
|
||||
|
||||
1. `@deepseek-ai/dsh-lsp` at `packages/lsp/lsp` owns `ctx.lsp`, provider registration and selection, normalized requests/results, execution control, and structured LSP errors.
|
||||
2. `@deepseek-ai/dsh-lsp-local` at `packages/lsp/lsp-local` adapts configured stdio language servers to the seam. One plugin instance accepts a named server table and registers one isolated provider for each command and extension-to-language-id mapping.
|
||||
3. `@deepseek-ai/dsh-tool-lsp` at `packages/lsp/tool-lsp` owns the model-facing `lsp` schema, prompt guidance, argument validation, result limits and formatting, and ACP presentation.
|
||||
|
||||
`dsh-lsp-local` is a generic host, not a language-server catalog or installer. Deployments explicitly configure commands and mappings; future presets belong in composition plugins or `cordis.yml` overlays.
|
||||
|
||||
The model and seam expose exactly `goToDefinition`, `findReferences`, `goToImplementation`, and `hover`; no arbitrary JSON-RPC method escapes through `ctx.lsp`. These operation literals match Claude Code's familiar camelCase names while the tool name and `file_path` field remain harness-owned.
|
||||
|
||||
The prompt positions LSP as a precision aid: `Use search/read for ordinary navigation. Use lsp when textual matches are ambiguous or before a change requires precise definitions, implementations, or references.`
|
||||
|
||||
## Package and ownership boundaries
|
||||
|
||||
`dsh-lsp` registers providers by branded id and extension-to-language-id mapping. `registerProvider()` atomically reserves the id and every normalized extension: invalid input or any conflict publishes nothing, and its disposer releases all reservations. Provider plugins register through `ctx.effect()`. Selection is per query and order-independent; no match returns a structured unavailable error. The first version has no glob, language-id, or explicit route selector and no statically declared operation capabilities.
|
||||
|
||||
The seam exposes one `query(request, signal?)` operation because no fields need implementation defaulting: `workspaceRoot` is required, `languageId` comes from the registration, and consumers own timeouts and result limits. `query()` selects and derives without hidden `??` fallbacks, leaving no executable spec to resolve. `dsh-tool-lsp` validates model arguments and passes only `exec.signal` as a bare `AbortSignal`, matching web and keeping `dsh-lsp` independent of `dsh-tools`. Removal before selection fails as unavailable; later disposal follows the selected provider's cancellation lifecycle without rerouting.
|
||||
|
||||
The intended contract shape is:
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type LspOperation = 'goToDefinition' | 'findReferences' | 'goToImplementation' | 'hover'
|
||||
type LspProviderId = Branded<'LspProviderId'>
|
||||
|
||||
interface LspPosition {
|
||||
readonly line: number
|
||||
readonly character: number
|
||||
}
|
||||
|
||||
interface LspRange {
|
||||
readonly start: LspPosition
|
||||
readonly end: LspPosition
|
||||
}
|
||||
|
||||
interface LspQueryRequest {
|
||||
readonly operation: LspOperation
|
||||
readonly filePath: string
|
||||
readonly position: LspPosition
|
||||
readonly workspaceRoot: string
|
||||
}
|
||||
|
||||
interface LspProviderQuery extends LspQueryRequest {
|
||||
readonly languageId: string
|
||||
}
|
||||
|
||||
type LspQueryResult =
|
||||
| { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string }
|
||||
| { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null }
|
||||
|
||||
interface LspProvider {
|
||||
readonly id: LspProviderId
|
||||
readonly extensionToLanguage: Readonly<Record<string, string>>
|
||||
query(request: LspProviderQuery, signal?: AbortSignal): Promise<LspQueryResult>
|
||||
}
|
||||
|
||||
interface LspService {
|
||||
registerProvider(provider: LspProvider): () => void
|
||||
query(request: LspQueryRequest, signal?: AbortSignal): Promise<LspQueryResult>
|
||||
}
|
||||
```
|
||||
|
||||
Mapping keys normalize to lowercase, leading-dot extensions selected from `filePath`'s final extension; language ids only synchronize documents. Seam positions and ranges are zero-based UTF-16. `findReferences` always includes declarations: providers enforce this internally, the local mapping sets `context.includeDeclaration: true`, and callers get no flag. Closed result unions normalize navigation to locations and hover to content or `null`; navigation results carry the provider's resolved workspace root so consumers relativize file URIs in the same canonical namespace. The seam exposes no protocol types, process or document controls, or generic request escape hatch.
|
||||
|
||||
`dsh-lsp-local` owns host files, server configuration, JSON-RPC, process and transient-document state, and protocol translation; it depends on `dsh-lsp` and Node APIs, not `dsh-fs`. The server-table key is its provider id. The plugin resolves every server-local setting before registration, rolls back earlier registrations if a later mapping is invalid or conflicts, and retains an independent process pool per provider. `dsh-tool-lsp` runtime-injects only `tools`, `lsp`, and `systemPrompt`, obtains the workspace from `exec.agent?.session.header.cwd` through a package-local `sessionCwd(exec)` helper matching the filesystem tools' lookup, and imports no provider.
|
||||
|
||||
## Model-facing contract
|
||||
|
||||
The single `lsp` tool accepts:
|
||||
|
||||
```ts
|
||||
interface LspToolInput {
|
||||
readonly operation: 'goToDefinition' | 'findReferences' | 'goToImplementation' | 'hover'
|
||||
readonly file_path: string
|
||||
readonly line: number
|
||||
readonly character: number
|
||||
}
|
||||
```
|
||||
|
||||
`line` and `character` are positive, one-based UTF-16 cursor coordinates; the tool converts them to the seam's zero-based `LspPosition` and converts rendered locations back. `findReferences` includes declarations so impact analysis does not omit the defining site. Provider, language id, workspace root, limits, timeout, initialization, and executable remain outside model input.
|
||||
|
||||
The tool requires `workspaceRoot` from session `header.cwd`, with no fallback; absence fails as `LSP_WORKSPACE_REQUIRED` before querying or startup. The local provider resolves relative paths against that root and accepts absolute paths directly; both forms are canonicalized and rejected before startup when the target is outside the canonical workspace.
|
||||
|
||||
Locations render as stable, file-grouped `path:line:character` entries. A `file:` URI accepted by Node `fileURLToPath()` becomes a relative path inside the workspace or an absolute path outside it; other URIs remain verbatim. `maxLocations` defaults to `100` and reports omitted items; `maxResultChars` defaults to `16_000` and bounds every complete rendered result, including its truncation metadata. Empty locations and `null` hover are successful no-result responses; missing or malformed server payloads fail with structured `LSP_MALFORMED_RESPONSE` errors.
|
||||
|
||||
ACP uses `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }` with an args-derived operation/cursor `title`. Because `FileLocation` has no character, follow-along focuses the input line while the title preserves the cursor; presentation remains pure.
|
||||
|
||||
## Timeout ownership
|
||||
|
||||
`dsh-tool-lsp` attaches one configurable `timeoutMs` budget, default `60_000`, to the tool definition. `dsh-timeout-policy` enforces it and supplies `exec.signal`, which reaches `ctx.lsp.query`; the budget covers the complete queued open/query/close lifecycle and is not model-configurable.
|
||||
|
||||
The seam and provider add no startup or request deadline. Non-tool callers therefore receive no hidden timeout and must supply an `AbortSignal`, using `deadline()` when they need a budget.
|
||||
|
||||
Provider disposal occurs outside tool execution, so `dsh-lsp-local` keeps `shutdownTimeoutMs` (default `5_000`) for `shutdown`/`exit` and `killGraceMs` (default `2_000`) for both request-cancel grace and SIGTERM-to-SIGKILL escalation; the same bounds govern failed-instance cleanup. Timer values above Node's `2_147_483_647` ms scheduling range fail at load. The provider uses `deadline()` and `timeoutOf()` but owns request cancellation, process signals, and awaiting close because timeout notification does not terminate work.
|
||||
|
||||
## Workspace, filesystem, and document synchronization
|
||||
|
||||
`dsh-lsp-local` canonicalizes and reads through Node APIs in the subprocess's host namespace. It rejects missing, non-regular, non-UTF-8, oversized, or canonical out-of-workspace sources and keeps one `O_NOFOLLOW | O_NONBLOCK` handle through validation and reading, so a FIFO with no writer cannot block before the regular-file check. It observes caller cancellation around each filesystem operation. It does not consume `ctx.fs` or emit `fs/observed`: only the LSP result is model-visible, so the query does not satisfy read-before-write policy.
|
||||
|
||||
The `read` tool is unsuitable source because its output is windowed, numbered, transcript-visible, and observed. Reading in `tool-lsp` would also assign provider-specific synchronization to the consumer and preclude non-local providers.
|
||||
|
||||
The local provider uses a compatibility-first transient-open sequence for every query. It accepts legacy `textDocumentSync` `Full` or `Incremental`, or options with `openClose: true`; omitted, `None`, or explicitly incompatible synchronization fails as unsupported before `didOpen`.
|
||||
|
||||
1. Canonicalize and validate the host path, then read the current source with Node filesystem APIs.
|
||||
2. Send `textDocument/didOpen` with version `1`, full text, and the configured language id. Its write remains abortable; failure or cancellation invalidates the instance and awaits bounded process termination before the pool can reuse it.
|
||||
3. Send the requested `textDocument/definition`, `textDocument/references`, `textDocument/implementation`, or `textDocument/hover` request.
|
||||
4. If `didOpen` succeeded, attempt `textDocument/didClose` in `finally` after the request settles or aborts. A close-write failure does not replace the settled result or error, but invalidates the instance and awaits bounded process termination.
|
||||
|
||||
Documents close after each call, so the first version needs no `didChange`, `didSave`, content cache, mutation listener, or document LRU. One abortable per-workspace provider queue serializes source-read/open/query/close lifecycles, so a waiting query reads current bytes only when its turn starts; the instance also keeps protocol lifecycles serialized. Distinct workspaces may run in parallel. The server's workspace index remains responsible for closed files reached from the source.
|
||||
|
||||
The canonical workspace `realpath` must be a directory and supplies process cwd, `rootUri`, the sole `workspaceFolders` entry, and pool identity; symlink aliases therefore share an instance. Result locations may be external, but an external path cannot become a query source. Remote, virtual, or independently sandboxed filesystems require another provider.
|
||||
|
||||
## Local server lifecycle and protocol behavior
|
||||
|
||||
`dsh-lsp-local` lazily single-flights one server per `(provider id, canonical workspace realpath)`. At load it resolves the executable after credential scrubbing and environment overrides, failing before registration if unavailable; server process launch stays lazy (first query spawns it) and uses no shell. `maxMessageBytes` defaults to `16_000_000`, `maxStderrBytes` to `1_000_000`, and `maxDocumentBytes` to `4_000_000`. A crash fails the active query without replay; a later query may replace the process. Each query starts at most one process, so the MVP has no cross-request restart counter.
|
||||
|
||||
Initialization advertises `general.positionEncodings: ['utf-16']`, `workspace: { workspaceFolders: true, configuration: true }`, `textDocument.hover.contentFormat: ['markdown', 'plaintext']`, and `linkSupport: true` for definition and implementation, with no dynamic registration. Returned operation and synchronization capabilities are authoritative. An omitted server `positionEncoding` defaults to `utf-16`; any other value is a protocol error. Configuration may supply initialization options and `workspace/configuration` responses, but the client rejects `workspace/applyEdit` and never executes commands or edits.
|
||||
|
||||
Navigation maps `Location` directly and `LocationLink` from `targetUri` plus `targetSelectionRange`. Positions must be nonnegative integers. Hover normalization accepts only valid `MarkupContent` and `MarkedString` shapes, preserves string values, renders language-tagged values as fenced code, and joins arrays with one blank line. The model-facing tool applies `maxResultChars` after rendering.
|
||||
|
||||
Abort reaches every query phase and sends `$/cancelRequest` once an id exists. An unresponsive server is terminated and awaited without collateral active work because the instance is serialized. Disposal rejects and cancels work, attempts graceful shutdown, escalates through bounded termination, and awaits quiescence.
|
||||
|
||||
## Deliberately deferred surface
|
||||
|
||||
Symbols are deferred because they need different schemas and overlap read/search; a future workspace-symbol tool must accept a search query. Call hierarchy is deferred because support is uneven, and `prepareCallHierarchy` remains an internal prerequisite rather than a model operation.
|
||||
|
||||
Diagnostics need separate freshness, accumulation, and transcript rules. Mutations such as rename, code actions, and formatting require separate tools with preview, permission, and write-policy integration.
|
||||
|
||||
The local provider trusts its configured server and claims no sandbox confinement. Supporting untrusted binaries requires a later process/filesystem contract for workspace reads plus private cache and temporary writes; restricted, remote, or virtual workspaces require another provider.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Copy Claude Code's unified schema.** Its cursor operations validate the core use case, but symbols and call hierarchy need different arguments. Copying all nine operations would freeze speculative surface, so the proposal aligns only on the four semantic queries.
|
||||
|
||||
**Let providers register tools.** Loaded servers would then control model schema and prompts, preventing one stable contract across local and remote providers.
|
||||
|
||||
**Expose arbitrary LSP methods.** A JSON-RPC escape hatch would leak protocol payloads and admit unreviewed mutation or command execution; the operation union stays closed.
|
||||
|
||||
**Expose `resolve(request)` / `query(spec)`.** With no defaulted fields, resolution would only expose provider selection, and a public spec could outlive provider disposal or replacement. One operation keeps selection and invocation atomic to the registration lifetime.
|
||||
|
||||
**Wrap the signal in a per-seam execution-context object.** Web passes a bare `AbortSignal`; wrapping this single field would add unexplained asymmetry. `query()` gains a context object only when another field requires it.
|
||||
|
||||
**Read through `ctx.fs` or the `read` tool.** This could mix the document with a server index from another filesystem namespace; tool output is also windowed, numbered, and observed. The host-local provider reads unobserved full text beside its subprocess.
|
||||
|
||||
**Keep documents open.** Mirroring edits requires version ownership, all-path `didChange`, HMR recovery, eviction, and stale-state rules. Transient opens avoid that MVP state machine.
|
||||
|
||||
**Configure phase timeouts.** Nested timers create competing classifications and fresh budgets. One caller-owned deadline covers query work; only out-of-call teardown keeps local bounds.
|
||||
|
||||
**Query without `didOpen`.** Although permitted, support is inconsistent and may use stale server state. Transient open supplies an explicit current snapshot.
|
||||
|
||||
**Add routes or select the first match.** Registration order and HMR timing are not product semantics, while a route table duplicates unique extension ownership. Overlaps therefore fail registration.
|
||||
|
||||
**Run concurrent queries in one instance.** If cancellation fails, terminating the shared process would kill unrelated work. Per-instance serialization limits that blast radius; instances remain parallel.
|
||||
|
||||
**Ship presets or PATH discovery.** A catalog would make the generic host own language policy, while discovery cannot infer arguments, language ids, or initialization. Deployments configure providers explicitly; composition plugins may package presets.
|
||||
|
||||
## Testing
|
||||
|
||||
- Package tests pin the three-package dependency direction, runtime injections, and `ctx.lsp`-only communication.
|
||||
- Tool tests pin the four operations, coordinate validation, configured bounds and omission markers, prompt, and ACP presentation.
|
||||
- Registry tests pin atomic reservation/release, order-independent selection, and structured unavailable, disposed, conflict, and unsupported-operation errors.
|
||||
- Fake-stdio tests pin exact initialization capabilities, four protocol mappings, `Location`/`LocationLink` and hover normalization, and `findReferences` mapping to `references.includeDeclaration`.
|
||||
- Synchronization tests pin UTF-16 negotiation and conversion, supported and rejected `textDocumentSync` forms, blocked and failed open writes, balanced transient open/close, close-write failure, and malformed-response rejection.
|
||||
- Timeout tests pin one `TOOL_TIMEOUT` budget, unclassified upstream cancellation, no hidden seam deadline, and bounded awaited teardown.
|
||||
- Lifecycle tests pin startup single-flight, complete-lifecycle serialization with fresh queued source reads, cross-workspace parallelism, abortable queues, crash replacement without replay, failed-stdin teardown, and quiescent disposal.
|
||||
- Host-filesystem tests pin session-cwd requirements, relative and absolute source containment through symlinks, document validation, file/non-file URI rendering, unformatted source, and no `fs/observed` event.
|
||||
- A keyless pinned TypeScript real-server e2e exercises all four operations; runnable configuration uses the same explicit provider mapping.
|
||||
- Snapshots cover model-visible schema, prompt, results, omissions, and ACP rendering; a built-artifact smoke test covers framing and cleanup.
|
||||
- Package and architecture docs cover configuration, security boundaries, and search/read guidance; the new `packages/lsp/` group is added to the AGENTS.md repository-layout block, the packages/README.md group table, and architecture.md in the same change.
|
||||
|
||||
## Consequences
|
||||
|
||||
Language servers vary in method support, capability interpretation, and indexing readiness; LSP has no universal “index complete” signal. Servers without compatible transient-open synchronization are unsupported even if closed-document queries work. Supported servers may still return empty or partial results, so the tool promises no cross-server completeness. The pinned TypeScript e2e establishes one compatibility floor, not a cross-language claim.
|
||||
|
||||
Transient opens repeat parsing and notifications. Per-instance serialization increases latency under parallel agents, and long-lived workspace processes consume memory until disposal.
|
||||
|
||||
Extension ownership is exclusive within one runtime. Two providers cannot both claim `.ts`, even with different language ids; this is a conscious MVP limit. The intended extension is a deployment-configured selector above registrations that can relax exclusive reservations without adding provider choice to model input or changing `LspProvider.query`.
|
||||
|
||||
UTF-16 cursor columns are exact for the protocol but difficult for a model to count around non-BMP characters. Invalid or off-symbol positions may produce empty results, so error text and prompt examples must explain the coordinate convention without encouraging broad LSP use.
|
||||
|
||||
Direct Node access aligns the query snapshot with the server index but bypasses `ctx.fs` and its policy. Canonical containment rejects source files outside the workspace; a trusted server may still read the workspace and use caches. The first implementation therefore requires trusted host-local deployment and provides no sandbox guarantee.
|
||||
@@ -0,0 +1,198 @@
|
||||
# Agent Note: LSP 能力服务边界与面向模型的查询工具
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-15-lsp-capability-seam.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
harness 已具备文本搜索与文件读取能力,但二者都无法识别程序符号。文本匹配无法可靠地区分同名函数、跟踪导入别名、关联接口与具体实现,也无法报告推断类型。因此,agent(智能体)在修改代码前缺少人类通过编辑器语言服务器获得的语义导航能力。
|
||||
|
||||
语言服务器协议(Language Server Protocol,LSP)支持分属三个职责方:模型需要稳定的查询 schema,harness 需要提供方选择与规范化结果,本地实现则负责进程、JSON-RPC、工作区、同步与文件系统行为。将三者合并会使模型契约绑定本地子进程,并阻碍远程或沙箱原生提供方。
|
||||
|
||||
许多语言服务器在查询文档已按当前文本打开时表现最佳。兼容的 agent 客户端必须限制这项状态、定义内部读取是否算作模型观察,并确保文档快照与服务器工作区索引位于同一文件系统命名空间。
|
||||
|
||||
## 决策
|
||||
|
||||
将 LSP 建成由三个包(package)组成的能力服务边界,其中包含一个只读模型工具和一个通用本地提供方实现:
|
||||
|
||||
1. `packages/lsp/lsp` 下的 `@deepseek-ai/dsh-lsp` 负责 `ctx.lsp`、提供方注册与选择、标准化请求与结果、执行控制,以及结构化 LSP 错误。
|
||||
2. `packages/lsp/lsp-local` 下的 `@deepseek-ai/dsh-lsp-local` 将配置的 stdio 语言服务器适配到该服务边界。一个插件实例接收具名服务器表,并为每组命令及扩展名到语言 id 的映射注册一个隔离的提供方。
|
||||
3. `packages/lsp/tool-lsp` 下的 `@deepseek-ai/dsh-tool-lsp` 负责面向模型的 `lsp` schema、提示词指导、参数校验、结果限制与格式化,以及 ACP(Agent Client Protocol)展示。
|
||||
|
||||
`dsh-lsp-local` 是通用 host,不是语言服务器目录或安装器。部署显式配置命令与映射;未来 preset 属于组合插件或 `cordis.yml` overlay。
|
||||
|
||||
模型与服务边界仅公开 `goToDefinition`、`findReferences`、`goToImplementation` 和 `hover`;`ctx.lsp` 不提供任意 JSON-RPC 方法。这些操作字面量与 Claude Code 熟悉的 camelCase 命名一致,而工具名与 `file_path` 字段仍由 harness 自行定义。
|
||||
|
||||
提示词将 LSP 定位为精确查询手段:`Use search/read for ordinary navigation. Use lsp when textual matches are ambiguous or before a change requires precise definitions, implementations, or references.`
|
||||
|
||||
## 包与职责边界
|
||||
|
||||
`dsh-lsp` 按带品牌类型的 id 和扩展名到语言 id 的映射注册提供方。`registerProvider()` 以原子方式占用 id 与所有规范化扩展名:输入无效或存在冲突时不发布任何状态,清理函数释放全部占用。提供方插件通过 `ctx.effect()` 注册。系统按查询且不受顺序影响地选择提供方;没有匹配项时返回结构化不可用错误。第一版不提供 glob、language-id 或显式路由选择器,也不静态声明操作能力。
|
||||
|
||||
服务边界只公开 `query(request, signal?)`,因为没有字段需要实现层填充默认值:`workspaceRoot` 是必填项,`languageId` 来自注册映射,超时与结果限制由消费方负责。`query()` 执行选择与推导时不使用隐藏的 `??` 后备逻辑,因此没有需要 resolve 的可执行 spec。`dsh-tool-lsp` 校验模型参数,并只把 `exec.signal` 作为裸 `AbortSignal` 传递,与 web 一致,并使 `dsh-lsp` 不依赖 `dsh-tools`。提供方在选择前被移除时按不可用失败;之后的释放遵循已选提供方的取消生命周期,不改路由。
|
||||
|
||||
预期契约如下:
|
||||
|
||||
```ts
|
||||
import type { Branded } from '@deepseek-ai/dsh-brand'
|
||||
|
||||
type LspOperation = 'goToDefinition' | 'findReferences' | 'goToImplementation' | 'hover'
|
||||
type LspProviderId = Branded<'LspProviderId'>
|
||||
|
||||
interface LspPosition {
|
||||
readonly line: number
|
||||
readonly character: number
|
||||
}
|
||||
|
||||
interface LspRange {
|
||||
readonly start: LspPosition
|
||||
readonly end: LspPosition
|
||||
}
|
||||
|
||||
interface LspQueryRequest {
|
||||
readonly operation: LspOperation
|
||||
readonly filePath: string
|
||||
readonly position: LspPosition
|
||||
readonly workspaceRoot: string
|
||||
}
|
||||
|
||||
interface LspProviderQuery extends LspQueryRequest {
|
||||
readonly languageId: string
|
||||
}
|
||||
|
||||
type LspQueryResult =
|
||||
| { readonly kind: 'locations'; readonly locations: readonly { readonly uri: string; readonly range: LspRange }[]; readonly resolvedWorkspaceRoot: string }
|
||||
| { readonly kind: 'hover'; readonly hover: { readonly contents: string; readonly range?: LspRange } | null }
|
||||
|
||||
interface LspProvider {
|
||||
readonly id: LspProviderId
|
||||
readonly extensionToLanguage: Readonly<Record<string, string>>
|
||||
query(request: LspProviderQuery, signal?: AbortSignal): Promise<LspQueryResult>
|
||||
}
|
||||
|
||||
interface LspService {
|
||||
registerProvider(provider: LspProvider): () => void
|
||||
query(request: LspQueryRequest, signal?: AbortSignal): Promise<LspQueryResult>
|
||||
}
|
||||
```
|
||||
|
||||
映射键规范化为带前导点的小写扩展名,并按 `filePath` 的最后一个扩展名选择;语言 id 仅用于文档同步。服务边界中的位置和范围从零开始按 UTF-16 计数。`findReferences` 始终包含声明:提供方在内部执行该约束,本地映射设置 `context.includeDeclaration: true`,调用方不能配置。封闭结果联合将导航统一为位置,将 `hover` 统一为内容或 `null`;导航结果携带提供方解析后的工作区根目录,使消费方依据同一规范化根目录相对化文件 URI。服务边界不公开协议类型、进程或文档控制,也不提供通用请求逃生口。
|
||||
|
||||
`dsh-lsp-local` 负责主机文件、服务器配置、JSON-RPC、进程与临时文档状态和协议转换;它依赖 `dsh-lsp` 与 Node API,不依赖 `dsh-fs`。服务器表的键是提供方 id。插件在注册前解析每个服务器的本地设置;如果后续映射无效或发生冲突,插件会撤销此前的注册,并为每个提供方保留独立进程池。`dsh-tool-lsp` 在运行时只注入 `tools`、`lsp` 和 `systemPrompt`,通过包内的 `sessionCwd(exec)` 辅助函数从 `exec.agent?.session.header.cwd` 取得工作区,其取值方式与文件系统工具一致,也不导入提供方。
|
||||
|
||||
## 面向模型的契约
|
||||
|
||||
单一 `lsp` 工具接受以下参数:
|
||||
|
||||
```ts
|
||||
interface LspToolInput {
|
||||
readonly operation: 'goToDefinition' | 'findReferences' | 'goToImplementation' | 'hover'
|
||||
readonly file_path: string
|
||||
readonly line: number
|
||||
readonly character: number
|
||||
}
|
||||
```
|
||||
|
||||
`line` 和 `character` 是从一开始计数的正数 UTF-16 光标坐标;工具将其转换为服务边界中从零开始的 `LspPosition`,并将渲染位置转回。`findReferences` 包含声明,避免影响分析漏掉定义位置。提供方、语言 id、工作区根目录、限制、超时、初始化和可执行文件均不进入模型输入。
|
||||
|
||||
工具必须从会话 `header.cwd` 取得 `workspaceRoot`,没有后备值;缺失时在查询或启动前以 `LSP_WORKSPACE_REQUIRED` 失败。本地提供方基于根目录解析相对路径并直接接受绝对路径;两种路径都会进行规范化,如果目标位于规范工作区外,则在启动前拒绝。
|
||||
|
||||
位置按文件稳定分组并渲染为 `path:line:character`。Node `fileURLToPath()` 可接受的 `file:` URI 在工作区内转换为相对路径,在工作区外转换为绝对路径;其他 URI 保持原样。`maxLocations` 默认值为 `100`,并报告省略的条目;`maxResultChars` 默认值为 `16_000`,并限制每个完整渲染结果,其中包括截断元数据。空位置与 `null` hover 是成功的无结果响应;服务器载荷缺失或格式错误时,以结构化 `LSP_MALFORMED_RESPONSE` 错误失败。
|
||||
|
||||
ACP 使用 `{ card: 'generic', kind: 'search', title, locations: [{ path: file_path, line }] }`,`title` 由参数推导并标明操作与光标。由于 `FileLocation` 没有 character,跟随位置聚焦输入行,标题保留完整光标;展示保持纯函数。
|
||||
|
||||
## 超时归属
|
||||
|
||||
`dsh-tool-lsp` 将一个可配置的 `timeoutMs` 预算附加到工具定义,默认值为 `60_000`。`dsh-timeout-policy` 执行预算并提供传入 `ctx.lsp.query` 的 `exec.signal`;该预算覆盖排队、打开、查询和关闭的完整生命周期,模型不可配置。
|
||||
|
||||
服务边界和提供方不增加启动或请求截止时间。非工具调用方不会获得隐藏超时,必须自行提供 `AbortSignal`,并在需要预算时使用 `deadline()`。
|
||||
|
||||
提供方释放发生在工具执行之外,因此 `dsh-lsp-local` 保留 `shutdownTimeoutMs`(默认 `5_000`)限制 `shutdown`/`exit`,以及 `killGraceMs`(默认 `2_000`),同时用于限制请求取消宽限期和从 SIGTERM 升级到 SIGKILL 的宽限期;失败实例的清理也使用相同边界。定时器值超过 Node `2_147_483_647` ms 的调度范围时,插件加载失败。提供方使用 `deadline()` 和 `timeoutOf()`,但仍负责请求取消、进程信号和等待关闭,因为超时通知不会终止工作。
|
||||
|
||||
## 工作区、文件系统与文档同步
|
||||
|
||||
`dsh-lsp-local` 通过 Node API 在子进程所在的主机命名空间中规范化并读取文件。它拒绝缺失、非普通、非 UTF-8、超大或规范路径越出工作区的源文件,并在校验与读取期间保持同一个 `O_NOFOLLOW | O_NONBLOCK` 句柄,因此没有写入方的 FIFO 不会在普通文件校验前造成阻塞。它在每项文件系统操作前后检查调用方是否取消。它不使用 `ctx.fs` 或发送 `fs/observed`:只有 LSP 结果对模型可见,因此查询不满足写前读取策略。
|
||||
|
||||
`read` 工具的输出带窗口与行号,进入 transcript(文本记录)且已被观察,不适合作为源文件。在 `tool-lsp` 内读取还会把提供方专用同步职责交给消费方,并排除非本地提供方。
|
||||
|
||||
本地提供方对每次查询都采用兼容优先的临时打开流程。它接受旧式 `textDocumentSync` 的 `Full` 或 `Incremental`,也接受设置了 `openClose: true` 的选项;同步能力缺失、为 `None` 或明确不兼容时,在 `didOpen` 前以不支持错误失败。
|
||||
|
||||
1. 规范化并校验主机路径,再使用 Node 文件系统 API 读取当前源文件。
|
||||
2. 发送 `textDocument/didOpen`,其中包含版本 `1`、完整文本和配置的语言 id。该写入仍可取消;写入失败或遭取消会使实例失效,并等待有界进程终止完成,池才能复用它。
|
||||
3. 发送所请求的 `textDocument/definition`、`textDocument/references`、`textDocument/implementation` 或 `textDocument/hover` 请求。
|
||||
4. 如果 `didOpen` 成功,则在请求完成或取消后于 `finally` 中尝试发送 `textDocument/didClose`。关闭写入失败不会覆盖已经确定的结果或错误,但会使实例失效,并等待有界进程终止完成。
|
||||
|
||||
每次调用后都关闭文档,因此第一版不需要 `didChange`、`didSave`、内容缓存、变更监听器或文档 LRU。每个工作区的提供方队列可取消,并串行执行源文件读取、打开、查询和关闭的完整生命周期,因此等待中的查询只在轮到它时才读取当前字节;实例也会串行执行协议生命周期。不同工作区可以并行。服务器工作区索引仍负责从源文件跳转到的已关闭文件。
|
||||
|
||||
规范工作区 `realpath` 必须是目录,并用于进程 cwd、`rootUri`、唯一的 `workspaceFolders` 条目和进程池 identity;符号链接别名因此共享实例。结果位置可以在工作区外,但外部路径不能成为查询源。远程、虚拟或独立沙箱化文件系统需要另一种提供方。
|
||||
|
||||
## 本地服务器生命周期与协议行为
|
||||
|
||||
`dsh-lsp-local` 按 `(provider id, canonical workspace realpath)` 懒启动一个服务器,并通过 single-flight 合并启动。插件加载时,它在清除凭据并应用环境变量覆盖后解析可执行文件;命令不可用时在注册前失败。服务器进程的启动保持懒执行(首次查询时才拉起),且不经过 shell。`maxMessageBytes` 默认值为 `16_000_000`,`maxStderrBytes` 默认值为 `1_000_000`,`maxDocumentBytes` 默认值为 `4_000_000`。崩溃使当前查询失败且不重放;后续查询可以替换进程。每次查询最多启动一个进程,因此 MVP 不设置跨请求重启计数器。
|
||||
|
||||
初始化声明 `general.positionEncodings: ['utf-16']`、`workspace: { workspaceFolders: true, configuration: true }`、`textDocument.hover.contentFormat: ['markdown', 'plaintext']`,以及 definition 与 implementation 的 `linkSupport: true`,但不支持动态注册。服务器返回的操作与同步能力均为真源。服务器省略 `positionEncoding` 时默认为 `utf-16`;其他值均属于协议错误。配置可以提供初始化选项和 `workspace/configuration` 响应,但客户端拒绝 `workspace/applyEdit`,绝不执行命令或编辑。
|
||||
|
||||
导航结果直接映射 `Location`,并将 `LocationLink` 的 `targetUri` 与 `targetSelectionRange` 映射为统一位置。位置必须是非负整数。`hover` 归一化只接受有效的 `MarkupContent` 和 `MarkedString` 结构,保留字符串值,把带语言标签的值渲染为围栏代码块,并以一个空行连接数组。面向模型的工具在渲染后应用 `maxResultChars`。
|
||||
|
||||
取消信号传递到查询的所有阶段,请求 id 创建后还会发送 `$/cancelRequest`。无响应的服务器会被终止并等待关闭;实例串行化保证没有其他正在执行的工作被连带中断。资源释放会拒绝并取消工作、尝试优雅关闭、通过有界终止流程升级处理,并等待完全停稳。
|
||||
|
||||
## 明确延后的接口
|
||||
|
||||
符号操作因需要不同 schema 且与读取或搜索重叠而延后;未来的工作区符号工具必须接收搜索词。调用层级因支持度不一而延后,`prepareCallHierarchy` 仍是内部准备步骤,不是模型操作。
|
||||
|
||||
诊断需要独立的新鲜度、累积与 transcript 规则。重命名、代码操作和格式化等变更能力需要单独工具,并集成预览、权限和写入策略。
|
||||
|
||||
本地提供方信任配置的服务器,不声称具备沙箱隔离。支持不受信任的二进制文件需要后续补充允许读取工作区并写入私有缓存与临时目录的进程/文件系统契约;受限、远程或虚拟工作区需要另一种提供方。
|
||||
|
||||
## 备选方案
|
||||
|
||||
**照搬 Claude Code 的统一 schema。** 它的光标操作验证了核心场景,但符号与调用层级需要不同参数。照搬九种操作会固化尚未验证的接口,因此本提案只对齐四种语义查询。
|
||||
|
||||
**允许提供方注册工具。** 已加载服务器会控制模型 schema 和提示词,无法在本地与远程提供方之间维持统一契约。
|
||||
|
||||
**公开任意 LSP 方法。** JSON-RPC 逃生口会泄露协议载荷,并允许未经评审的变更或命令执行;操作联合保持封闭。
|
||||
|
||||
**公开 `resolve(request)` / `query(spec)`。** 没有需要填充默认值的字段时,resolve 只会暴露提供方选择,而公开 spec 可能活过提供方释放或替换。单一操作让选择与调用共用注册生命周期。
|
||||
|
||||
**将信号包装为每服务边界的执行上下文对象。** Web 传递裸 `AbortSignal`;仅包装这一个字段会造成无谓的不对称。只有另一个字段确有需要时,`query()` 才引入上下文对象。
|
||||
|
||||
**通过 `ctx.fs` 或 `read` 工具读取。** 这可能把文档与另一文件系统命名空间中的服务器索引混合;工具输出还带窗口、行号且已被观察。host-local 提供方在子进程旁读取未观察的完整文本。
|
||||
|
||||
**保持文档打开。** 镜像编辑需要版本归属、覆盖所有路径的 `didChange`、HMR 恢复、淘汰和陈旧状态规则。临时打开避免在 MVP 引入这套状态机。
|
||||
|
||||
**配置分阶段超时。** 嵌套定时器会产生相互竞争的分类与新预算。一个由调用方负责的截止时间覆盖查询;只有调用外清理保留本地限制。
|
||||
|
||||
**不发送 `didOpen`。** 协议虽允许,但支持不一致且可能使用陈旧服务器状态。临时打开提供明确的当前快照。
|
||||
|
||||
**增加路由或选择首个匹配项。** 注册顺序与 HMR 时机不是产品语义,路由表又会重复唯一扩展名所有权。因此,扩展名重叠时注册失败。
|
||||
|
||||
**在一个实例中并发查询。** 取消失败时,终止共享进程会杀死无关工作。实例内串行可限制影响范围;不同实例仍可并行。
|
||||
|
||||
**内置 preset 或 PATH 发现。** 目录会让通用 host 承担语言策略,而发现机制无法推断参数、语言 id 或初始化配置。部署显式配置提供方,组合插件可以封装 preset。
|
||||
|
||||
## 测试
|
||||
|
||||
- 包测试固定三个包的依赖方向、运行时注入和仅通过 `ctx.lsp` 通信的边界。
|
||||
- 工具测试固定四种操作、坐标校验、配置限制与省略标记、提示词和 ACP 展示。
|
||||
- 注册表测试固定原子占用/释放、不受顺序影响的选择,以及结构化的不可用、已释放、冲突和不支持操作错误。
|
||||
- 测试用 stdio server 固定精确的初始化能力、四种协议映射、`Location`/`LocationLink` 与 `hover` 归一化,以及 `findReferences` 到 `references.includeDeclaration` 的映射。
|
||||
- 同步测试固定 UTF-16 协商与转换、受支持和被拒绝的 `textDocumentSync` 形式、打开写入阻塞与失败、配对的临时打开/关闭、关闭写入失败和错误响应拒绝。
|
||||
- 超时测试固定一个 `TOOL_TIMEOUT` 预算、不对上游取消错误分类、服务边界无隐藏截止时间,以及受限且等待完成的清理。
|
||||
- 生命周期测试固定启动 single-flight、完整生命周期串行化及排队查询读取最新源文件、跨工作区并行、可取消队列、崩溃后不重放的替换、stdin 失败后的进程拆除,以及释放后完全停稳。
|
||||
- 主机文件系统测试固定 session cwd 要求、符号链接下相对与绝对源路径的规范 containment、文档校验、file/non-file URI 渲染、无格式源文本和不发送 `fs/observed`。
|
||||
- 无密钥且固定版本的 TypeScript 真实服务器 e2e 覆盖四种操作;可运行配置使用同一项显式提供方映射。
|
||||
- 快照覆盖模型可见 schema、提示词、结果、省略提示和 ACP 渲染;构建产物冒烟测试覆盖分帧与清理。
|
||||
- 包与架构文档覆盖配置、安全边界和搜索/读取指导;同一改动中,新的 `packages/lsp/` 包组要加入 AGENTS.md 的仓库布局块、packages/README.md 的分组表和 architecture.md。
|
||||
|
||||
## 影响
|
||||
|
||||
各语言服务器对方法支持、能力解释和索引就绪时机的处理不同;LSP 没有统一的“索引完成”信号。无法声明兼容临时打开同步能力的服务器不受支持,即使它能查询已关闭文档。受支持的服务器仍可能返回空结果或不完整结果,因此工具不承诺跨服务器完整性。固定的 TypeScript e2e 只建立一条兼容性基线,不代表跨语言承诺。
|
||||
|
||||
临时打开会重复解析并产生通知。实例内串行会增加并发 agent 的延迟,长期运行的工作区进程则持续占用内存直到释放。
|
||||
|
||||
同一运行时内的扩展名所有权互斥。即使 language id 不同,两个提供方也不能同时占用 `.ts`;这是有意接受的 MVP 限制。预期扩展方式是在注册之上增加由部署配置的 selector,允许放宽互斥占用,同时不向模型输入增加提供方选择,也不改变 `LspProvider.query`。
|
||||
|
||||
UTF-16 光标列与协议完全一致,但模型难以在包含非 BMP 字符的文本中准确计数。无效位置或不在符号上的位置可能返回空结果,因此错误文本和提示词示例必须说明坐标约定,同时避免鼓励模型广泛使用 LSP。
|
||||
|
||||
直接访问 Node 文件系统会对齐查询快照与服务器索引,但绕过 `ctx.fs` 及其策略。规范路径 containment 会拒绝工作区外的源文件;受信任的服务器仍可读取工作区并使用缓存。因此,第一版要求受信任的 host-local 部署,不提供沙箱保证。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-replay-token-meter-service.md: 4bedbb0cb9fa383108a688dbbb32546c7f39bd20
|
||||
2026-07-15-replay-token-meter-service.zh.md: ccf1b014cd86d16ef498b4209818e78be562e66f
|
||||
@@ -0,0 +1,60 @@
|
||||
# Agent Note: Replay token meter service
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-15-replay-token-meter-service.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Context pressure is useful outside compaction. A compaction backend, an overflow guard, or a future request-policy plugin can all need the same answer: how many tokens does the durable request consume? Keeping that fold inside `dsh-compact-basic` duplicates replay logic, makes measurement unavailable without compaction, and encourages callers to reuse stale accounting.
|
||||
|
||||
Provider usage is not a complete answer. It describes one successful call under one exact request envelope, while the current surface can grow, shrink, or be replaced afterward. Sessions also switch providers and models, old logs can lack chunk provenance, and usage fields separate input, cache-read, cache-write, output, and reasoning counts. A useful service therefore combines the latest exact anchor with conservative heuristic repricing and exposes the log revision consumed by each result.
|
||||
|
||||
## Decision
|
||||
|
||||
### One concrete LLM-family service
|
||||
|
||||
`@deepseek-ai/dsh-token-meter` is one concrete package under `packages/llm/` and registers `ctx.tokenMeter`. It is not split into an interface and backend before a second implementation exists. `TokenMeterService` itself exposes `measure(session, requestHeader?)` and `estimateMessage(message)`; consumers call the singleton service directly.
|
||||
|
||||
The service has no configuration. Estimation uses a fixed four-characters-per-token heuristic plus structural overhead. There are no model profiles, capacity settings, density settings, tokenizer backends, or language-specific strategies. Exact provider/model capacity is a separate adapter-owned query, as specified by the [routed model context and compaction policy Agent Note](2026-07-20-routed-model-context-and-compaction-policy.md).
|
||||
|
||||
### Per-session replay folds
|
||||
|
||||
Each session owns one isolated incremental fold. Active folds advance from `session/event`; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical full request-header snapshots, step boundaries, surface appends and replacements, assistant usage, and assistant-chunk provenance. A malformed next event fails transactionally and remains unread rather than partially mutating state.
|
||||
|
||||
`measure(session, requestHeader?)` synchronizes the fold once and returns scalar pressure together with positional per-node prices. `totalTokens` remains request-and-response pressure; `surfaceTokens` is the surface-only heuristic total and equals the sum of `nodes[].tokens`. A `requestHeader` override changes pressure pricing only, while the surface fields always describe the current session. `estimateMessage(message)` applies the fixed heuristic without session state. Each result is one detached, deeply immutable snapshot carrying one `logRevision`. Every measurement clones the current nodes and is therefore O(surface).
|
||||
|
||||
Provider usage is reused only when the measured canonical request envelope equals the latest successful-call anchor. Any provider, model, system, prefix, tool, or call-config change causes complete heuristic repricing. Surface changes remain a signed delta from a matching anchor, including negative values after a shrinking replacement. A later successful request replaces the earlier anchor, including across provider or model switches.
|
||||
|
||||
Usage sums the disjoint input, cache-read, cache-write, and output buckets. Reasoning is not added a second time. Every successful model call records an `assistant/message`, including content-less and max-token calls, with its exact earlier chunk seqs. An explicit empty provenance list means a known empty provider stream; absent legacy provenance conservatively treats the durable assistant output as provider output.
|
||||
|
||||
### Compact-basic consumes, but does not own, measurement
|
||||
|
||||
`dsh-compact-basic` requires `ctx.tokenMeter`; `CompactService` gains no token methods or types. Configuration, the region transaction, and summarization stay in separate modules; the service registers automatic listeners itself, while `summarize()` remains its sole subclass hook. The singleton meter consistently prices pressure, retention, shadowed content, provenance, and non-shrinking-summary rejection.
|
||||
|
||||
Automatic compaction uses one unified measurement for each threshold-and-retention decision. The region transaction measures after appending its durable `compact/start` lock and again after asynchronous summarization; any intervening durable append changes `logRevision` and prevents replacement.
|
||||
|
||||
Compact policy has service-wide defaults: threshold ratio `0.8`, retained-tail ratio `0.16`, `summarizationProvider: ''`, `summarizationModel: ''`, `maxTokens: 8192`, `compactionRetries: 1`, `maxOverflowRetries: 1`, and `auto: true`. Top-level fields apply to every routed target; exact provider/model entries in `modelPolicies` partially override them. Pressure scales ratios against capacity resolved from the owning adapter, and `retainTokens` may replace `retainRatio`; retention must remain below the resulting threshold. The summarization provider and model must both be set or both be empty; an empty pair resolves the latest logged request target, then the `AgentOptions` pair.
|
||||
|
||||
Automatic pressure runs at `agent/post-step` and measures the canonical durable envelope produced under the provider/model actually selected by `agent/request`. A headerless session has no completed routed request to assess and produces no work; any routed target can use the singleton estimator. Canonical overflow recovery uses the same measurement for forced range selection and retries only after a proven surface replacement.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests cover fixed estimation, envelope invalidation and anchor replacement, replay boundaries, immutable snapshots, routed pressure, convergence, overflow generation proof, and rollback. A real Loader/Include fixture verifies the zero-config token-meter and compact-basic load path in dependency order.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep estimation inside `CompactService`** — rejected because measurement has consumers and replay semantics independent of compaction; it would also force every compactor to expose the same unrelated API.
|
||||
- **Split a token-meter interface from a heuristic backend immediately** — rejected because only one implementation exists. One concrete service preserves the future seam without speculative packages or configuration.
|
||||
- **Put model-keyed windows and density profiles in the meter** — rejected because replay estimation does not own model routing or capacity facts. The route-owning adapter exposes capacity, while compact-basic owns the consumer-specific threshold and retention policy.
|
||||
- **Keep separate scalar and surface measurements** — rejected because callers would need two reads and revision matching for one decision. A scalar-only read could avoid cloning nodes below threshold, but the split API introduces a caller-side race window; the unified snapshot accepts O(surface) cloning in exchange for coherence.
|
||||
- **Treat provider usage as portable between envelopes** — rejected because model, tools, prefixes, and call config are request facts. Mismatch reprices the whole current request.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Token pressure has one replay-aware owner that compaction and future plugins can share.
|
||||
- The default makes the meter a zero-config composition entry; deployments configure capacity on each route-owning adapter and optional policy overrides on compact-basic.
|
||||
- Fixed heuristic pricing remains an estimate of provider behavior and is not an exact tokenizer or request serializer.
|
||||
- Every measurement clones the current positional surface and therefore costs O(surface), including pressure checks that finish below threshold.
|
||||
- Measurements fail loudly on malformed durable boundaries. This turns corrupted replay into a named integration failure instead of silently drifting pressure.
|
||||
- Post-step pressure reads the exact logged routing/tools/prefix boundary; provider overflow classification remains the adapter-maintained backstop for requests rejected before a successful usage anchor.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Agent Note: 回放式 token 计量服务
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-15-replay-token-meter-service.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
上下文压力并不只对压缩有用。压缩后端、溢出保护或未来的请求策略插件都可能需要回答同一个问题:持久请求消耗了多少 token?如果把该折叠逻辑留在 `dsh-compact-basic` 内部,就会重复实现回放逻辑,使未加载压缩的调用方无法使用计量,并诱使调用方复用陈旧的核算结果。
|
||||
|
||||
提供方 usage 也不是完整答案。它只描述某个精确请求信封下的一次成功调用,而当前表层之后还可能增长、缩小或被替换。会话也可能切换提供方与模型,旧日志可能缺少分片来源,usage 字段还会分别报告输入、缓存读取、缓存写入、输出与推理计数。因此,可用的服务必须把最新精确锚点与保守的启发式重新定价结合起来,并公开每个结果已经消费的日志修订号。
|
||||
|
||||
## 决策
|
||||
|
||||
### 一个具体的 LLM 家族服务
|
||||
|
||||
`@deepseek-ai/dsh-token-meter` 是 `packages/llm/` 下的单个具体包,并注册 `ctx.tokenMeter`。在第二种实现出现之前,它不会被拆成接口与后端。`TokenMeterService` 本身公开 `measure(session, requestHeader?)` 与 `estimateMessage(message)`;消费方直接调用这个单例服务。
|
||||
|
||||
服务没有配置。估算采用固定的每 token 四个字符启发式规则,并加上结构开销。服务不提供模型 profile、容量设置、密度设置、分词器后端或语言专用策略。精确提供方/模型容量由独立的适配器查询拥有,具体见[路由模型上下文与压缩策略 Agent Note](2026-07-20-routed-model-context-and-compaction-policy.md)。
|
||||
|
||||
### 逐会话回放折叠
|
||||
|
||||
每个会话都有一个隔离的增量折叠。活跃折叠通过 `session/event` 前进;每次读取都会追到持久日志尾部,因此监听器顺序、种子会话与服务重载不会改变答案。折叠跟踪规范的完整请求头快照、步骤边界、表层追加与替换、assistant usage,以及 assistant 分片来源。下一个畸形事件会以事务方式失败并保持未读,不会让状态只修改一半。
|
||||
|
||||
`measure(session, requestHeader?)` 只同步一次折叠,并在返回标量压力的同时给出逐位置节点价格。`totalTokens` 仍表示请求与响应压力;`surfaceTokens` 是仅针对表层的启发式总量,并等于 `nodes[].tokens` 之和。`requestHeader` 覆盖只改变压力定价,表层字段始终描述当前会话。`estimateMessage(message)` 不依赖会话状态,直接应用固定启发式规则。每个结果都是一个分离且深度不可变的快照,只携带一个 `logRevision`。每次计量都会复制当前节点,因此成本为 O(surface)。
|
||||
|
||||
只有当待计量的规范请求信封等于最近一次成功调用的锚点时,服务才复用提供方 usage。提供方、模型、系统提示词、前缀、工具或调用配置任一变化都会触发完整的启发式重新定价。表层变化相对匹配锚点保留有符号增量,包括缩小替换后的负值。后续成功请求会替换先前锚点,提供方或模型切换时也一样。
|
||||
|
||||
Usage 会对互不重叠的输入、缓存读取、缓存写入与输出 bucket 求和,不会再次加入推理计数。每次成功模型调用都会记录 `assistant/message`,包括无内容调用与达到 token 上限的调用,并带上精确的更早分片 seq。显式空来源列表表示已知为空的提供方流;旧日志中缺失的来源则保守地把持久 assistant 输出视为提供方输出。
|
||||
|
||||
### compact-basic 消费计量,但不拥有计量
|
||||
|
||||
`dsh-compact-basic` 要求 `ctx.tokenMeter`;`CompactService` 不增加 token 方法或类型。配置、区域事务与摘要器分别保留在独立模块中,服务自身注册自动监听器,而 `summarize()` 仍是唯一的子类 hook。单例计量器一致用于压力、保留、被遮蔽内容、来源以及非缩小摘要拒绝的定价。
|
||||
|
||||
自动压缩的每次阈值与保留联合决策只使用一次统一计量。区域事务先追加持久 `compact/start` 锁,再执行一次计量,并在异步摘要完成后再次计量;期间任何持久追加都会改变 `logRevision`,从而阻止替换。
|
||||
|
||||
压缩策略采用服务级默认值:阈值比例 `0.8`、保留尾部比例 `0.16`、`summarizationProvider: ''`、`summarizationModel: ''`、`maxTokens: 8192`、`compactionRetries: 1`、`maxOverflowRetries: 1` 与 `auto: true`。顶层字段适用于每个路由目标;`modelPolicies` 中的精确提供方/模型项可以部分覆盖这些字段。压力检查根据所属适配器解析的容量缩放比例,`retainTokens` 可以替代 `retainRatio`;保留值必须小于最终阈值。摘要提供方与模型必须同时设置或同时为空;空组合先解析最近记录的请求目标,再使用 `AgentOptions` 中的组合。
|
||||
|
||||
自动压力检查运行在 `agent/post-step`,并计量 `agent/request` 实际所选提供方/模型产生的规范持久信封。没有请求头的会话尚无已完成的路由请求可供判断,因此不执行工作;任意路由目标都可使用这个单例估算器。规范化溢出恢复使用同一计量结果强制选择范围,并且只有在表层替换得到证明后才重试。
|
||||
|
||||
## 测试
|
||||
|
||||
单元测试覆盖固定估算、信封失效与锚点替换、回放边界、不可变快照、已路由压力、收敛、溢出 generation 证明与回滚。真实 Loader/Include fixture 验证零配置 token-meter 与 compact-basic 按依赖顺序加载的路径。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **把估算保留在 `CompactService` 内**——不予采纳,因为计量拥有独立于压缩的消费方与回放语义;它还会强迫每个压缩器暴露同一套无关 API。
|
||||
- **立即把 token meter 拆成接口与启发式后端**——不予采纳,因为目前只有一种实现。单个具体服务保留未来接缝,同时避免推测性的包与配置。
|
||||
- **把模型键控窗口与密度 profile 放进 meter**——不予采纳,因为回放估算不拥有模型路由或容量事实。路由所属适配器公开容量,compact-basic 则拥有消费方专用的阈值与保留策略。
|
||||
- **保留独立的标量与表层计量**——不予采纳,因为消费方必须为一次决策执行两次读取并匹配修订号。仅读取标量可以避免在低于阈值时复制节点,但拆分 API 会在消费方引入竞态窗口;统一快照接受 O(surface) 复制成本,以换取结果一致性。
|
||||
- **在不同信封之间移用提供方 usage**——不予采纳,因为模型、工具、前缀与调用配置都是请求事实。不匹配时会重新定价完整当前请求。
|
||||
|
||||
## 后果
|
||||
|
||||
- Token 压力拥有一个可供压缩与未来插件共享的回放感知所有者。
|
||||
- 默认值让 meter 成为零配置组合项;部署在各个路由所属适配器上配置容量,并在 compact-basic 上配置可选策略覆盖。
|
||||
- 固定启发式定价仍然只是提供方行为的估计,并不是精确分词器或请求序列化器。
|
||||
- 每次计量都会复制当前的位置表层,因此成本为 O(surface),低于阈值即可结束的压力检查也不例外。
|
||||
- 遇到畸形持久边界时,计量会明确失败。这会把损坏的回放转化为具名集成错误,而不是让压力静默漂移。
|
||||
- post-step 压力检查读取精确记录的路由、工具与前缀边界;对于在成功 usage 锚点出现前就被拒绝的请求,提供方溢出分类仍是由适配器维护的兜底路径。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-16-explicit-turn-cancellation.md: 7ac743221084e663294954bfd048ba7ef1114f60
|
||||
2026-07-16-explicit-turn-cancellation.zh.md: 3dca6339787ebef749c0d6a15609376ede994a97
|
||||
@@ -0,0 +1,55 @@
|
||||
# Agent Note: Explicit turn cancellation capability
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-16-explicit-turn-cancellation.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Cancellation is a control capability with a shorter lifetime than an Agent driver. A free-form string cannot distinguish callers exhaustively, and a step-local controller cannot interrupt prompt submission, prompt assembly, continuation, or terminal turn policy. Storing `Error`, `AbortSignal.reason`, or backend-private objects would also expose unstable runtime details to durable replay.
|
||||
|
||||
The [initiating Agent scope decision](2026-07-15-agent-initiator-scope.md) intentionally carries only the exact Agent through AsyncLocalStorage. Adding turn, step, or signal state to that driver-lifetime boundary would make stale asynchronous descendants appear to retain authority over later turns. Cancellation therefore needs one turn owner and explicit propagation without creating another ambient context or public turn wrapper.
|
||||
|
||||
## Decision
|
||||
|
||||
Agent owns the runtime-only `AgentCancelCause` union `{ kind: 'user' } | { kind: 'parent' }`; `agent.cancel()` defaults to `user`. TypeScript enforces that vocabulary at this typed same-process seam, with no runtime validator, fallback, or special compatibility contract for untyped callers. An active `TurnCancellation` copies the typed discriminant into a fresh frozen signal reason; idle cancellation has no holder to mutate and does not arm later work.
|
||||
|
||||
An interrupted live turn ends with the coarse durable `{ kind: 'aborted' }` outcome. The terminal event records what happened to the turn, while the runtime signal identifies who requested cancellation; it does not duplicate `user` or `parent` into replay. Session seed/load rejects legacy aborted records with a reason or any other extra field, so replay cannot reintroduce caller-owned cancellation detail. The process-local `agent/cancel-requested` notification is not durable; a future audit requirement uses a separate durable control-request event so a request and its eventual outcome remain distinct. Durable events contain no stack, signal, error object, free-form cancellation text, or backend-private detail.
|
||||
|
||||
AgentLoop privately owns one `TurnCancellation` per prospective turn. It installs the holder before notifying `agent/status = running`, retains its single `AbortController` through prompt processing, prompt assembly, every step, model and tool execution, continuation, and `agent/turn-stop`, then clears the exact holder immediately before publishing `turn/end`. Terminal event observers and the following durability flush therefore cannot cancel already-completed turn work even though driver status may remain `running` until the flush settles. Every participating method, event, and request value receives that same explicit signal; the next turn receives a fresh signal.
|
||||
|
||||
The driver keeps only a cause-less pre-run marker for queued work cancelled before a turn is claimed. An effective `cancel()` emits the observe-only `agent/cancel-requested` notification with its resolved typed cause before clearing queued and steering work or aborting the holder; notification failures cannot veto the stop, and an idle call emits nothing. Work synchronously queued by a notification observer is included in that clear, while work queued by a later signal abort observer belongs to the next turn. If a `running` listener synchronously cancels old work and sends a replacement, the driver discards the aborted holder and creates a fresh one for the replacement. Repeated cancellation is first-wins for the active holder, while later calls may still clear newly queued pending work.
|
||||
|
||||
The explicit event signatures keep their positional form and place `signal` immediately before a waterfall's final `next`. Prompt submission, request configuration, step-result processing, continuation, and terminal stop join the pre-existing explicit signal seams for pre-step, session prefix, model generation, tool execution, approval, and subagent or workflow requests. Hook bridges must also supply `RunHookOptions.signal`, so a turn cancellation reaches the bash executor's process-group kill and join boundary. `SystemPrompt.assemble()` carries `signal?: AbortSignal` in `AssembleContext` because that object is an explicit request value that can also represent signal-less assembly outside a turn. Listeners may cooperate with the signal but must not retain it to control another turn.
|
||||
|
||||
`ctx.agents` continues to carry only the initiating Agent. Ambient Agent presence does not imply liveness, a current turn, or cancellation authority, and `agentInterruptReasonOf(signal)` reads only its explicit argument. Concurrent Agents isolate both their initiator identities and their turn signals; a child driver shadows the parent initiator while its parent request signal still travels through the subagent seam.
|
||||
|
||||
Agent disposal requests the runtime-only `{ kind: 'disposed' }` interruption on the active holder. If cancellation already won the controller reason, the reason cannot be rewritten, so terminal classification first checks lifecycle state: disposed wins, then a supported `user` or `parent` cause becomes the coarse aborted outcome, and unrelated exceptions retain the existing error path. ACP cancellation maps to `user`; in-process spawn and fork propagation map to `parent`. Remote ACP subagents retain their existing wire protocol.
|
||||
|
||||
Cancellation remains cooperative. The loop checks interruption before and after awaited boundaries but does not use `Promise.race` to abandon an in-process listener, adapter, or tool Promise. Work that ignores the signal must settle before `whenIdle()`, handle disposal, and scope teardown report quiescence.
|
||||
|
||||
## Verification
|
||||
|
||||
Contract tests verify the typed caller union, frozen detachment, default and first-wins behavior, the coarse Session JSON round trip and legacy-record rejection, ACP `user`, in-process subagent `parent`, and disposal precedence. Loop tests make cooperative listeners wait on the signal at prompt submission, system-prompt assembly, session prefix, pre-step, request, model stream, step result, tool execution, continuation, and terminal stop; they assert one signal within a turn, a fresh signal across turns, and no cancellation authority during terminal publication or a blocked durability flush. A real hook bridge test cancels and reaps a blocked prompt hook before idle.
|
||||
|
||||
Initiator-scope tests assert that every hook still observes the exact Agent and no ambient turn signal, concurrent Agents retain independent identities and signals, and a nested child driver shadows only identity. Race tests cover idle cancellation, pre-run cancellation, replacement submission from a `running` listener, repeated cancellation, and cancel-versus-dispose quiescence.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Store the signal in ALS.** ALS follows asynchronous descendants for the entire driver lifetime, while cancellation authority ends with one turn. A leaked callback could observe a stale signal or require mutable ambient state, so the initiator scope continues to carry only the Agent and control remains explicit.
|
||||
|
||||
**Persist a free-form string reason.** Strings admit spelling drift, prevent exhaustive switching, and encourage consumers to parse presentation text. The runtime uses a closed discriminated union, while the terminal record needs only the stable aborted outcome.
|
||||
|
||||
**Persist the typed caller cause in `turn/end`.** No production replay, UI, ACP, telemetry, or workflow consumer distinguishes `user` from `parent`. Copying the request source into the terminal result would conflate two facts and add Session-specific validation without a consumer; a future audit surface can record a separate cancellation-request event.
|
||||
|
||||
**Define speculative `superseded`, `timeout`, and `shutdown` variants now.** No current Agent cancellation producer implements those semantics. `shutdown` is already lifecycle disposal, and timeout or supersession should enter the union only with an owning policy and unique terminal meaning.
|
||||
|
||||
**Expose public turn or step context wrappers.** Existing positional seams already identify Agent, turn, and step. A wrapper would widen every API, duplicate ownership, and tempt callers to treat a captured object as durable authority.
|
||||
|
||||
**Abandon uncooperative work after a grace period.** Returning idle while same-process work still runs breaks teardown and resource-ownership guarantees. Hard termination requires a worker or process isolation boundary and is outside this control seam.
|
||||
|
||||
## Consequences
|
||||
|
||||
Cancellation has one runtime owner, one signal per live turn, and one typed runtime caller vocabulary. Session retains the coarse `aborted` outcome that its consumers actually use, rejects reason-bearing legacy forms, and stays isolated from runtime objects. Cooperative cancellation reaches every asynchronous turn seam, including work before the first step and after the last one, while terminal publication and persistence remain outside its authority.
|
||||
|
||||
The explicit signal adds parameters to several public events and requires plugins to forward cancellation deliberately. This is intentional: authority is visible at the call boundary, lifetime matches the turn, and stale ambient descendants cannot acquire control. Uncooperative in-process work may delay cancellation, but the reported quiescent state remains truthful.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Agent Note:显式轮次取消能力
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-16-explicit-turn-cancellation.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
取消是一种生命周期短于 Agent(智能体)驱动器的控制能力。自由文本字符串无法穷尽地区分调用方,步骤级控制器也无法中断提示词提交、提示词组装、继续决策或轮次终止策略。持久化 `Error`、`AbortSignal.reason` 或后端私有对象还会向持久化回放暴露不稳定的运行时细节。
|
||||
|
||||
[发起 Agent 作用域决策](2026-07-15-agent-initiator-scope.md)有意让 AsyncLocalStorage 只携带同一个 Agent。若把轮次、步骤或 signal 状态加入这个与驱动器同生命周期的边界,陈旧的异步后代就会看似仍对后续轮次拥有权限。因此,取消需要一个轮次归属方并显式传播,且不创建另一套环境上下文或公开的轮次包装层。
|
||||
|
||||
## 决策
|
||||
|
||||
Agent 拥有仅用于运行时的 `AgentCancelCause` 联合类型 `{ kind: 'user' } | { kind: 'parent' }`;`agent.cancel()` 默认使用 `user`。TypeScript 在这份类型化同进程契约中强制执行该词汇,不提供运行时校验器、后备行为,也不为无类型调用方提供特殊兼容性契约。活跃的 `TurnCancellation` 会把类型化判别字段复制为一个全新且已冻结的 signal 原因;空闲状态下没有可修改的持有者,也不会让后续工作预先进入取消状态。
|
||||
|
||||
正在运行的轮次被中断后,以粗粒度的持久化结果 `{ kind: 'aborted' }` 结束。终态事件记录轮次发生了什么,运行时 signal 标识谁请求了取消;回放不会重复保存 `user` 或 `parent`。Session seed/load 会拒绝携带取消原因或任何其他额外字段的旧式中止记录,因此回放无法重新引入由调用方持有的取消细节。仅限进程内的 `agent/cancel-requested` 通知不会持久化;未来若有审计需求,应使用独立的持久化控制请求事件,让请求与最终结果保持为两项事实。持久化事件不包含调用栈、signal、错误对象、自由文本取消原因或后端私有细节。
|
||||
|
||||
AgentLoop 为每个待启动轮次私有地持有一个 `TurnCancellation`。它在通知 `agent/status = running` 前安装该持有者,使其中唯一的 `AbortController` 持续覆盖提示词处理、提示词组装、每个步骤、模型与工具执行、继续决策和 `agent/turn-stop`;随后在发布 `turn/end` 前立即清除所安装的那个持有者。因此,即使驱动器状态可能在持久化刷新结算前保持 `running`,终态事件观察者及其后的持久化刷新也无法取消已完成的轮次工作。所有参与的方法、事件和请求值都会收到同一个显式 signal;下一个轮次会收到全新的 signal。
|
||||
|
||||
对于轮次被认领前已取消的排队工作,驱动器只保留一个不携带取消原因的运行前标记。实际生效的 `cancel()` 会先发出仅供观察的 `agent/cancel-requested` 通知并携带最终确定的类型化取消原因,然后才清除排队工作和 steering(中途引导)工作或中止持有者;通知失败不能阻止此次停止,空闲状态下调用则不发出任何通知。通知观察者同步加入队列的工作也会被这次清除,而稍后由 signal 中止观察者加入队列的工作属于下一个轮次。若 `running` 监听器同步取消旧工作并发送替代提示词,驱动器会丢弃已中止的持有者,并为替代提示词创建全新的持有者。同一活跃持有者上的重复取消遵循首次请求优先,后续调用仍可清除新入队的待处理工作。
|
||||
|
||||
显式事件签名保留位置参数形式,并把 `signal` 放在 waterfall(瀑布式事件)的最后一个参数 `next` 之前。提示词提交、请求配置、步骤结果处理、继续决策和终止停止加入已有的步骤前处理、会话前缀、模型生成、工具执行、审批以及 subagent 或工作流请求的显式 signal seam。钩子桥接器也必须提供 `RunHookOptions.signal`,使轮次取消能够到达 Bash 执行器终止进程组并等待其退出的边界。`SystemPrompt.assemble()` 在 `AssembleContext` 中携带 `signal?: AbortSignal`,因为该对象是显式请求值,也可表示轮次之外不携带 signal 的组装。监听器可以配合该 signal 取消,但不得保留它来控制其他轮次。
|
||||
|
||||
`ctx.agents` 仍只携带发起 Agent。环境中的 Agent 并不代表存活、当前轮次或取消权限,`agentInterruptReasonOf(signal)` 也只读取其显式参数。并发 Agent 会同时隔离各自的发起方身份和轮次 signal;子驱动会遮蔽父发起方,而父请求 signal 仍通过 subagent seam 传递。
|
||||
|
||||
Agent dispose(资源释放)会在活跃持有者上请求仅用于运行时的 `{ kind: 'disposed' }` 中断。若取消已经先占用控制器的中断原因,该原因便无法改写,因此终态分类会先检查生命周期状态:资源释放结果优先,之后受支持的 `user` 或 `parent` 取消原因形成粗粒度的中止结果,其他异常保留现有错误路径。ACP(Agent Client Protocol)取消映射为 `user`;进程内 spawn 和 fork 的传播映射为 `parent`。远程 ACP subagent 保持现有协议。
|
||||
|
||||
取消仍然是协作式的。AgentLoop 会在异步等待边界前后检查中断,但不会用 `Promise.race` 放弃进程内监听器、适配器或工具 Promise。忽略 signal 的工作必须真正结算,`whenIdle()`、句柄 dispose 和作用域清理才会报告静止状态。
|
||||
|
||||
## 验证
|
||||
|
||||
契约测试验证类型化调用方联合类型、冻结且与调用方分离、默认行为与首次请求优先行为、粗粒度的会话 JSON 往返与旧式记录拒绝、ACP `user`、进程内 subagent `parent` 以及 dispose 优先级。AgentLoop 测试让协作式监听器在提示词提交、系统提示词组装、会话前缀、步骤前处理、请求、模型流、步骤结果、工具执行、继续决策和终止停止处等待 signal;并断言同一轮次使用一个 signal,不同轮次使用全新的 signal,终态发布期间和持久化刷新受阻期间不存在取消权限。真实钩子桥接器测试会在报告空闲状态前取消并回收受阻的提示词钩子。
|
||||
|
||||
发起方作用域测试断言所有钩子仍观察到同一个 Agent 且没有环境中的轮次 signal,并发 Agent 保持独立的身份与 signal,嵌套子驱动只遮蔽身份。竞态测试覆盖空闲状态取消、运行前取消、从 `running` 监听器提交替代提示词、重复取消以及取消与 dispose 竞争下的静止状态。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**把 signal 存入 ALS。** ALS 会在整个驱动器生命周期内跟随异步后代,而取消权限在一个轮次结束时就已终止。泄漏的回调可能观察到陈旧 signal,或者迫使实现使用可变的环境状态,因此发起方作用域继续只携带 Agent,控制能力继续显式传递。
|
||||
|
||||
**持久化自由文本原因。** 字符串允许拼写漂移、阻碍穷尽分支判断,还会鼓励消费方解析展示文本。运行时使用封闭的可辨识联合类型,终态记录只需要稳定的中止结果。
|
||||
|
||||
**在 `turn/end` 中持久化类型化调用方取消原因。** 当前没有任何生产环境中的回放、UI、ACP、遥测或工作流消费方区分 `user` 与 `parent`。把请求来源复制到终态结果会混淆两项事实,还会在没有消费方的情况下引入会话特有校验;未来的审计接口可以记录独立的取消请求事件。
|
||||
|
||||
**现在就定义推测性的 `superseded`、`timeout` 和 `shutdown` 变体。** 当前没有 Agent 取消生产方实现这些语义。`shutdown` 已经属于生命周期 dispose;超时或替代只有在拥有明确归属策略和唯一终态含义时才应进入联合类型。
|
||||
|
||||
**公开轮次或步骤上下文包装类型。** 现有位置参数 seam 已经标识 Agent、轮次和步骤。包装类型会加宽所有 API、重复归属,并诱导调用方把捕获的对象当成持久权限。
|
||||
|
||||
**在宽限期后放弃不协作的工作。** 同进程工作仍在运行时就报告空闲状态,会破坏资源清理与资源归属保证。硬终止需要 worker 或进程隔离边界,不属于该控制 seam。
|
||||
|
||||
## 后果
|
||||
|
||||
取消拥有一个运行时归属方、每个活跃轮次一个 signal,以及一套类型化的运行时调用方词汇。会话保留其消费方实际使用的粗粒度 `aborted` 结果,拒绝携带原因的旧式形式,并与运行时对象保持隔离。协作式取消覆盖每个异步轮次 seam,包括第一个步骤之前和最后一个步骤之后的工作,而终态发布和持久化仍在其权限范围之外。
|
||||
|
||||
显式 signal 会给多个公开事件增加参数,并要求插件有意识地转发取消。这是有意设计:权限在调用边界可见,生命周期与轮次匹配,陈旧的环境异步后代无法获得控制能力。不协作的进程内工作可能延迟取消,但所报告的静止状态仍然真实。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-cooperative-tool-cancellation.md: 559012f10d41963698cc932727125de1b9ccfef7
|
||||
2026-07-19-cooperative-tool-cancellation.zh.md: 6af8e57349bba026ab22f257014c084c5c3c3f54
|
||||
@@ -0,0 +1,73 @@
|
||||
# Agent Note: Cooperative tool cancellation at the registry boundary
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-19-cooperative-tool-cancellation.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Every typed tool invocation needs a caller-owned cancellation signal. An optional `ToolExecutionInput.signal` lets direct callers omit ownership, makes `exec.signal` optional in every tool body, and encourages registry fallbacks that cannot represent the caller's actual lifetime.
|
||||
|
||||
The pipeline also has different mutability needs at different stages. Tool implementations, pre-policy, post-policy, and result observers only borrow cancellation state, while an around-dispatch wrapper must temporarily replace the signal to add a deadline or another lexical cancellation scope. One mutable public type either grants mutation too broadly or prevents that composition.
|
||||
|
||||
Cancellation can arrive before policy, during approval, inside an around-dispatch wait, after a tool body starts, or while post-policy waits. One undifferentiated `ABORTED` result cannot tell durable consumers whether body side effects were possible. Racing a tool promise against cancellation is not a safe fallback because abandoned same-process work continues after the registry reports completion.
|
||||
|
||||
## Decision
|
||||
|
||||
`ToolExecutionInput.signal` is a required readonly `AbortSignal`. `ToolExecution.signal` and `ToolRunContext.signal` are therefore required and readonly as well. Every typed caller supplies the signal it owns; the registry provides no overload, default controller, never-abort sentinel, or convenience execution path.
|
||||
|
||||
`ToolDefinition.execute(args, exec)` keeps its existing signature. `defineTool()` contextually types `exec.signal` as a required `AbortSignal`, so every registered TypeScript tool can observe or forward cancellation without a cast. First-party direct callers and nested Code Mode dispatches pass their current operation signal explicitly.
|
||||
|
||||
The registry trusts this typed same-process contract. It does not perform runtime `AbortSignal` validation or add hostile-input tests for an omitted or malformed signal. Validation remains at parser/config, model/tool JSON, durable/file, worker, process, and wire boundaries; untyped JavaScript that violates the TypeScript interface has no compatibility contract.
|
||||
|
||||
### Mutability follows the pipeline stage
|
||||
|
||||
`ToolDispatchExecution` is identical to `ToolExecution` except that its required `signal` is mutable. Only the `tools/execute` waterfall receives this type. Pre-policy, post-policy, result observers, guards, and tool implementations receive readonly views of a private registry-owned mutable run object.
|
||||
|
||||
An around-dispatch wrapper may replace `exec.signal` for its delegated lifetime but cannot typefully delete it or assign `undefined`. The registry captures the required caller signal outside that mutable object, fuses every wrapper replacement with the caller signal immediately before body invocation, removes dispatch-scoped listeners after settlement, and restores the required upstream signal unconditionally.
|
||||
|
||||
### Cancellation codes record whether dispatch occurred
|
||||
|
||||
`dsh-tools` exports `TOOL_ABORTED = 'ABORTED'` and `TOOL_ABORTED_BEFORE_DISPATCH = 'ABORTED_BEFORE_DISPATCH'`. The registry records body invocation immediately before calling `ToolDefinition.execute()`.
|
||||
|
||||
`ABORTED_BEFORE_DISPATCH` carries `{ name: 'AbortError' }` and model text `Error: tool call aborted before dispatch`. It applies whenever cancellation prevents body invocation, including pre-aborted entry, cancellation during pre-policy or approval, an aborted wrapper signal, a wrapper success overtaken by caller cancellation before delegation, and agent-loop siblings skipped after turn cancellation.
|
||||
|
||||
`ABORTED` carries model text `Error: tool call aborted` and applies only after the body was invoked, including cancellation while an around wrapper or post-policy listener waits after body completion. A denial, wrapper failure, tool failure, or post-policy failure remains more specific than generic cancellation. A timeout owned by timeout-policy remains `TOOL_TIMEOUT`, and contexts deferred before a successful outcome is replaced remain attached.
|
||||
|
||||
### Pre-aborted entry short-circuits after materialization
|
||||
|
||||
The registry first creates the call token and losslessly snapshots and freezes the arguments. A materialization failure wins even when the caller signal is already aborted. After successful materialization, a pre-aborted signal skips `tools/pre-execute`, approval, `tools/execute`, `tools/post-execute`, and the tool body, then publishes exactly one frozen authoritative `tools/result` with `ABORTED_BEFORE_DISPATCH`.
|
||||
|
||||
### Started work still reaches quiescence
|
||||
|
||||
Once a tool body starts, the registry awaits it. Cancellation reaches the body through the fused signal but never races or abandons its promise. A cooperative implementation stops or forwards cancellation and settles after its owned work reaches quiescence; an uncooperative same-process implementation can keep the registry pending indefinitely. Process, worker, network, and provider layers retain responsibility for their own termination mechanisms.
|
||||
|
||||
This decision requires cancellation at the tool invocation seam only. Making signals required on asynchronous capabilities reachable from tool bodies is a separate migration proposed in [Required cancellation through tool-reachable capability seams](../../proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.md).
|
||||
|
||||
## Verification
|
||||
|
||||
[`execution-signal-types.spec.ts`](../../../../packages/core/tools/tests/execution-signal-types.spec.ts) proves the required exact signal types, readonly observer and tool views, mutable-but-required around-dispatch view, and `defineTool()` inference. [`tools.spec.ts`](../../../../packages/core/tools/tests/tools.spec.ts) covers pre-aborted materialization, phase skipping, policy and wrapper races, body invocation classification, caller-signal fusion, error precedence, context retention, and quiescent drainage. [`tool-calls.spec.ts`](../../../../packages/core/agent-loop/tests/tool-calls.spec.ts) and [`contract-regressions.spec.ts`](../../../../packages/core/agent-loop/tests/contract-regressions.spec.ts) cover balanced durable results for undispatched siblings. [`code-mode.spec.ts`](../../../../packages/core/tools/tests/code-mode.spec.ts) and first-party integration suites cover explicit forwarding, while [`timeout-policy.spec.ts`](../../../../packages/timeout/timeout-policy/tests/timeout-policy.spec.ts) preserves timeout ownership.
|
||||
|
||||
No registry test can prove that arbitrary third-party same-process code observes the signal or stops in bounded time. Capability tests continue to prove cancellation and quiescence at the boundary that owns each side effect.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep the signal optional and synthesize a fallback.** Rejected because a registry-owned fallback has no caller lifetime to represent and preserves the exact omission the type should prevent.
|
||||
|
||||
**Validate `AbortSignal` at runtime.** Rejected because this is a typed same-process seam, not a serialization boundary. Runtime checks would duplicate the static contract without making cooperative use enforceable.
|
||||
|
||||
**Add `supportsCancellation` metadata, callback-arity checks, or signal-use linting.** Rejected because none proves that asynchronous work observes or correctly forwards cancellation. Availability is a type contract; behavior remains a tool and capability responsibility.
|
||||
|
||||
**Expose one mutable execution type to every stage.** Rejected because observers and tool implementations only borrow the signal. Stage-specific types make replacement possible only where the pipeline owns that operation.
|
||||
|
||||
**Forbid around wrappers from replacing the signal.** Rejected because deadlines and nested operational scopes need lexical derivation. Capturing and fusing the caller signal preserves composition without allowing detachment.
|
||||
|
||||
**Race the tool promise against cancellation.** Rejected because it reports completion while side effects may remain live, violating the [quiescent-disposal rule](../../../../docs/defensive-patterns.md#dispose-must-reach-quiescence-not-just-request-it).
|
||||
|
||||
## Consequences
|
||||
|
||||
- TypeScript rejects every `ToolExecutionInput` that omits `signal`, every tool or observer mutation of a readonly signal, and every around-dispatch attempt to remove the signal.
|
||||
- Durable consumers can distinguish calls whose body may have produced side effects (`ABORTED`) from calls that never entered the body (`ABORTED_BEFORE_DISPATCH`).
|
||||
- The change is intentionally breaking under the repository's pre-release stance; no compatibility overload or runtime fallback remains.
|
||||
- Cooperative tools stop promptly and reach quiescence; an implementation that ignores its signal remains observable as a pending call.
|
||||
- Downstream capability interfaces remain unchanged until the linked proposed Agent Note is accepted and implemented.
|
||||
@@ -0,0 +1,73 @@
|
||||
# Agent Note: 注册表边界上的协作式工具取消
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-19-cooperative-tool-cancellation.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
每次类型化工具调用都需要一个由调用方持有的取消信号。可选的 `ToolExecutionInput.signal` 允许直接调用方不承担所有权,使每个工具主体中的 `exec.signal` 都成为可选值,也会诱使注册表提供无法表达真实调用方生命周期的后备信号。
|
||||
|
||||
流水线各阶段对可变性的需求也不同。工具实现、前置策略、后置策略和结果观察者只借用取消状态,而环绕调度包装层必须临时替换信号,以加入截止时间或其他词法取消作用域。单一的可变公开类型要么把修改权限授予过多阶段,要么阻止这种组合。
|
||||
|
||||
取消可能发生在策略之前、审批期间、环绕调度等待期间、工具主体启动之后,或后置策略等待期间。单一的 `ABORTED` 结果无法让持久化结果的使用方判断工具主体是否可能产生过副作用。让工具 promise 与取消竞速也不是安全的后备方案,因为注册表报告完成后,被丢弃的同进程工作仍会继续运行。
|
||||
|
||||
## 决策
|
||||
|
||||
`ToolExecutionInput.signal` 是必填且只读的 `AbortSignal`,因此 `ToolExecution.signal` 和 `ToolRunContext.signal` 也都是必填且只读。每个类型化调用方显式提供自己持有的信号;注册表不提供重载、默认控制器、永不中止哨兵或便捷执行路径。
|
||||
|
||||
`ToolDefinition.execute(args, exec)` 保持现有签名。`defineTool()` 会把 `exec.signal` 上下文推断为必填的 `AbortSignal`,因此每个已注册的 TypeScript 工具都能在无需类型断言的情况下观察或转发取消。所有第一方直接调用方和 Code Mode 嵌套调度都会显式传入当前操作的信号。
|
||||
|
||||
注册表信任这份类型化同进程契约。它不在运行时校验 `AbortSignal`,也不为缺失或畸形信号添加敌意输入测试。校验仍位于解析器与配置、队列、模型与工具 JSON、持久化与文件、worker、进程和线协议边界;违反 TypeScript 接口的无类型 JavaScript 不享有兼容性契约。
|
||||
|
||||
### 可变性由流水线阶段决定
|
||||
|
||||
`ToolDispatchExecution` 与 `ToolExecution` 相同,唯一差异是其必填 `signal` 可修改。只有 `tools/execute` waterfall(瀑布式事件)接收这个类型。前置策略、后置策略、结果观察者、守卫和工具实现接收注册表私有可变运行对象的只读视图。
|
||||
|
||||
环绕调度包装层可以在委托期间替换 `exec.signal`,但无法通过类型系统删除它或赋值为 `undefined`。注册表在可变对象之外捕获必填的调用方信号,在工具主体调用前把每次包装层替换与调用方信号融合,在完成后移除仅属于本次调度的监听器,并无条件恢复必填的上游信号。
|
||||
|
||||
### 取消代码记录是否发生过调度
|
||||
|
||||
`dsh-tools` 导出 `TOOL_ABORTED = 'ABORTED'` 和 `TOOL_ABORTED_BEFORE_DISPATCH = 'ABORTED_BEFORE_DISPATCH'`。注册表在调用 `ToolDefinition.execute()` 的前一刻记录工具主体已经开始。
|
||||
|
||||
`ABORTED_BEFORE_DISPATCH` 携带 `{ name: 'AbortError' }` 和模型可见文本 `Error: tool call aborted before dispatch`。凡取消阻止工具主体调用时都使用该结果,包括进入时已中止、前置策略或审批期间取消、包装层信号已中止、包装层在委托前返回的成功结果被调用方取消抢先,以及轮次取消后 agent loop 跳过的同批调用。
|
||||
|
||||
`ABORTED` 携带模型可见文本 `Error: tool call aborted`,并且只在工具主体已经调用后使用,包括工具主体完成后环绕包装层或后置策略监听器等待期间发生的取消。拒绝、包装层失败、工具失败或后置策略失败比通用取消更具体。timeout-policy 自身拥有的超时仍为 `TOOL_TIMEOUT`,成功结果被取消替换前延后附加的上下文仍会保留。
|
||||
|
||||
### 进入时已中止会在物化后短路
|
||||
|
||||
注册表先创建调用 token,并对参数进行无损快照和冻结。即使调用方信号已经中止,参数物化失败仍优先返回。物化成功后,进入时已中止的信号会跳过 `tools/pre-execute`、审批、`tools/execute`、`tools/post-execute` 和工具主体,然后发布且只发布一次冻结的权威 `tools/result`,其代码为 `ABORTED_BEFORE_DISPATCH`。
|
||||
|
||||
### 已启动工作仍必须完全停稳
|
||||
|
||||
工具主体一旦启动,注册表就会等待它完成。取消通过融合信号到达工具主体,但注册表不会与其 promise 竞速或丢弃该 promise。协作式实现会停止自身工作或继续转发取消,并在所持有的工作完全停稳后完成;不协作的同进程实现可能让注册表无限期保持等待。进程、worker、网络和提供方层仍负责各自的终止机制。
|
||||
|
||||
这项决策只要求工具调用接缝携带取消信号。让工具主体可达的异步能力也必须接收信号,属于另一项迁移,见提议中的[工具可达能力接缝中的必填取消](../../proposed/architecture/2026-07-19-required-cancellation-through-tool-capability-seams.md)。
|
||||
|
||||
## 验证
|
||||
|
||||
[`execution-signal-types.spec.ts`](../../../../packages/core/tools/tests/execution-signal-types.spec.ts) 证明必填的精确信号类型、观察者与工具的只读视图、环绕调度可替换但不可删除的视图,以及 `defineTool()` 推断。[`tools.spec.ts`](../../../../packages/core/tools/tests/tools.spec.ts) 覆盖进入时已中止的物化与阶段跳过、策略和包装层竞态、工具主体调用分类、调用方信号融合、错误优先级、上下文保留和完全停稳。[`tool-calls.spec.ts`](../../../../packages/core/agent-loop/tests/tool-calls.spec.ts) 与 [`contract-regressions.spec.ts`](../../../../packages/core/agent-loop/tests/contract-regressions.spec.ts) 覆盖未调度同批调用的持久化配对结果。[`code-mode.spec.ts`](../../../../packages/core/tools/tests/code-mode.spec.ts) 和第一方集成测试覆盖显式转发,[`timeout-policy.spec.ts`](../../../../packages/timeout/timeout-policy/tests/timeout-policy.spec.ts) 保持超时归属。
|
||||
|
||||
任何注册表测试都无法证明任意第三方同进程代码会观察信号或在有界时间内停止。各能力的测试仍需在拥有相应副作用的边界证明取消与完全停稳。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**保留可选信号并生成后备值。** 不予采纳,因为注册表持有的后备信号不代表任何调用方生命周期,也会保留类型系统本应阻止的缺失情况。
|
||||
|
||||
**在运行时校验 `AbortSignal`。** 不予采纳,因为这是类型化同进程接缝,不是序列化边界。运行时检查只会重复静态契约,仍无法强制实现协作式使用信号。
|
||||
|
||||
**添加 `supportsCancellation` 元数据、回调参数数量检查或信号使用 lint。** 不予采纳,因为这些方法都无法证明异步工作会观察或正确转发取消。信号可用性属于类型契约;具体行为仍由工具和能力负责。
|
||||
|
||||
**向所有阶段公开同一个可变执行类型。** 不予采纳,因为观察者和工具实现只需要借用信号。按阶段划分类型可以把替换权限限制在流水线拥有该操作的位置。
|
||||
|
||||
**禁止环绕包装层替换信号。** 不予采纳,因为截止时间和嵌套运行时作用域需要词法派生信号。捕获并融合调用方信号既保留组合能力,也不允许切断调用方取消。
|
||||
|
||||
**让工具 promise 与取消竞速。** 不予采纳,因为这种方式会在副作用仍可能存活时报告完成,违反[资源释放必须完全停稳的规则](../../../../docs/defensive-patterns.md#dispose-must-reach-quiescence-not-just-request-it)。
|
||||
|
||||
## 后果
|
||||
|
||||
- TypeScript 会拒绝所有缺少 `signal` 的 `ToolExecutionInput`、工具或观察者对只读信号的修改,以及环绕调度删除信号的尝试。
|
||||
- 持久化结果的使用方可以区分工具主体可能产生过副作用的调用(`ABORTED`)和从未进入工具主体的调用(`ABORTED_BEFORE_DISPATCH`)。
|
||||
- 根据仓库的预发布原则,这项变更刻意保持破坏性;不保留兼容重载或运行时后备行为。
|
||||
- 协作式工具会及时停止并完全停稳;忽略信号的实现会表现为仍在等待的调用。
|
||||
- 下游能力接口保持不变,直到关联的提议 Agent Note 被接受并实现。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-package-invariant-runtime-contracts.md: 7d1fb1ad5a2e7563bdddffde1f49368b9f0c13f7
|
||||
2026-07-19-package-invariant-runtime-contracts.zh.md: 669eb02221aea4b0654497bb81327d725648dabe
|
||||
@@ -0,0 +1,78 @@
|
||||
# Agent Note: Meaningful package invariant contracts
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-19-package-invariant-runtime-contracts.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The package-owned invariant seam made publication and registration exhaustive, but its first generated baseline accepted empty installers. A follow-up then replaced those empties with generic assertions about plugin names, injections, effects, service methods, and fixed pure-library examples. Those assertions made every companion executable without making the system safer: TypeScript, Cordis startup, package tests, and module-load tests already enforce those shapes, while the invariant service should detect impossible runtime state.
|
||||
|
||||
A useful runtime invariant relates observations over time or across a mutable data structure. Examples include a terminal event without its start, an LLM delta for a block that is not open, or a durable result whose identity differs from its request. Merely confirming that a declared method exists, that a plugin has its expected name, or that a constant example still returns a known value is not such a relation.
|
||||
|
||||
Some packages genuinely own no continuously observable relation. Pure utilities, composition-only packages, thin adapters, binaries, and test-support packages may have important contracts, but those contracts are better enforced by types, load checks, focused unit tests, or integration tests. Requiring a synthetic runtime assertion for those packages would optimize for satisfying a gate instead of detecting corruption.
|
||||
|
||||
## Decision
|
||||
|
||||
### Registration is exhaustive; assertions must be meaningful
|
||||
|
||||
Every workspace package publishes a separately built `./invariant` companion and registers its exact npm package name. A companion does one of two things:
|
||||
|
||||
- installs a package-owned check over an event stream or relevant mutable data structure and reports violations through its bound `fail(message)` reporter; or
|
||||
- uses an empty installer whose declaration has an owner-specific `No runtime invariant:` comment explaining why the package has no plausible runtime relation to observe.
|
||||
|
||||
The empty form is an explicit architectural conclusion, not a generated placeholder. A future package change that introduces mutable state or an event protocol must replace the explanation with the corresponding check.
|
||||
|
||||
The central `dsh-invariants` service owns only configuration, registration uniqueness, child-fiber lifecycle, rollback, disposal, and package-attributed failure. It exposes no generic plugin-shape, service-shape, or startup-assertion helpers and imports no product package.
|
||||
|
||||
### Implemented checks
|
||||
|
||||
The current 103-package workspace has 21 executable companions and 82 justified empty companions.
|
||||
|
||||
| Owner | Runtime relationship |
|
||||
|---|---|
|
||||
| `dsh-session` | Strict sequence growth, turn/step enclosure, and same-step tool call/result pairing. |
|
||||
| `dsh-agent` | Non-repeating agent status and terminal disposal transitions. |
|
||||
| `dsh-scope` | Scoped-event carrier presence and routed-subject consistency. |
|
||||
| `dsh-agent-loop` | Explicitly marked, frozen loop request reconstruction from the session event log. |
|
||||
| `dsh-llm` | Stream block grammar, delta type/index matching, single usage, closed blocks, and terminal finish. |
|
||||
| `dsh-llm-retry` | Durable retry records identify the open turn's latest closed step, remain unique per step, increase monotonically, and stay within retry and non-negative timer bounds. |
|
||||
| `dsh-tools` | Monotonic pre/execute/post stages and immutable final execution/result snapshots. |
|
||||
| `dsh-system-prompt` | Authoritative assembly section, tool, and variable data constraints. |
|
||||
| `dsh-compact` | Compaction start/summary/end pairing, range endpoints, token counts, and successful-summary presence. |
|
||||
| `dsh-hook-protocol` | Hook invocation/result correlation, dialect, identity, and duration constraints. |
|
||||
| `dsh-sandbox-policy` | Durable `sandbox/mode` events use the closed sandbox-mode vocabulary. |
|
||||
| `dsh-fs` | Filesystem decision/observation events carry usable target and version identities. |
|
||||
| `dsh-goal` | Durable goal snapshots preserve source attribution, rendered content, revisions, lifecycle and timestamp relationships, and sequential admitted rounds. |
|
||||
| `dsh-goal-session` | Goal-sourced continuation messages match the prompt reconstructed from the preceding durable goal state. |
|
||||
| `dsh-subagent` | Provider add/remove and child start/end events preserve identity and pairing. |
|
||||
| `dsh-permission` | Durable permission decisions name a preset in the active permission table. |
|
||||
| `dsh-user-approval` | Approval asked/decided records pair by call and use valid outcomes and policies. |
|
||||
| `dsh-workflow` | Workflow and child-agent start/end events preserve run metadata, identity, outcome, count, and error relations. |
|
||||
| `dsh-tasks` | Current and terminal task snapshots preserve id/kind, owner, status, and timestamp relationships. |
|
||||
| `dsh-tool-todo` | Durable whole-list snapshots use unique trimmed items, closed statuses, and at most one active item. |
|
||||
| `dsh-time-context` | Plugin-attributed clock readings agree with the session's open turn, next pre-step position, and elapsed baseline; rendered time parses and does not postdate its event. |
|
||||
|
||||
Session-backed companions validate existing durable events when they load, using the prefix preceding each candidate where the relationship depends on event order. Other checks observe the authoritative live event boundary or mutable service result. Validation runs before publication where accepting an invalid event would otherwise commit bad state.
|
||||
|
||||
### Repository gate and tests
|
||||
|
||||
`verify-package-invariants` discovers every workspace package and enforces companion source, exact-name registration, named-only Loader shape, `./invariant` exports, publication files, dependencies, TypeScript references, and bundle entries. Its AST rule rejects generated markers, default exports, and unexplained empty installers. A non-empty installer must accept and use the failure reporter, and registration must pass that checked local `install` function. The gate deliberately does not infer semantic quality from method names or helper calls.
|
||||
|
||||
Vitest mounts `InvariantService` with `{ enabled: true }` for every package test topology and loads the owning companion. The invariant subpath path mapping resolves source companions instead of stale built output. Focused suites cover every executable companion's valid and invalid observations, and the exhaustive topology runs every source companion through the real Loader namespace normalization. An artifact gate stages each package's exact `npm pack` file inventory, imports its compiled `./invariant` self-reference under plain Node, and repeats that Loader-shape check, so an unpublished shared runtime chunk fails before release. Tests that synthesize event streams must produce a valid surrounding lifecycle unless the test is intentionally asserting a violation.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep generated empty companions.** Rejected because an unexplained placeholder can survive after a package gains a meaningful runtime relation.
|
||||
- **Require an assertion from every package.** Rejected because method-presence, plugin-shape, and fixed-example assertions duplicate stronger type, load, and unit-test contracts without checking runtime consistency.
|
||||
- **Keep generic shape helpers in the service.** Rejected because they blur compile-time API validation with runtime invariants and encourage centrally defined product assumptions.
|
||||
- **Move the product checks into the service.** Rejected because product vocabulary, dependencies, tests, and change ownership belong with the package that emits the data.
|
||||
- **Register companions implicitly from root entrypoints.** Rejected because composition order and optional service presence would create hidden effects.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Every package has visible ownership and publication wiring, but only packages with a plausible runtime relation add listeners or trace state.
|
||||
- Empty companions remain reviewable decisions with package-specific explanations and fail the gate if the explanation is removed.
|
||||
- Type declarations, Cordis loadability, plugin metadata, service method surfaces, and pure algebra remain covered by their owning compile, load, unit, or integration gates.
|
||||
- Runtime failures identify the owning npm package and point to an inconsistent observation rather than restating a required API shape.
|
||||
- The original selection, blocklist precedence, duplicate ownership, rollback, disposal, and HMR service contracts remain unchanged.
|
||||
@@ -0,0 +1,78 @@
|
||||
# Agent Note: 有意义的包不变量契约
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-19-package-invariant-runtime-contracts.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
包自有不变量接缝让发布和注册实现了全覆盖,但最初的生成基线允许空安装器。后续方案又用针对插件名称、注入、effect、服务方法和固定纯函数示例的通用断言替代这些空实现。这些断言虽然让每个 companion 都能执行,却没有提高系统安全性:TypeScript、Cordis 启动、包测试和模块加载测试已经约束这些形状,而不变量服务应当发现不可能出现的运行时状态。
|
||||
|
||||
有用的运行时不变量会关联时间上的多个观测,或关联可变数据结构中的多个部分。例如:终止事件没有对应的开始事件、LLM delta 指向未打开的 block,或持久化结果的身份与请求不同。仅确认声明的方法存在、插件名称符合预期,或常量示例仍返回已知值,都不属于这种关系。
|
||||
|
||||
有些包确实没有可持续观测的关系。纯工具、仅负责组合的包、薄适配器、可执行入口和测试支持包可能仍有重要契约,但类型检查、加载检查、聚焦单元测试或集成测试更适合执行这些契约。强迫这些包添加合成运行时断言,只会让实现围绕通过门禁优化,而不是检测损坏。
|
||||
|
||||
## 决策
|
||||
|
||||
### 注册必须全覆盖;断言必须有意义
|
||||
|
||||
每个 workspace 包都发布单独构建的 `./invariant` companion,并用完整 npm 包名注册。companion 只能采用以下两种形式之一:
|
||||
|
||||
- 安装包自有的事件流或相关可变数据结构检查,并通过绑定的 `fail(message)` 报告器报告违规;或
|
||||
- 使用空安装器,并在其声明前写一条该包专属的 `No runtime invariant:` 注释,说明为什么该包没有合理的运行时关系可供观测。
|
||||
|
||||
空形式是明确的架构结论,不是生成占位符。如果后续包变更引入可变状态或事件协议,就必须用相应检查替换该说明。
|
||||
|
||||
中央 `dsh-invariants` 服务只负责配置、注册唯一性、子 fiber 生命周期、回滚、释放和归属到包的失败。它不暴露通用插件形状、服务形状或启动断言 helper,也不导入产品包。
|
||||
|
||||
### 已实施的检查
|
||||
|
||||
当前 103 个包的 workspace 包含 21 个可执行 companion 和 82 个有理由的空 companion。
|
||||
|
||||
| 所有者 | 运行时关系 |
|
||||
|---|---|
|
||||
| `dsh-session` | 序号严格递增、turn/step 包围关系,以及同一 step 内的工具调用/结果配对。 |
|
||||
| `dsh-agent` | agent 状态不得重复,并且不能离开终态 disposed。 |
|
||||
| `dsh-scope` | scoped event 必须携带 carrier,且路由 subject 保持一致。 |
|
||||
| `dsh-agent-loop` | 从 session 事件日志重建带显式标记的冻结 loop 请求。 |
|
||||
| `dsh-llm` | stream block 文法、delta 类型/索引匹配、单次 usage、block 闭合和终止 finish。 |
|
||||
| `dsh-llm-retry` | 持久化重试记录指向当前打开 turn 中最近关闭的 step;每个 step 的记录保持唯一,重试次数单调递增,并且重试次数和非负的定时器延迟均保持在边界内。 |
|
||||
| `dsh-tools` | pre/execute/post 阶段单调推进,以及最终 execution/result 快照不可变。 |
|
||||
| `dsh-system-prompt` | 权威 assembly 中 section、tool 和 variable 的数据约束。 |
|
||||
| `dsh-compact` | compaction start/summary/end 配对、范围端点、token 数量和成功时必须存在 summary。 |
|
||||
| `dsh-hook-protocol` | hook invocation/result 的关联、dialect、身份和 duration 约束。 |
|
||||
| `dsh-sandbox-policy` | 持久化 `sandbox/mode` 事件必须使用封闭的 sandbox-mode 词表。 |
|
||||
| `dsh-fs` | 文件系统决策/观测事件必须携带可用的 target 和 version 身份。 |
|
||||
| `dsh-goal` | 持久化目标快照保持来源归属、渲染内容、修订号、生命周期和时间戳关系,并保证已准入的目标回合连续编号。 |
|
||||
| `dsh-goal-session` | 目标来源的继续执行消息必须匹配根据此前持久化目标状态重建的提示词。 |
|
||||
| `dsh-subagent` | provider add/remove 和 child start/end 事件必须保持身份与配对。 |
|
||||
| `dsh-permission` | 持久化 permission 决策必须引用当前 permission 表中的 preset。 |
|
||||
| `dsh-user-approval` | approval asked/decided 记录按 call 配对,并使用有效 outcome 和 policy。 |
|
||||
| `dsh-workflow` | workflow 和 child-agent start/end 事件保持 run metadata、身份、outcome、数量和 error 关系。 |
|
||||
| `dsh-tasks` | 当前与终态 task snapshot 保持 id/kind、owner、status 和 timestamp 关系。 |
|
||||
| `dsh-tool-todo` | 持久化全量 snapshot 使用唯一且已 trim 的条目、封闭 status,并且最多有一个活动条目。 |
|
||||
| `dsh-time-context` | 标注插件来源的时钟 reading 必须匹配 session 当前打开的 turn、下一个 step 开始前的位置和 elapsed baseline;渲染时间必须可解析,且不得晚于对应事件。 |
|
||||
|
||||
基于 session 的 companion 在加载时验证已有持久化事件;关系依赖事件顺序时,会使用每个候选事件之前的事件前缀。其他检查观测权威 live event 边界或可变服务结果。如果接受无效事件会提交错误状态,验证就在发布前执行。
|
||||
|
||||
### 仓库门禁与测试
|
||||
|
||||
`verify-package-invariants` 发现每个 workspace 包,并强制 companion 源文件、完整名称注册、仅含具名 export 的 Loader 形状、`./invariant` export、发布文件、依赖、TypeScript reference 和 bundle entry 完整。其 AST 规则拒绝生成标记、默认导出和没有解释的空安装器。非空安装器必须接收并使用失败报告器,注册时还必须传入该经检查的本地 `install` 函数。门禁不会通过方法名或 helper 调用推断语义质量。
|
||||
|
||||
Vitest 为每个包测试拓扑使用 `{ enabled: true }` 挂载 `InvariantService`,并加载所有者 companion。不变量 subpath 的 path mapping 会解析源 companion,而不是陈旧的构建输出。聚焦 suite 覆盖每个可执行 companion 的有效和无效观测;穷举拓扑通过真实 Loader 命名空间归一化运行每个源 companion。产物门禁会按每个包的精确 `npm pack` 文件清单暂存文件,在 plain Node 下导入该包已编译的 `./invariant` 自引用,并重复执行该 Loader 形状检查;这样,未发布的共享运行时分片会在正式发布前导致门禁失败。合成事件流的测试必须构造有效的外围生命周期,除非测试本身就是在断言违规。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **保留生成的空 companion。** 拒绝,因为包获得有意义的运行时关系后,没有解释的占位符仍可能继续存在。
|
||||
- **要求每个包都执行断言。** 拒绝,因为方法存在性、插件形状和固定示例断言会重复更强的类型、加载和单元测试契约,却没有检查运行时一致性。
|
||||
- **在服务中保留通用形状 helper。** 拒绝,因为这会混淆编译期 API 验证和运行时不变量,并鼓励在中央定义产品假设。
|
||||
- **把产品检查移入服务。** 拒绝,因为产品词汇、依赖、测试和变更所有权应归属于产生这些数据的包。
|
||||
- **从根入口隐式注册 companion。** 拒绝,因为组合顺序和可选服务存在性会产生隐藏 effect。
|
||||
|
||||
## 后果
|
||||
|
||||
- 每个包都有可见的所有权与发布 wiring,但只有具备合理运行时关系的包才会增加 listener 或 trace 状态。
|
||||
- 空 companion 是带包专属说明、可评审的决策;删除说明后门禁会失败。
|
||||
- 类型声明、Cordis 可加载性、插件 metadata、服务方法形状和纯代数继续由所属的编译、加载、单元或集成门禁覆盖。
|
||||
- 运行时失败会标明所属 npm 包,并指出不一致的观测,而不是复述必要的 API 形状。
|
||||
- 原有 selection、blocklist 优先级、重复所有权、回滚、释放和 HMR 服务契约保持不变。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-package-owned-invariant-service.md: 2443a8f7d04b96f51bb798130078a7457f78b2a1
|
||||
2026-07-19-package-owned-invariant-service.zh.md: 3c71d3b7f99a507d4c0236b7ef6dc0794814cdc8
|
||||
@@ -0,0 +1,105 @@
|
||||
# Agent Note: Package-owned invariant service seam
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-19-package-owned-invariant-service.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Runtime invariant checks span session traces, agent state, scoped dispatch, and request reconstruction. Putting all checks in one diagnostics package makes that package import product vocabularies from unrelated domains, centralizes tests away from their owners, and requires the central package to change whenever a product package adds or removes a check.
|
||||
|
||||
Deployments also need more than presence or absence of one plugin. A standard composition should carry the known invariant contributions while permitting a global off switch and package-selective diagnostics. Selection must remain stable when a package loads later or reloads under HMR, and disabled contributions must not allow two plugins to claim the same package name silently.
|
||||
|
||||
Package ownership must also be exhaustive. Without a mechanical repository rule, a new package can omit the companion, dependency, or publication wiring and remain invisible to diagnostics until a maintainer notices the gap.
|
||||
|
||||
## Decision
|
||||
|
||||
### One registry service, package-owned contributions
|
||||
|
||||
`@deepseek-ai/dsh-invariants` is a product-independent Cordis service plugin that registers `ctx.invariants`. It owns configuration, registration uniqueness, child-fiber lifecycle, and package-attributed failures. It imports no session, agent, scope, or agent-loop package and contains none of their checks.
|
||||
|
||||
Every workspace package publishes a `./invariant` companion plugin that registers its exact full npm name. A companion checks a meaningful event or mutable-data relationship when its owner has one; otherwise it carries an owner-specific explanation for its empty installer. Generated ownership placeholders and synthetic API-shape assertions are forbidden by the follow-up [runtime-contract Agent Note](2026-07-19-package-invariant-runtime-contracts.md). Package root entrypoints do not import or register diagnostics implicitly, so loading a root package does not change runtime checking or require the invariant service.
|
||||
|
||||
### Configuration and selection
|
||||
|
||||
```ts
|
||||
interface Config {
|
||||
enabled?: boolean
|
||||
package_allowlist?: string[]
|
||||
package_blocklist?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
Defaults are `enabled: true`, `package_allowlist: []`, and `package_blocklist: []`. For a full registration name, selection is:
|
||||
|
||||
```ts
|
||||
export function selected(enabled: boolean, package_allowlist: RegExp[], package_blocklist: RegExp[], packageName: string): boolean {
|
||||
return enabled
|
||||
&& (
|
||||
package_allowlist.length === 0
|
||||
|| package_allowlist.some(pattern => pattern.test(packageName))
|
||||
)
|
||||
&& !package_blocklist.some(pattern => pattern.test(packageName))
|
||||
}
|
||||
```
|
||||
|
||||
Blocklist matches override allowlist matches. Each list entry is a case-sensitive JavaScript regex source compiled by `new RegExp(pattern)`. Matching is unanchored unless callers supply `^` and `$`; slash-delimited syntax and flags are not interpreted. Startup rejects blank, whitespace-padded, invalid, or duplicate sources within either list. A source that matches no loaded package remains valid because registration order, later loading, and HMR must not change config validity.
|
||||
|
||||
### Registration and failure ownership
|
||||
|
||||
The public registration boundary is `ctx.invariants.register(packageName, installer)`. It reserves one active registration per full npm package name even when filters disable installation, and returns the effect disposer. Disposing the companion or service releases the reservation and all contribution state.
|
||||
|
||||
An enabled installer runs in a dedicated child Cordis fiber owned by the service. `InvariantInstaller.inject` declares the child fiber's service surface explicitly; the registry carries no product-specific dependency metadata. The service joins a returned installer promise before registration succeeds, so asynchronous startup checks remain transactional. The installer receives a bound `fail(message)` reporter. Calling it throws an `Error` subclass named `InvariantError` with stable code `INVARIANT` and the registering `packageName`; it does not extend a product-package error base.
|
||||
|
||||
Registration setup is transactional. If an installer fails after registering listeners, the child fiber is disposed completely and the name reservation is released before the failure escapes. Filtered registrations create no child but retain their reservation until disposal. Reloading a companion therefore begins with one clean installer state; stateful contributions rebuild baselines from their owning services.
|
||||
|
||||
The former functional-plugin entrypoint and one-argument `InvariantError` constructor are not retained as compatibility surfaces. The repository is pre-release and all call sites move to the service and package-attributed error together.
|
||||
|
||||
### Initial stateful companions and exhaustive ownership
|
||||
|
||||
| Companion entry | Registration name | Owned checks |
|
||||
|---|---|---|
|
||||
| `@deepseek-ai/dsh-session/invariant` | `@deepseek-ai/dsh-session` | session sequence, turn/step enclosure, and same-step call/result trace |
|
||||
| `@deepseek-ai/dsh-agent/invariant` | `@deepseek-ai/dsh-agent` | agent-status transitions |
|
||||
| `@deepseek-ai/dsh-scope/invariant` | `@deepseek-ai/dsh-scope` | scoped-event carrier presence and subject consistency |
|
||||
| `@deepseek-ai/dsh-agent-loop/invariant` | `@deepseek-ai/dsh-agent-loop` | model-request reconstruction |
|
||||
|
||||
These four owners supplied the initial stateful checks. The follow-up runtime-contract decision adds checks for seventeen more owners with real event or mutable-data relationships and records justified empty companions for the rest. Every companion is a separately bundled `./invariant` export with its own declarations and Loader-safe namespace plugin shape; the service package's own companion imports its local service type to avoid a self-dependency.
|
||||
|
||||
`verify-package-invariants` discovers every workspace package and rejects missing companion source, generated markers, unexplained empty installers, non-empty installers that omit or ignore the reporter, foreign or unresolved registration names, missing `./invariant` exports or published files, missing invariant peer/development dependencies and project references, and bundle overrides that omit the companion entry.
|
||||
|
||||
### Scoped-event semantic map
|
||||
|
||||
The generated scoped-event subject resolver lives in `dsh-scope`, beside the contract and invariant that consume it. `gen-scoped-events` uses the root TypeScript Program to enumerate `this: Scoped<Base>` declarations, infer routing-key types from real `scopeTarget(base, key)` calls, and require one unambiguous payload subject or an explicit unsupported marker. The committed runtime map imports no event-owner package, so semantic completeness does not expand either the service or scope package's runtime closure.
|
||||
|
||||
### Standard composition and SDK output
|
||||
|
||||
The standard agent spine mounts the service and all four stateful companion subpaths, forwarding `enabled`, `package_allowlist`, and `package_blocklist` to the service. Generated SDK Cordis composition emits the same entries. A subpath entry adds its installable root npm package rather than treating the subpath as a package name.
|
||||
|
||||
Workspace constraints recognize the separate invariant bundle, and package exports, project references, build configuration, dependency declarations, and the lockfile describe the same publication surface. Generated config catalogs, module graphs, and API documentation derive from those sources.
|
||||
|
||||
## Testing
|
||||
|
||||
Service tests cover defaults, global disablement, allow/block selection, blocklist precedence, anchoring, unanchored matching, case sensitivity, invalid configuration, zero-match patterns, late registration, duplicate ownership, disposal, rollback, and HMR re-registration. Owners with executable checks keep positive and negative behavior beside the companion source.
|
||||
|
||||
Composition tests cover standard-spine forwarding and generated SDK entries. Loader tests preserve each companion namespace, while built plain-Node smokes exercise the compiled subpath exports. The scoped-event freshness gate reruns its semantic Program analysis.
|
||||
|
||||
Every Vitest configuration loads a test host that mounts an explicitly enabled service before an ordinary Cordis root's first plugin and adds the current test package's companion. One exhaustive topology mounts all package companions once; focused service and owner tests construct their own invariant topology so they can exercise disablement, filtering, rollback, and reload without duplicate ownership. Gate tests also execute every companion's `apply` function and verify that it calls `register` with its manifest name, rather than accepting source text alone.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep all checks in `dsh-invariants`.** Rejected because the registry would continue importing every checked product domain, owner changes would require central edits, and package tests would remain detached from the contracts they protect.
|
||||
- **Let root package entrypoints register checks implicitly when `ctx.invariants` happens to exist.** Rejected because root behavior would depend on composition order and optional service presence, diagnostics could not be selected independently, and package loading would hide a registration effect outside an explicit companion.
|
||||
- **Discover every `invariant.ts` file automatically at runtime.** Rejected because filesystem/package discovery is not a runtime ownership contract, makes bundled publication ambiguous, and cannot express explicit Cordis load order or dependency installation. Build-time generation, verification, and the test host may enumerate the source tree because they validate repository completeness rather than composing a shipped deployment.
|
||||
- **Validate allow/block entries against the currently loaded package set.** Rejected because a zero-match pattern can intentionally target a later or HMR-loaded contribution; current load order must not determine config validity.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Product packages own and test their relational assertions while the service stays product-independent.
|
||||
- Every package pays the publication and dependency cost of a companion; only owners with a meaningful runtime relationship add listener or trace-state cost.
|
||||
- Standard compositions can disable all checks or select package names without changing their plugin tree.
|
||||
- Explicit companion entries make diagnostic cost and ownership visible in Cordis config and package exports.
|
||||
- One selected executable contribution adds one child fiber and its listener/state cost; a selected empty contribution has no listener or trace-state cost, while filtered registrations retain only name ownership.
|
||||
- Regex sources are deployment configuration and remain fixed until the service reloads.
|
||||
- Ordinary Vitest roots install the owning test package's selected companion; one exhaustive topology pays the full child-fiber cost once for repository-wide registration coverage.
|
||||
- Session storage validation, snapshotting, freezing, provenance, and surface acceptance remain always on and are not affected by invariant selection.
|
||||
@@ -0,0 +1,105 @@
|
||||
# Agent Note: 包拥有的不变式服务接缝
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-19-package-owned-invariant-service.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
运行时不变式检查跨越会话轨迹、agent 状态、作用域 dispatch 和请求重建。如果所有检查都放在一个诊断包中,该包就必须导入彼此无关的产品领域词汇,测试也会离开真正的所有者;任何产品包新增或移除检查时,都要修改中央包。
|
||||
|
||||
部署还需要比“是否加载一个插件”更细的控制。标准组合应携带已知的不变式贡献,同时允许全局关闭或按包选择诊断。包稍后加载或在 HMR 下重载时,选择结果必须保持稳定;被过滤的贡献也不能让两个插件静默占用同一个包名。
|
||||
|
||||
包所有权还必须覆盖完整。若没有机械化的仓库规则,新包可能遗漏伴随插件、依赖或发布配置,并一直不会进入诊断范围,直到维护者发现这一缺口。
|
||||
|
||||
## 决策
|
||||
|
||||
### 一个注册服务,贡献归包所有
|
||||
|
||||
`@deepseek-ai/dsh-invariants` 是与产品无关的 Cordis 服务插件,注册 `ctx.invariants`。它只负责配置、注册唯一性、子 fiber 生命周期和带包归属的失败;不导入 session、agent、scope 或 agent-loop 包,也不包含这些包的检查。
|
||||
|
||||
工作区内的每个包都发布 `./invariant` 伴随插件,注册自己完整且准确的 npm 包名。如果所有者具备有意义的事件或可变数据关系,companion 就检查该关系;否则空 installer 必须携带该所有者专属的说明。后续的[运行时契约 Agent Note](2026-07-19-package-invariant-runtime-contracts.md) 禁止生成的所有权占位符和合成 API 形状断言。包的根入口不会隐式导入或注册诊断,因此加载根包不会改变运行时检查,也不要求不变式服务存在。
|
||||
|
||||
### 配置与选择
|
||||
|
||||
```ts
|
||||
interface Config {
|
||||
enabled?: boolean
|
||||
package_allowlist?: string[]
|
||||
package_blocklist?: string[]
|
||||
}
|
||||
```
|
||||
|
||||
默认值为 `enabled: true`、`package_allowlist: []` 和 `package_blocklist: []`。对完整注册名的选择规则为:
|
||||
|
||||
```ts
|
||||
export function selected(enabled: boolean, package_allowlist: RegExp[], package_blocklist: RegExp[], packageName: string): boolean {
|
||||
return enabled
|
||||
&& (
|
||||
package_allowlist.length === 0
|
||||
|| package_allowlist.some(pattern => pattern.test(packageName))
|
||||
)
|
||||
&& !package_blocklist.some(pattern => pattern.test(packageName))
|
||||
}
|
||||
```
|
||||
|
||||
blocklist 匹配优先于 allowlist 匹配。每个条目都是区分大小写的 JavaScript 正则表达式源,通过 `new RegExp(pattern)` 编译。除非调用方提供 `^` 与 `$`,否则匹配不锚定;系统不会解析斜杠包围语法或 flags。服务启动会拒绝空白、首尾带空白、无效或同一列表内重复的源。没有匹配当前已加载包的有效源仍然合法,因为注册顺序、稍后加载和 HMR 不应改变配置有效性。
|
||||
|
||||
### 注册与失败归属
|
||||
|
||||
公开注册边界是 `ctx.invariants.register(packageName, installer)`。即使过滤器禁止安装,它也会为每个完整 npm 包名保留唯一的活跃注册,并返回 effect disposer。卸载伴随插件或服务都会释放注册名及全部贡献状态。
|
||||
|
||||
启用的 installer 在服务拥有的独立 Cordis 子 fiber 中运行。`InvariantInstaller.inject` 显式声明该子 fiber 的服务表面;注册服务不携带产品专用依赖元数据。服务会在注册成功前等待 installer 返回的 promise,因此异步启动检查仍具有事务性。installer 接收绑定后的 `fail(message)` 报告器。调用它会抛出名为 `InvariantError` 的 `Error` 子类,保留稳定代码 `INVARIANT` 并记录注册方 `packageName`;该错误不继承产品包中的错误基类。
|
||||
|
||||
注册启动是事务性的。如果 installer 在注册监听器后失败,子 fiber 会完整释放,并在失败向外传播前解除包名占用。被过滤的注册不创建子 fiber,但会保留占用直到 dispose。伴随插件重载时总会从干净的 installer 状态开始;有状态贡献从其所属服务重建基线。
|
||||
|
||||
原有函数式插件入口与单参数 `InvariantError` 构造函数不作为兼容表面保留。仓库尚未发布,所有调用方会一起迁移到服务和带包归属的错误。
|
||||
|
||||
### 首批有状态伴随插件与完整所有权
|
||||
|
||||
| 伴随入口 | 注册名 | 所属检查 |
|
||||
|---|---|---|
|
||||
| `@deepseek-ai/dsh-session/invariant` | `@deepseek-ai/dsh-session` | 会话序号、turn/step 包围关系和同 step 的 call/result 轨迹 |
|
||||
| `@deepseek-ai/dsh-agent/invariant` | `@deepseek-ai/dsh-agent` | agent 状态转换 |
|
||||
| `@deepseek-ai/dsh-scope/invariant` | `@deepseek-ai/dsh-scope` | scoped event carrier 存在性与主体一致性 |
|
||||
| `@deepseek-ai/dsh-agent-loop/invariant` | `@deepseek-ai/dsh-agent-loop` | 模型请求重建 |
|
||||
|
||||
这四个所有者提供了首批有状态检查。后续运行时契约决策为另外十七个确有事件或可变数据关系的所有者增加检查,并为其余包记录有理由的空 companion。每个伴随入口都是单独打包的 `./invariant` export,具有独立声明和对 Loader 安全的命名空间插件形态;服务包自身的伴随插件导入本地服务类型,避免形成自依赖。
|
||||
|
||||
`verify-package-invariants` 会发现每个工作区包,并拒绝缺失的伴随插件源码、生成标记、没有解释的空 installer、缺少或不使用失败报告器的非空 installer、外部或无法解析的注册名、缺失的 `./invariant` export 或发布文件、缺失的不变式对等依赖(peer dependency)、开发依赖及项目引用,以及遗漏伴随入口的自定义构建配置。
|
||||
|
||||
### Scoped event 语义映射
|
||||
|
||||
生成的 scoped event 主体解析表位于 `dsh-scope`,与消费它的契约和不变式相邻。`gen-scoped-events` 使用根 TypeScript Program 枚举 `this: Scoped<Base>` 声明,从真实 `scopeTarget(base, key)` 调用推断路由键类型,并要求唯一、无歧义的 payload 主体或显式 unsupported 标记。提交的运行时映射不导入事件所有者包,因此语义完整性不会扩大服务包或 scope 包的运行时依赖闭包。
|
||||
|
||||
### 标准组合与 SDK 输出
|
||||
|
||||
标准 agent spine 会挂载服务和四个有状态伴随子路径,并把 `enabled`、`package_allowlist` 与 `package_blocklist` 转发给服务。生成的 SDK Cordis 组合输出相同条目。子路径条目添加可安装的根 npm 包,而不会把子路径误当成包名。
|
||||
|
||||
Workspace 约束识别独立的不变式 bundle;包 exports、项目引用、构建配置、依赖声明和 lockfile 描述同一发布表面。生成的配置目录、模块图和 API 文档都从这些源派生。
|
||||
|
||||
## 测试
|
||||
|
||||
服务测试覆盖默认值、全局关闭、allow/block 选择、blocklist 优先级、锚定与非锚定匹配、大小写敏感、无效配置、零匹配模式、延迟注册、重复所有权、dispose、回滚和 HMR 重新注册。具备可执行检查的所有者会把正向与负向行为保留在 companion 源码旁边。
|
||||
|
||||
组合测试覆盖标准 spine 转发和生成的 SDK 条目。Loader 测试固定每个伴随命名空间,构建后的纯 Node smoke 覆盖编译子路径 export。scoped event 新鲜度门禁会重新执行语义 Program 分析。
|
||||
|
||||
每个 Vitest 配置都会加载测试宿主;在普通 Cordis 根上下文启动第一个插件之前,宿主会挂载显式启用的服务,并添加当前测试包的伴随插件。一个完整拓扑会一次挂载所有包的伴随插件;服务与所有者的聚焦测试自行构建不变式拓扑,从而在不发生重复所有权冲突的前提下覆盖关闭、过滤、回滚与重载。门禁测试还会执行每个伴随插件的 `apply` 函数,并验证它调用 `register` 时使用包清单中的包名,而不是只检查源码文本。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **把所有检查保留在 `dsh-invariants`。** 不予采纳,因为注册包仍要导入所有被检查的产品领域,所有者变更仍需中央编辑,测试也继续远离被保护的契约。
|
||||
- **当 `ctx.invariants` 恰好存在时,让根包入口隐式注册检查。** 不予采纳,因为根入口行为会依赖组合顺序与可选服务是否存在,诊断无法独立选择,而且包加载会隐藏一个不在显式伴随插件中的注册 effect。
|
||||
- **在运行时自动发现所有 `invariant.ts` 文件。** 不予采纳,因为文件系统或包发现不是运行时所有权契约,会让 bundle 发布含义不清,也无法表达显式 Cordis 加载顺序或依赖安装。构建期生成与校验以及测试 host 可以枚举源码树,因为它们验证的是仓库完整性,而不是组合已发布的部署。
|
||||
- **根据当前已加载包集合验证 allow/block 条目。** 不予采纳,因为零匹配模式可能有意指向稍后加载或 HMR 加载的贡献;当前加载顺序不能决定配置有效性。
|
||||
|
||||
## 后果
|
||||
|
||||
- 产品包拥有并测试自己的关系断言,服务保持与产品无关。
|
||||
- 每个包都承担 companion 的发布与依赖成本;只有具备有意义运行时关系的所有者才增加 listener 或 trace 状态成本。
|
||||
- 标准组合无需改变插件树即可关闭全部检查或按包名选择。
|
||||
- 显式伴随条目让诊断成本和所有权在 Cordis 配置与包 export 中可见。
|
||||
- 每个选中的可执行贡献增加一个子 fiber 及其 listener/状态成本;选中的空贡献不增加 listener 或 trace 状态成本,被过滤注册则只保留包名占用。
|
||||
- 正则表达式源属于部署配置,在服务重载前保持固定。
|
||||
- 普通 Vitest 根上下文会安装当前测试包中被选中的伴随插件;一个完整拓扑只支付一次全部子 fiber 成本,用于覆盖整个仓库的注册。
|
||||
- 会话存储验证、快照、冻结、provenance 与 surface 接受规则始终启用,不受不变式选择影响。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-19-zstandard-jsonl-session-logs.md: 09d30594fe31eed138a128dabc1947b15857808d
|
||||
2026-07-19-zstandard-jsonl-session-logs.zh.md: 131531d9dba7cb01407191bf937f8b0ee3c6860a
|
||||
@@ -0,0 +1,57 @@
|
||||
# Agent Note: Zstandard JSONL session logs
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-19-zstandard-jsonl-session-logs.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The JSONL persistence backend keeps every `SessionEvent` verbatim, including high-volume `assistant/chunk` records. Raw text makes logs inspectable but spends storage and I/O on repeated JSON keys and model text. Compression must retain the existing append/fsync commit boundary, collision-safe first materialization, crash repair, and metadata-only listing; rewriting a whole compressed file after every turn would discard those properties.
|
||||
|
||||
The encoding also has to remain explicit at the deployment boundary. Snapshot fixtures and external line readers require raw JSONL, while a backend cannot safely guess between compressed and raw artifacts in one root or silently migrate pre-release session data.
|
||||
|
||||
## Decision
|
||||
|
||||
### Configuration and suffix ownership
|
||||
|
||||
`dsh-session-persistence-jsonl` accepts `compression?: 'zstd' | 'none'` and explicitly resolves omission to `'zstd'`. Zstandard artifacts end in `.jsonl.zstd`; `'none'` retains the original newline-delimited UTF-8 `.jsonl` representation. `SessionLocation.kind` remains `'jsonl'`, because both encodings carry the same logical record format, and `SESSION_FORMAT_VERSION` remains `0` under the repository's pre-release reject-without-migration policy.
|
||||
|
||||
Each persistence root belongs to one encoding. A one-time discovery preflight rejects any opposite suffix, and targeted load, live-adoption, listing, and materialization paths repeat the relevant suffix check after an initially empty preflight. The error names the incompatible artifact and directs the deployment to the matching configuration or a separate root. There is no migration, dual read, dual write, or extension-based fallback.
|
||||
|
||||
### Frame and write path
|
||||
|
||||
The compressed artifact is a standard concatenation of independent [Zstandard frames](https://datatracker.ietf.org/doc/html/rfc8878): one checksummed frame containing exactly the header line, followed by one checksummed frame for every durable append batch. Normal loop batches are turn commits, so frame boundaries preserve the existing persistence checkpoint without making the storage layer depend on turn event types.
|
||||
|
||||
Compression uses Node's built-in [`zstdCompress` and `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html), available at the repository's Node 22.19 floor. The backend enables `ZSTD_c_checksumFlag`, otherwise accepts Node's defaults, and exposes neither a compression-level knob nor a new dependency. The API is marked experimental by Node, so the Node 22.19, 24, and 26 compatibility gate exercises the exact helper.
|
||||
|
||||
First materialization compresses the two initial frames before opening the temporary file, then keeps the existing write, file `fsync`, collision-safe hard-link publication, and directory `fsync` sequence. Later batches are compressed before opening the destination and appended at EOF. A caught write or file-sync failure truncates to the prior byte length, syncs the rollback, and rethrows so the coordinator can retry the unchanged batch.
|
||||
|
||||
### Read, listing, and crash recovery
|
||||
|
||||
A frame-boundary scanner reads the standard magic, variable header fields, block headers and payload sizes, and optional checksum trailer. It does not interpret compressed blocks. Complete frames are decompressed independently and sequentially, which validates their checksums, and their plaintext is passed to the existing JSONL scanner. A checksum/decompression failure in any complete frame, a malformed complete-frame JSONL tail, or invalid frame structure is corruption and rejects.
|
||||
|
||||
Listing reads in bounded chunks only until the first complete frame is available, validates and decompresses that header frame, and never reads an event frame. The dedicated header frame therefore preserves metadata-only listing even for very large session logs.
|
||||
|
||||
EOF inside the final frame is a recoverable torn tail. Node's decoder is given the available frame prefix; every complete newline-terminated event it emits is retained. Repair truncates from that frame's starting byte and appends one new checksummed frame containing the recovered complete events followed by the coordinator's synthetic tool, step, and turn closers. If the tear occurs before any complete event is decodable, repair drops the partial frame and retains all prior complete frames.
|
||||
|
||||
### Consumers and verification
|
||||
|
||||
The CLI, ACP, and stdio app bundles expose symmetric `persistenceCompression` pass-through configuration. Snapshot recording and replay compositions select `'none'` explicitly because committed fixtures are raw JSONL inputs to replay and normalization; ordinary runtime compositions use the compressed default.
|
||||
|
||||
The shared persistence and coordinator contracts run against both encodings. Backend tests cover standard framing and checksum interoperability, header-only listing, append rollback, encoding mismatch rejection, complete-frame corruption, and final-frame tears through headers, blocks, and checksum trailers. Default runtime, built-bin, headless, ACP, and Python smokes assert the compressed suffix and Zstandard magic or decode the header; raw-content tests opt out explicitly.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **One frame per JSONL record** — rejected because it multiplies frame headers and checksums for high-volume chunk events and makes a physical boundary unrelated to the durable append batch.
|
||||
- **Rewrite one whole compressed stream after every append** — rejected because cost grows with log size and replacement would give up append/fsync rollback and the established collision-safe materialization mechanics.
|
||||
- **Use a streaming compressor across appends** — rejected because an interrupted encoder state does not leave independently checksummed append units, complicating bounded listing and frame-start repair.
|
||||
- **Add an external native Zstandard dependency** — rejected because the supported Node floor already provides the required codec; another native artifact would enlarge installation and executable-packaging risk without adding a required behavior.
|
||||
- **Expose compression level or keep raw JSONL as the default** — rejected because there is no deployment evidence for a second tuning policy, while `'none'` preserves the line-readable path for fixtures and integrations that need it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Ordinary session roots store `.jsonl.zstd` and retain append-only, fsync, rollback, and interrupted-turn recovery semantics.
|
||||
- Raw JSONL remains a deliberate configuration, but changing encoding requires a fresh/separate root or selecting the mode that matches existing artifacts.
|
||||
- One frame per durable batch adds bounded framing/checksum overhead and allows header-only listing plus repair from an exact append boundary.
|
||||
- External tools must understand concatenated Zstandard frames or consume raw-mode artifacts; generic one-shot Node decompression reads only the first independent frame, so backend reads walk frames explicitly.
|
||||
- The implementation depends on Node's experimental built-in Zstandard API without an npm dependency; the supported-version compatibility gate makes drift visible.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Agent Note: Zstandard JSONL 会话日志
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-19-zstandard-jsonl-session-logs.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
JSONL 持久化后端会逐字保留每个 `SessionEvent`,其中包括数量庞大的 `assistant/chunk` 记录。原始文本便于检查,但重复的 JSON 键和模型文本会增加存储与 I/O 开销。压缩编码必须保留既有的 append/fsync 提交边界、首次物化时的无冲突发布、崩溃修复以及仅元数据列举;如果每轮都重写整个压缩文件,就会失去这些属性。
|
||||
|
||||
编码还必须在部署边界上保持显式。快照 fixture 与外部逐行读取器需要原始 JSONL,而后端无法在同一根目录中安全猜测压缩产物与原始产物,也不能静默迁移预发布会话数据。
|
||||
|
||||
## 决策
|
||||
|
||||
### 配置与后缀归属
|
||||
|
||||
`dsh-session-persistence-jsonl` 接受 `compression?: 'zstd' | 'none'`,并将省略值显式解析为 `'zstd'`。Zstandard 产物使用 `.jsonl.zstd` 后缀;`'none'` 保留原有的换行分隔 UTF-8 `.jsonl` 表示。`SessionLocation.kind` 仍为 `'jsonl'`,因为两种编码承载同一逻辑记录格式;按照仓库的预发布拒绝且不迁移策略,`SESSION_FORMAT_VERSION` 仍为 `0`。
|
||||
|
||||
每个持久化根目录只归属于一种编码。一次性的发现预检会拒绝任何相反后缀,而针对性的加载、活跃采用、列举与物化路径会在最初空目录预检之后再次执行对应后缀检查。错误会指出不兼容产物,并要求部署选择匹配配置或单独根目录。系统不提供迁移、双重读取、双重写入或基于扩展名的兜底。
|
||||
|
||||
### 帧与写入路径
|
||||
|
||||
压缩产物是标准独立 [Zstandard 帧](https://datatracker.ietf.org/doc/html/rfc8878)的串联:第一个带校验和的帧只包含头部行,后续每个持久追加批次各占一个带校验和的帧。正常 agent loop 批次就是轮次提交,因此帧边界保留既有持久化检查点,同时不让存储层依赖轮次事件类型。
|
||||
|
||||
压缩使用 Node 内置的 [`zstdCompress` 与 `zstdDecompress`](https://nodejs.org/download/release/v22.19.0/docs/api/zlib.html),仓库最低支持的 Node 22.19 已提供这些 API。后端启用 `ZSTD_c_checksumFlag`,其余采用 Node 默认值,不公开压缩级别调节项,也不增加依赖。Node 将该 API 标记为实验性,因此 Node 22.19、24 与 26 兼容性门禁会执行同一个辅助实现。
|
||||
|
||||
首次物化会在打开临时文件之前压缩两个初始帧,然后保留既有的写入、文件 `fsync`、避免冲突的硬链接发布与目录 `fsync` 顺序。后续批次也会先压缩,再打开目标并在 EOF 追加。捕获到写入或文件同步失败时,后端会截断到原有字节长度,同步回滚结果,再重新抛出错误,让协调器重试未变化的批次。
|
||||
|
||||
### 读取、列举与崩溃恢复
|
||||
|
||||
帧边界扫描器会读取标准魔数、可变头字段、块头与负载长度,以及可选校验和尾部,但不会解释压缩块。后端独立且按顺序解压完整帧,由此验证各帧校验和,再把明文交给既有 JSONL 扫描器。任何完整帧的校验和或解压失败、完整帧中畸形的 JSONL 尾部,或者无效帧结构都属于损坏并拒绝加载。
|
||||
|
||||
列举只按有界分片读取到第一个完整帧可用为止,验证并解压该头部帧,绝不读取事件帧。因此,即使会话日志很大,专用头部帧仍能维持仅元数据列举。
|
||||
|
||||
最终帧内部遇到 EOF 属于可恢复的撕裂尾部。后端把已有帧前缀交给 Node 解码器,并保留其产出的每个完整、以换行结束的事件。修复从该帧起始字节截断,再追加一个新的带校验和帧,其中依次包含恢复出的完整事件,以及协调器生成的工具、步骤与轮次闭合事件。如果撕裂位置尚不足以解码任何完整事件,修复会丢弃该不完整帧并保留此前全部完整帧。
|
||||
|
||||
### 消费方与验证
|
||||
|
||||
CLI、ACP 与 stdio 应用包公开对称的 `persistenceCompression` 透传配置。快照录制与回放组合显式选择 `'none'`,因为提交的 fixture 是回放与规范化过程使用的原始 JSONL 输入;普通运行时组合使用压缩默认值。
|
||||
|
||||
共享持久化契约与协调器契约会针对两种编码运行。后端测试覆盖标准帧与校验和互操作性、仅头部列举、追加回滚、编码不匹配拒绝、完整帧损坏,以及横跨头部、块和校验和尾部的最终帧撕裂。默认运行时、构建后二进制、headless、ACP 与 Python 冒烟测试会断言压缩后缀与 Zstandard 魔数,或解码头部;读取原始内容的测试则显式退出压缩。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **每条 JSONL 记录一个帧**——不予采纳,因为它会让大量分片事件各自承担帧头与校验和开销,并让物理边界脱离持久追加批次。
|
||||
- **每次追加都重写一个完整压缩流**——不予采纳,因为成本会随日志大小增长,而且替换操作会放弃追加/fsync 回滚和既有的无冲突物化机制。
|
||||
- **跨追加使用流式压缩器**——不予采纳,因为编码器状态中断后不会留下可独立校验的追加单元,从而使有界列举与按帧起点修复更复杂。
|
||||
- **增加外部原生 Zstandard 依赖**——不予采纳,因为受支持的 Node 最低版本已经提供所需编解码器;另一个原生产物会增加安装与可执行文件打包风险,却不增加必需行为。
|
||||
- **公开压缩级别或继续默认使用原始 JSONL**——不予采纳,因为没有部署证据支持第二种调节策略,而 `'none'` 已为需要逐行读取的 fixture 与集成保留路径。
|
||||
|
||||
## 后果
|
||||
|
||||
- 普通会话根目录存储 `.jsonl.zstd`,并保留仅追加、fsync、回滚与中断轮次恢复语义。
|
||||
- 原始 JSONL 仍是显式配置,但切换编码需要使用全新或单独根目录,或者选择与既有产物匹配的模式。
|
||||
- 每个持久批次一个帧会增加有界的帧与校验和开销,同时支持仅头部列举和从精确追加边界开始修复。
|
||||
- 外部工具必须理解串联的 Zstandard 帧,或者消费原始模式产物;Node 通用的一次性解压只读取第一个独立帧,因此后端读取会显式遍历各帧。
|
||||
- 实现依赖 Node 的实验性内置 Zstandard API,但不增加 NPM 依赖;受支持版本兼容性门禁会暴露 API 漂移。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-20-routed-model-context-and-compaction-policy.md: f0b9288d3d864bfcc2964862b1ff294406daa345
|
||||
2026-07-20-routed-model-context-and-compaction-policy.zh.md: cda740a5671a3ef8a5bb415e5cc45ca8397c1c59
|
||||
@@ -0,0 +1,57 @@
|
||||
# Agent Note: Routed model context and compaction policy
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-20-routed-model-context-and-compaction-policy.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Compaction cannot safely apply one global context window when a process routes requests to models with different capacities. The same model id can also exist under multiple providers, and an adapter may accept dynamic ids absent from its advisory catalog. A wrong capacity either compacts too late and triggers avoidable overflow or compacts too early and discards useful context.
|
||||
|
||||
Neither obvious configuration owner is sufficient. Compact-basic is optional and does not know which models an adapter accepts. LLM adapters own model routing but must not depend on an optional compaction plugin or absorb consumer-specific threshold, retention, summarizer, and retry policy. The design needs an authoritative capacity fact and optional per-target compaction policy without creating a second model registry.
|
||||
|
||||
## Decision
|
||||
|
||||
### Adapters own exact-route capacity
|
||||
|
||||
`LlmAdapter.resolveModelContext(provider, model)` optionally returns `LlmModelContext` for one exact route. `LlmService.resolveModelContext()` selects the registered route owner, validates a positive integer `contextWindow`, and returns a detached value. The query is independent of `listModels()`: an unlisted dynamic model may have capacity metadata, and `undefined` means only that the adapter cannot describe capacity.
|
||||
|
||||
The hand-rolled DeepSeek adapter accepts optional `contextWindow` on each configured model. Its two default model entries publish 128,000 tokens; an explicit entry without capacity and an unlisted pass-through id return `undefined`. The pi-ai adapter resolves capacity from the same catalog descriptor that authoritatively resolves the request model.
|
||||
|
||||
### Token measurement remains model-agnostic
|
||||
|
||||
`dsh-token-meter` has no configuration and no model profiles. It owns one fixed replay fold and returns absolute estimated token pressure plus positional surface prices. Removing global capacity keeps measurement reusable when compact-basic is absent and prevents replay accounting from becoming another model registry.
|
||||
|
||||
### Compact-basic resolves a target spec
|
||||
|
||||
Compact-basic owns consumer policy. Top-level fields define defaults; `modelPolicies` contains partial overrides keyed by the exact `{ provider, model }` pair. Duplicate targets and unknown or invalid fields fail plugin load. `thresholdRatio` defaults to `0.8`, and retention defaults to `retainRatio: 0.16`; callers may use an absolute `retainTokens` instead, but the two retention forms are mutually exclusive. After inheritance, a ratio retention that is not below its threshold ratio also fails plugin load because no model capacity can make that policy valid.
|
||||
|
||||
For proactive pressure, compact-basic reads the latest durable request route, resolves its adapter capacity and exact-target policy, and scales ratios into a `ResolvedCompactSpec`. It performs this resolution on every check, so a provider or model switch in one session changes capacity and policy immediately. An absolute retained budget that is not below the scaled threshold fails when the target capacity first makes that comparison possible.
|
||||
|
||||
The same exact-target override can select summarization provider/model, summarization output cap, convergence retries, and overflow retry cap. These are compaction concerns and never enter the adapter seam.
|
||||
|
||||
### Target-specific pressure failures preserve optional composition
|
||||
|
||||
An adapter that lacks capacity metadata remains a valid LLM route. Manual proactive pressure fails with a target-specific configuration error; the automatic listener warns once per exact route and continues with full history. The same per-route suppression applies when resolved capacity exposes an invalid absolute retention budget, while unrelated operational failures remain independently visible. Canonical provider-confirmed overflow does not need capacity metadata: it bypasses the proactive threshold and normal retention budget, attempts one maximal balanced reduction, and preserves the original provider error unless replacement proves progress.
|
||||
|
||||
## Testing
|
||||
|
||||
Service tests cover detached context metadata, invalid adapter output, catalog independence, and default absence. Adapter tests cover DeepSeek configured/default/unlisted behavior and pi-ai exact descriptor resolution. Compact tests cover ratio scaling, exact provider/model overrides, load-time rejection of invalid merged ratios, runtime absolute-budget validation, same-model-id provider switches, target-specific warning suppression, and capacity-independent overflow recovery. Loader fixtures reject the removed token-meter capacity setting, and examples configure capacity on adapters.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Put capacity and all policies in compact-basic** — rejected because compact-basic would duplicate adapter model knowledge, dynamic unlisted models would require parallel registration, and capacity would disappear when compaction is not installed.
|
||||
- **Put compaction policy in each LLM adapter** — rejected because adapters must remain independent of optional consumers, while summarization and retry policy are not provider facts.
|
||||
- **Make `listModels()` authoritative** — rejected because discovery is advisory and some adapters intentionally accept dynamic ids. Correctness metadata must not turn selector membership into a routing whitelist.
|
||||
- **Add per-model folds to token-meter** — rejected because the replay algorithm is shared; only the capacity and consumer policy change. Multiple folds would duplicate state without improving estimation.
|
||||
- **Create a standalone model-context registry** — rejected because the adapter already owns authoritative route resolution. A second registry would introduce lifecycle ordering, duplicate-key, and drift problems without an independent backend.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Capacity has one authoritative owner at the provider seam, while compaction policy stays in the optional consuming plugin.
|
||||
- The same compact-basic instance safely handles different windows, provider switches, and identical model ids under different providers without consulting discovery metadata.
|
||||
- LLM-only and meter-only compositions remain valid; loading compact-basic adds no reverse dependency from adapters.
|
||||
- Deployments using explicit DeepSeek model lists must provide `contextWindow` for proactive pressure on those entries. Missing metadata is visible instead of silently applying a wrong global fallback.
|
||||
- Ratio defaults scale naturally across models, while exact-target absolute retention remains available for deployment-specific behavior.
|
||||
|
||||
This note supersedes the global-capacity and no-model-policy parts of the [replay token meter service Agent Note](2026-07-15-replay-token-meter-service.md). Its single-fold measurement decision remains unchanged.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Agent Note: 路由模型上下文与压缩策略
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-20-routed-model-context-and-compaction-policy.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
当一个进程把请求路由到不同容量的模型时,压缩不能安全地应用同一个全局上下文窗口。相同模型 id 也可能存在于多个提供方下,适配器还可能接受不在建议目录中的动态 id。错误容量要么让压缩触发过晚并造成原本可避免的溢出,要么让压缩触发过早并丢弃有用上下文。
|
||||
|
||||
两个直观的配置归属方都无法独立解决问题。Compact-basic 是可选插件,不知道适配器接受哪些模型。LLM 适配器拥有模型路由,但不能依赖可选压缩插件,也不应吸收消费方专用的阈值、保留、摘要器与重试策略。该设计既需要权威容量事实和可选的逐目标压缩策略,又不能建立第二套模型注册表。
|
||||
|
||||
## 决策
|
||||
|
||||
### 适配器拥有精确路由容量
|
||||
|
||||
`LlmAdapter.resolveModelContext(provider, model)` 可以为一条精确路由返回 `LlmModelContext`。`LlmService.resolveModelContext()` 选择已注册的路由所属方,验证 `contextWindow` 为正整数,并返回分离值。该查询独立于 `listModels()`:不在目录中的动态模型也可以拥有容量元数据,而 `undefined` 只表示适配器无法描述容量。
|
||||
|
||||
手写 DeepSeek 适配器允许每个已配置模型提供可选 `contextWindow`。两个默认模型项都公开 128,000 token;未提供容量的显式模型项与未列出的透传 id 返回 `undefined`。pi-ai 适配器从同一个目录描述符解析容量,该描述符也用于权威解析请求模型。
|
||||
|
||||
### Token 计量保持模型无关
|
||||
|
||||
`dsh-token-meter` 没有配置,也没有模型 profile。它拥有一个固定回放折叠,并返回绝对估算 token 压力与逐位置表层价格。移除全局容量后,未加载 compact-basic 时仍可复用计量,同时避免让回放核算变成另一套模型注册表。
|
||||
|
||||
### Compact-basic 解析目标规格
|
||||
|
||||
Compact-basic 拥有消费方策略。顶层字段定义默认值;`modelPolicies` 包含以精确 `{ provider, model }` 组合为键的部分覆盖。重复目标、未知字段或无效字段都会让插件加载失败。`thresholdRatio` 默认为 `0.8`,保留策略默认为 `retainRatio: 0.16`;调用方也可以改用绝对 `retainTokens`,但两种保留形式互斥。完成继承后,如果保留比例不小于阈值比例,插件也会加载失败,因为任何模型容量都无法让该策略有效。
|
||||
|
||||
对于主动压力检查,compact-basic 读取最新持久请求路由,解析其适配器容量与精确目标策略,再把比例缩放为 `ResolvedCompactSpec`。每次检查都会重新解析,因此同一会话切换提供方或模型后,容量与策略会立即变化。若绝对保留预算不小于缩放后的阈值,系统会在目标容量首次允许比较两者时失败。
|
||||
|
||||
同一精确目标覆盖还可以选择摘要提供方/模型、摘要输出上限、收敛重试次数与溢出重试上限。这些都属于压缩问题,不会进入适配器 seam。
|
||||
|
||||
### 目标专用压力错误仍保留可选组合
|
||||
|
||||
缺少容量元数据的适配器仍是有效 LLM 路由。手动主动压力检查会返回目标专用配置错误;自动监听器按精确路由只警告一次,并继续保留完整历史。当已解析容量暴露出无效的绝对保留预算时,系统也按路由抑制重复警告;其他运行故障仍会各自对外可见。提供方已经确认的规范化溢出不需要容量元数据:它绕过主动阈值与普通保留预算,尝试一次最大的平衡缩减,并在替换无法证明进展时保留原始提供方错误。
|
||||
|
||||
## 测试
|
||||
|
||||
服务测试覆盖分离上下文元数据、无效适配器输出、目录独立性与默认缺失行为。适配器测试覆盖 DeepSeek 的配置值、默认值与未列出行为,以及 pi-ai 的精确描述符解析。压缩测试覆盖比例缩放、精确提供方/模型覆盖、加载期拒绝无效合并比例、运行时校验绝对预算、相同模型 id 的提供方切换、目标专用警告抑制与不依赖容量的溢出恢复。Loader fixture 会拒绝已经移除的 token-meter 容量设置,示例则在适配器上配置容量。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
- **把容量与所有策略都放进 compact-basic**——不予采纳,因为 compact-basic 会复制适配器的模型知识,未列出的动态模型需要并行注册,而且未安装压缩时容量也会消失。
|
||||
- **把压缩策略放进各个 LLM 适配器**——不予采纳,因为适配器必须独立于可选消费方,而摘要与重试策略也不是提供方事实。
|
||||
- **让 `listModels()` 成为权威来源**——不予采纳,因为发现能力只是建议信息,一些适配器有意接受动态 id。正确性元数据不能把选择器成员关系变成路由白名单。
|
||||
- **给 token-meter 增加逐模型折叠**——不予采纳,因为回放算法可以共享,变化的只有容量与消费方策略。多个折叠会重复状态,却不会改善估算。
|
||||
- **建立独立模型上下文注册表**——不予采纳,因为适配器已经拥有权威路由解析。第二套注册表会引入生命周期顺序、重复键与漂移问题,却没有独立后端。
|
||||
|
||||
## 后果
|
||||
|
||||
- 容量在提供方 seam 上拥有唯一权威归属方,而压缩策略留在可选消费插件中。
|
||||
- 同一个 compact-basic 实例无需查询发现元数据,就能安全处理不同窗口、提供方切换,以及不同提供方下的相同模型 id。
|
||||
- 仅 LLM 与仅 meter 的组合仍然有效;加载 compact-basic 不会让适配器产生反向依赖。
|
||||
- 使用显式 DeepSeek 模型列表的部署必须为需要主动压力检查的条目提供 `contextWindow`。系统会暴露缺失元数据,而不是静默应用错误的全局回退值。
|
||||
- 比例默认值会随模型自然缩放,同时仍可按精确目标使用绝对保留值,以满足部署专用行为。
|
||||
|
||||
本记录取代[回放式 token 计量服务 Agent Note](2026-07-15-replay-token-meter-service.md) 中的全局容量与无模型策略部分,单折叠计量决策保持不变。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-20-error-cause-chain-diagnostics.md: 391e35997bb1bb050dd2ca620920961d77bb1c46
|
||||
2026-07-20-error-cause-chain-diagnostics.zh.md: 90d6559a9410e8a4e5475db9560a2a177ba7a1a7
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: Render error cause chains at every diagnostic seam
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-20-error-cause-chain-diagnostics.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
A TUI run against an unreachable DeepSeek endpoint failed with the single notice `fetch failed` and no further detail. Two independent gaps produced that dead end:
|
||||
|
||||
1. undici's `fetch` wraps every transport failure (DNS, refused connection, TLS, proxy) in a bare `TypeError: fetch failed` whose actionable detail — `ECONNREFUSED`, `bad port`, the Happy Eyeballs AggregateError — lives on `error.cause`. Every diagnostic seam in the harness rendered only `error.message` (or `String(error)`, which is equivalent for Errors), so the wrapper masked the diagnosis in the TUI notice, the durable `turn/end` reason, and every logger line.
|
||||
2. The readline front door (`dsh-stdio`) rendered no failure reason at all: a `turn/end` with `reason.kind === 'error'` printed nothing but the next `> ` prompt, so the same failure in `demo:repl` was pure silence.
|
||||
|
||||
## Decision
|
||||
|
||||
- `dsh-llm` exports `errorChain(value)`: renders a thrown value with its full `cause` chain (`outer: inner: …`) and AggregateError members (`msg [m1; m2]`), with circular-cause and hostile-coercion containment. It is a diagnostic-surface renderer only; routing stays on `HarnessError.code`.
|
||||
- The DeepSeek adapter wraps a pre-response transport failure in `LlmError('TRANSPORT')` naming the configured `baseURL` and chaining the original rejection as `cause`. An aborted request becomes `LlmError('ABORTED')`; because the turn signal is already aborted, the loop still classifies the turn as cancellation rather than recovery.
|
||||
- Every diagnostic seam renders through `errorChain` instead of `error.message`/`String(error)`: the agent-loop's durable `turn/end` error message (`errorData`), its logger warnings, the TUI's `agent/error` notice and startup-failure line, and `dsh-stdio`'s startup-failure log lines. The per-package `renderThrown` copies in `dsh-agent-loop`, `dsh-stdio`, and `dsh-tui` are deleted in favor of the one shared renderer.
|
||||
- `dsh-stdio` renders failure `turn/end` reasons: `[turn failed <code>] <message>`, `[turn aborted] <reason>`, `[turn rejected] <reason>`, `[turn interrupted by a previous process exit]`, and the output-token-limit notice. Unknown merge-extended kinds fall through as ordinary turn ends.
|
||||
|
||||
`errorChain` lives in `dsh-llm` beside `HarnessError` for the same reason the base class does: it is the leaf package every consumer already imports, so sharing costs no new dependency edge.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Chain rendering inside each error's constructor (bake the cause into `message`).** Rejected: it double-renders once consumers also walk `cause` (the first draft of the adapter fix produced `… fetch failed: bad port: fetch failed: bad port`), and it destroys the structured chain for consumers that want to route on the inner error.
|
||||
|
||||
**A `cause`-aware logger exporter only.** Rejected: the durable `turn/end` reason and the TUI notice are not logger lines; the masked message would persist in the session log — the single durable record of an in-turn failure — and in the primary UI surface.
|
||||
|
||||
**Per-package `renderThrown` upgrades.** Rejected: three packages already carried near-identical private copies; upgrading each separately entrenches the duplication the shared renderer removes.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A transport failure now reads `DeepSeek API request to <baseURL> failed: fetch failed: connect ECONNREFUSED …` in the TUI notice, the readline transcript, and the persisted session log, at the cost of longer diagnostic strings.
|
||||
- Durable `turn/end` error messages include cause detail. Existing snapshot fixtures replay byte-identically because their scripted errors carry no `cause` (for such errors `errorChain(err)` equals `err.message`); only unit-test expectation strings changed. A fixture recorded from a real transport failure would carry the chain.
|
||||
- `errorChain` renders `message` without the class name (`String(error)` rendered `Error: <message>`), so a bare `TypeError` in a log line loses its type label unless its message is empty (then the name is the fallback). The chain detail was judged worth more than the class name at these seams.
|
||||
- `dsh-stdio` output for failed turns is no longer silent; piped consumers that parsed the transcript see new `[turn …]` lines.
|
||||
- Remaining `renderThrown` copies in `dsh-subagent`, `dsh-workflow`, `dsh-skill`, `dsh-workflow-workerthread`, and `cli-demo` still render without the chain; they wrap package-local errors that carry their own messages, and can adopt `errorChain` when their diagnostics prove insufficient.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Agent Note: 在每个诊断接缝处渲染错误 cause 链
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-20-error-cause-chain-diagnostics.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
TUI 连接不可达的 DeepSeek 端点时,失败只显示一条 `fetch failed` 通知,没有任何进一步细节。两个独立缺口共同造成了这个死胡同:
|
||||
|
||||
1. undici 的 `fetch` 把所有传输层失败(DNS、连接被拒、TLS、代理)包装成裸的 `TypeError: fetch failed`,可操作的细节——`ECONNREFUSED`、`bad port`、Happy Eyeballs 的 AggregateError——都在 `error.cause` 上。harness 里的每个诊断接缝都只渲染 `error.message`(或对 Error 等价的 `String(error)`),于是包装层在 TUI 通知、持久化的 `turn/end` reason 和所有日志行里都掩盖了诊断信息。
|
||||
2. readline 前门(`dsh-stdio`)完全不渲染失败原因:`reason.kind === 'error'` 的 `turn/end` 只打印下一个 `> ` 提示符,同样的失败在 `demo:repl` 里就是纯粹的沉默。
|
||||
|
||||
## Decision
|
||||
|
||||
- `dsh-llm` 导出 `errorChain(value)`:渲染抛出值及其完整 `cause` 链(`outer: inner: …`)与 AggregateError 成员(`msg [m1; m2]`),并容错循环 cause 和恶意强制转换。它只是诊断表面的渲染器;路由仍然基于 `HarnessError.code`。
|
||||
- DeepSeek 适配器把拿到响应之前的传输失败包装成 `LlmError('TRANSPORT')`,写明配置的 `baseURL` 并把原始拒绝值链为 `cause`。被中止的请求变为 `LlmError('ABORTED')`;由于轮次信号已处于中止状态,循环仍将该轮次归类为取消而非恢复。
|
||||
- 每个诊断接缝改用 `errorChain` 而非 `error.message`/`String(error)`:agent-loop 的持久化 `turn/end` 错误消息(`errorData`)、其日志警告、TUI 的 `agent/error` 通知与启动失败行、以及 `dsh-stdio` 的启动失败日志行。`dsh-agent-loop`、`dsh-stdio`、`dsh-tui` 里各自的 `renderThrown` 副本被删除,统一使用这一个共享渲染器。
|
||||
- `dsh-stdio` 渲染失败的 `turn/end` reason:`[turn failed <code>] <message>`、`[turn aborted] <reason>`、`[turn rejected] <reason>`、`[turn interrupted by a previous process exit]` 以及输出 token 上限通知。未知的 merge 扩展 kind 按普通 turn 结束处理。
|
||||
|
||||
`errorChain` 与 `HarnessError` 一样放在 `dsh-llm` 里,理由相同:它是每个消费者都已导入的叶子包,共享不增加新的依赖边。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**在每个错误的构造函数里渲染链(把 cause 烤进 `message`)。** 否决:当消费者同时遍历 `cause` 时会双重渲染(适配器修复的第一版产出了 `… fetch failed: bad port: fetch failed: bad port`),并且破坏了想按内层错误路由的消费者所需的结构化链。
|
||||
|
||||
**只做一个感知 `cause` 的日志导出器。** 否决:持久化的 `turn/end` reason 和 TUI 通知不是日志行;被掩盖的消息会留在会话日志——回合内失败的唯一持久记录——以及主要 UI 表面里。
|
||||
|
||||
**逐包升级 `renderThrown`。** 否决:三个包已经各自持有几乎相同的私有副本;分别升级只会固化共享渲染器所要消除的重复。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 传输失败现在在 TUI 通知、readline transcript 和持久化会话日志里显示为 `DeepSeek API request to <baseURL> failed: fetch failed: connect ECONNREFUSED …`,代价是更长的诊断字符串。
|
||||
- 持久化的 `turn/end` 错误消息包含 cause 细节。现有 snapshot fixture 字节级一致地回放,因为其脚本化错误不带 `cause`(对这类错误 `errorChain(err)` 等于 `err.message`);只有单元测试的期望字符串有变化。从真实传输失败录制的 fixture 会携带完整链。
|
||||
- `errorChain` 渲染 `message` 而不带类名(`String(error)` 会渲染 `Error: <message>`),因此日志行里的裸 `TypeError` 会丢失类型标签,除非消息为空(此时回退到类名)。在这些接缝上,链细节被判断为比类名更有价值。
|
||||
- `dsh-stdio` 对失败回合的输出不再沉默;解析 transcript 的管道消费者会看到新的 `[turn …]` 行。
|
||||
- `dsh-subagent`、`dsh-workflow`、`dsh-skill`、`dsh-workflow-workerthread`、`cli-demo` 里剩余的 `renderThrown` 副本仍不渲染链;它们包装的是自带消息的包内错误,等诊断信息证明不足时再采用 `errorChain`。
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-21-compaction-summary-prefix-cache-reuse.md: 490eb57a5891bf9cd0799c5d49d25d4e9838041f
|
||||
2026-07-21-compaction-summary-prefix-cache-reuse.zh.md: 02412ff07e87e12c7e7de00b5c69e1282433f735
|
||||
@@ -0,0 +1,45 @@
|
||||
# Agent Note: The summarization call replays the conversation prefix for KV-cache reuse
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-07-21-compaction-summary-prefix-cache-reuse.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Automatic compaction fires mid-conversation, right after the loop has warmed the provider's KV cache with the last routed request (`system` + `tools` + `messagePrefix` + derived history). The default summarizer then issued a *separate* auxiliary request whose prefix shared nothing with that warm request: a bespoke summarizer `system` prompt followed by the older history flattened to a single rendered transcript string. A provider caches on the request's leading token sequence, so a first token that differs — a different system prompt — invalidates the entire cached prefix. Every compaction therefore paid full prompt-processing cost for the whole replayed history twice: once for the conversation request that tripped pressure, and again for the summarization call, defeating the cache exactly when the conversation is largest.
|
||||
|
||||
## Decision
|
||||
|
||||
The summarization directive moves from the **front** of the request (a fresh `system` prompt) to the **end** of the conversation (the final `user` message). The auxiliary call now reproduces the last routed request's prefix verbatim and appends one trailing instruction, so it is a genuine prefix-extension of the warm request and the provider reuses the cached tokens.
|
||||
|
||||
### `SummarizationInput` carries the replayed prefix, not a rendered string
|
||||
|
||||
`summarize()` (and the internal `summarizeWithLlm`) take a `SummarizationInput` — `{ system?, tools?, messages }` — instead of a flat transcript string. `region.ts` builds it from `session.requestHeader()` (the durable `system`, `tools`, and `messagePrefix`) plus the shadowed region mapped through `session.deriveEventMessage`, which yields byte-identical `Message` objects to what `deriveMessages()` folded into the routed request. `summarizeWithLlm` forwards `system` and `tools` onto `GenerateOptions` and sends `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`. `tools` ride along even though the summarizer never calls one: dropping them would shorten the token sequence and break alignment with the cached request.
|
||||
|
||||
### The instruction is a trailing user message
|
||||
|
||||
`COMPACTION_INSTRUCTION` opens "You are now acting as a compaction engine…" and directs the model to condense *the conversation ABOVE*. It keeps the prior checkpoint's structured headings and adds two rules the front-loaded system prompt did not need in its new position: do not mention the summarization request, and output only the checkpoint text without calling a tool. The shadowed region always ends on a tool-pairing-balanced boundary, so appending a `user` message after it is a valid message ordering for OpenAI-compatible and DeepSeek adapters.
|
||||
|
||||
### Cache reuse is best-effort, correctness is not
|
||||
|
||||
Auto-compaction always anchors at the surface head, so the shadowed region is the head of the routed request and the replayed prefix matches it exactly — the guaranteed-hit case. Manual mid-range `compactRegion` still replays the true prefix and stays correct, but forgoes reuse because its shadowed region is not the request head. A configured `summarizationProvider`/`summarizationModel` that differs from the conversation's route also forgoes reuse; that is the deployment's explicit trade-off, not a defect. Target resolution (configured override → latest routed header → agent options, else throw) is unchanged.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Keep the summarizer system prompt but reuse the rest** — rejected: the system slot is the very first token region a provider caches on, so a distinct summarizer system prompt invalidates the whole prefix regardless of what follows. Only moving the directive off the front recovers the cache.
|
||||
- **Send only the shadowed region without the `system`/`tools`/`messagePrefix` head** — rejected: a shorter or differently-headed sequence still diverges from the cached request at the first token, so it caches no better while losing the framing the summary needs.
|
||||
- **Omit `tools` from the summarization request** (the model never calls one) — rejected: tool schemas are part of the cached token sequence; omitting them misaligns every following token and defeats reuse.
|
||||
- **A dedicated `assistant/chunk`-emitting summarization sub-session for snapshot replay** — out of scope here; the replay gap predates this change and is tracked in the [compaction-seam note](../feature/2026-06-18-compaction-capability-seam.md).
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`dsh-compact-basic`** owns `SummarizationInput`; the protected `summarize(input, agent, signal?)` hook signature changed (acceptable pre-release), and `region.ts` gained `buildSummarizationInput` folding `deriveEventMessage` over the shadowed seqs behind the header prefix.
|
||||
- **Dead render surface removed.** The old flattening path (`renderTranscript` / `renderContentBlocks` and its spec in `dsh-compact`) had no remaining consumer and was deleted with its export.
|
||||
- **README model experience** for `dsh-compact-basic` now documents the auxiliary request as the replayed prefix plus a trailing compaction-instruction message, and its KV-cache effect as reuse of the warm conversation prefix.
|
||||
- **The framed checkpoint output is unchanged**, so the landed `user/message` and every conversation-request snapshot are unaffected; only the auxiliary request's shape changed.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit:** `compact-basic.spec.ts` asserts the auxiliary call forwards `system`/`tools`/leading messages and appends the compaction instruction as the final message, and that `compactRegion` replays the latest routed header prefix. Existing content assertions read the summarizer input through the replayed messages rather than a transcript string.
|
||||
- **Loop:** `compact-loop-repro.spec.ts` classifies the summarization request by the compaction instruction in its trailing user message, and the overflow-recovery tests continue to pin conversation-vs-summary request counts across the real loop.
|
||||
- **Snapshot gap unchanged:** the summarization call still emits no `assistant/chunk` events, so it remains outside keyless replay; the pre-existing gap is owned by the [compaction-seam note](../feature/2026-06-18-compaction-capability-seam.md).
|
||||
@@ -0,0 +1,45 @@
|
||||
# Agent Note: 摘要调用回放对话前缀以复用 KV 缓存
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-07-21-compaction-summary-prefix-cache-reuse.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
自动压缩(compaction)在对话中途触发,恰好在循环用最后一个已路由请求(`system` + `tools` + `messagePrefix` + 派生历史)预热了提供方的 KV 缓存之后。随后默认摘要器发出一个*独立的*辅助请求,其前缀与那个已预热请求没有任何共享部分:一个专门的摘要器 `system` 提示词,后接被拍平成单个渲染后 transcript(文本记录)字符串的较早历史。提供方基于请求起始的 token 序列做缓存,因此第一个 token 只要不同(即一个不同的系统提示词),整个已缓存前缀就会失效。于是每次压缩都要为整段回放的历史付出两次完整的提示词处理成本:一次用于触发压力的对话请求,另一次用于摘要调用,恰好在对话最大时让缓存失去作用。
|
||||
|
||||
## Decision
|
||||
|
||||
摘要指令从请求的**前端**(一个全新的 `system` 提示词)移到对话的**末尾**(最后一条 `user` 消息)。辅助调用现在逐字复现最后一个已路由请求的前缀,并追加一条尾部指令,因此它是已预热请求的真正前缀扩展,提供方会复用已缓存的 token。
|
||||
|
||||
### `SummarizationInput` 携带回放的前缀,而非渲染后的字符串
|
||||
|
||||
`summarize()`(以及内部的 `summarizeWithLlm`)接受一个 `SummarizationInput`(`{ system?, tools?, messages }`)而不是一个扁平的 transcript 字符串。`region.ts` 用 `session.requestHeader()`(持久的 `system`、`tools` 和 `messagePrefix`)加上经 `session.deriveEventMessage` 映射的被遮蔽区域来构建它,后者产出与 `deriveMessages()` 折叠进已路由请求的内容字节级一致的 `Message` 对象。`summarizeWithLlm` 把 `system` 和 `tools` 转发到 `GenerateOptions`,并发送 `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`。`tools` 会一同带上,即便摘要器从不调用任何工具:丢弃它们会缩短 token 序列,破坏与已缓存请求的对齐。
|
||||
|
||||
### 指令是一条尾部 user 消息
|
||||
|
||||
`COMPACTION_INSTRUCTION` 以 "You are now acting as a compaction engine…" 开头,指示模型浓缩*上方的对话*。它保留先前检查点的结构化标题,并在其新位置上新增了两条前置系统提示词此前不需要的规则:不要提及摘要请求,以及只输出检查点文本而不调用任何工具。被遮蔽区域总是结束在工具配对平衡的边界上,因此在其后追加一条 `user` 消息,对 OpenAI 兼容适配器和 DeepSeek 适配器而言是合法的消息排序。
|
||||
|
||||
### 缓存复用是尽力而为,正确性不是
|
||||
|
||||
自动压缩总是锚定在表层头部,因此被遮蔽区域就是已路由请求的头部,回放的前缀与之完全匹配,这就是保证命中的情形。手动的中段 `compactRegion` 仍然回放真实的前缀并保持正确,但会放弃复用,因为它的被遮蔽区域不是请求头部。配置的 `summarizationProvider`/`summarizationModel` 若与对话的路由不同,也会放弃复用;这是部署方明确的权衡,而非缺陷。目标解析(配置的覆盖值 → 最新的已路由 header → agent(智能体)选项,否则抛出)保持不变。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **保留摘要器系统提示词但复用其余部分**——否决:system 槽位正是提供方最先做缓存的 token 区域,因此一个不同的摘要器系统提示词无论后面跟着什么都会使整个前缀失效。只有把指令移离前端才能恢复缓存。
|
||||
- **只发送被遮蔽区域而不带 `system`/`tools`/`messagePrefix` 头部**——否决:更短或头部不同的序列在第一个 token 处仍然与已缓存请求分叉,因此缓存效果并不更好,反而丢失了摘要所需的框架。
|
||||
- **从摘要请求中省略 `tools`**(模型从不调用任何工具)——否决:工具 schema 是已缓存 token 序列的一部分;省略它们会让后续每个 token 失去对齐,破坏复用。
|
||||
- **为快照回放专门建立一个发出 `assistant/chunk` 的摘要子会话**——此处超出范围;该回放缺口早于本次改动,记录在 [compaction-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 中。
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`dsh-compact-basic`** 拥有 `SummarizationInput`;受保护的 `summarize(input, agent, signal?)` 钩子签名发生变化(发布前可接受),并且 `region.ts` 新增了 `buildSummarizationInput`,它在 header 前缀之后对被遮蔽的 seq 折叠 `deriveEventMessage`。
|
||||
- **移除无用的渲染表面。** 旧的拍平路径(`renderTranscript` / `renderContentBlocks` 及其在 `dsh-compact` 中的 spec)已无消费方,连同其导出一并删除。
|
||||
- **README 的 Model Experience** 现在把 `dsh-compact-basic` 的辅助请求记述为回放的前缀加上一条尾部压缩指令消息,并把其 KV 缓存效果记述为复用已预热的对话前缀。
|
||||
- **带框架的检查点输出未改变**,因此落地的 `user/message` 和每个对话请求快照都不受影响;只有辅助请求的形状发生了变化。
|
||||
|
||||
## Testing
|
||||
|
||||
- **单元:** `compact-basic.spec.ts` 断言辅助调用转发 `system`/`tools`/前导消息,并把压缩指令作为最后一条消息追加,且 `compactRegion` 回放最新的已路由 header 前缀。现有的内容断言通过回放的消息而非 transcript 字符串来读取摘要器输入。
|
||||
- **循环:** `compact-loop-repro.spec.ts` 依据摘要请求尾部 user 消息中的压缩指令对其分类,溢出恢复测试则继续在真实循环中固定对话请求与摘要请求的数量。
|
||||
- **快照缺口未变:** 摘要调用仍然不发出 `assistant/chunk` 事件,因此它仍处于无密钥回放之外;这一既有缺口归 [compaction-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 所有。
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Agent Client Protocol (ACP) support — drive the coding agent from external editors
|
||||
# Agent Note: Agent Client Protocol (ACP) support — drive the coding agent from external editors
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -24,7 +24,7 @@ Tool-call presentation remains tool-owned. A tool's `presentCall` and `presentRe
|
||||
|
||||
Permission handling is an answerer on the [user-approval seam](2026-07-06-approval-seam.md), not an ask-every-tool policy in ACP. An `approval/request` for a bridge-owned agent with a call id becomes `session/request_permission` on that agent's editor session, with one-shot allow/reject choices. Foreign or call-less requests delegate; a missing or failed answerer remains fail-closed. The plugin that asks—such as a pre-execute policy or bash escalation—owns the decision to ask.
|
||||
|
||||
The bridge advertises ACP config options instead of session modes. `sandbox-mode` exists only when the mounted bash executor reports sandbox capability, and `approval-policy` exists only when `ctx.approval` is composed. Each option is an independent select whose current value is the session event fold over the composition default. `session/set_config_option` validates against the owning domain vocabulary and writes through `setSandboxMode` or `setApprovalPolicy`. An open-turn switch appends immediately; an idle switch is overlaid in the response and anchored at the next turn start. Until that anchor it is memory-only and a crash reverts to the durable fold. ACP session modes are deliberately not modeled because one mode list cannot represent these orthogonal knobs and config options are the forward protocol surface. Runtime model selection remains outside this decision; `AcpConfig.model` is connection-wide.
|
||||
When `ctx.permission` is composed, the bridge exposes one `permission` select from the deployment's preset table. The shipped `workspace-write` and `danger-full-access` presets each bundle a sandbox mode with an approval policy; unmatched effective knobs produce the switch-away-only `custom` state. `session/set_config_option` validates through `PermissionService.set()` and writes both owning knob events. A switch during an open turn appends immediately; an idle switch is overlaid in responses and anchored at the next `agent/prompt-submit`, before request assembly. Until then it is memory-only, so a crash restores the durable fold. ACP session modes are not modeled because config options are the forward protocol surface; `AcpConfig.model` remains connection-wide.
|
||||
|
||||
The bridge also provides the ACP-backed `UserInteractionProvider`: `ask_user_question` requests become form elicitations on the owning session. Select, multi-select, option descriptions, and custom-answer override semantics are preserved.
|
||||
|
||||
@@ -40,7 +40,7 @@ The precise supported and deferred protocol rows live in [`packages/ui/acp/acp-f
|
||||
|
||||
**Execute bash through ACP `terminal/*`** — rejected. That would move execution outside the harness and bypass its sandbox, credential scrub, task ownership, cwd resolution, and session log. Terminal metadata is presentation only.
|
||||
|
||||
**Represent sandbox and approval as ACP session modes** — rejected. They are independent composable settings, while a single current mode is mutually exclusive. ACP config options represent both without a cross-product and match the protocol's forward direction.
|
||||
**Represent permission presets as ACP session modes** — rejected. The deployment-defined preset is already one config-option select, while session modes are the legacy surface slated for removal in ACP v2.
|
||||
|
||||
**Hijack stdout defensively** — rejected. Process-wide monkey-patching is outside Cordis effect ownership and races the protocol transport. The app composition owns stdout purity.
|
||||
|
||||
@@ -48,9 +48,9 @@ The precise supported and deferred protocol rows live in [`packages/ui/acp/acp-f
|
||||
|
||||
Editors can create, load, prompt, cancel, render, ask, and reconfigure multiple harness sessions over one ACP connection without a loop-specific dependency. The session event log remains the durable source for replay, prompt settlement, cwd, and per-session configuration. Tool presentation and human-answer channels remain extensible plugin contracts instead of ACP-specific behavior.
|
||||
|
||||
The bridge deliberately does not implement session list/delete/resume/close capabilities, MCP passthrough, additional directories, image/audio/embedded-resource prompts, runtime model selection, plans, slash commands, usage updates, editor filesystem delegation, or the ACP terminal execution sub-protocol. The feature checklist records these as unsupported rather than silently accepting them.
|
||||
The bridge deliberately does not implement session list/delete/resume/close capabilities, MCP passthrough, additional directories, image/audio/embedded-resource prompts, plans, slash commands, usage updates, editor filesystem delegation, or the ACP terminal execution sub-protocol. Runtime model selection was added later through standard session config options by the [LLM catalog and ACP selection Agent Note](../architecture/2026-07-15-llm-model-catalog-and-acp-selection.md).
|
||||
|
||||
An idle config selection is truthful in the live response but not durable until the next turn anchors it. Crashing before that boundary loses the pending selection; this is the cost of keeping session events turn-enclosed and replay-safe.
|
||||
An idle config selection is truthful in the live response but not durable until the next `agent/prompt-submit` anchors it inside the open turn. Crashing before that boundary loses the pending selection; this is the cost of keeping session events turn-enclosed and replay-safe.
|
||||
|
||||
## Verification
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Multiplex concurrent ACP sessions over one connection
|
||||
# Agent Note: Multiplex concurrent ACP sessions over one connection
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -8,11 +8,11 @@ An ACP editor can keep several conversations alive over one agent subprocess. A
|
||||
|
||||
## Decision
|
||||
|
||||
The ACP bridge stores live sessions in `Map<SessionId, SessionRecord>` and keeps a `WeakMap<Agent, SessionId>` reverse index for agent-scoped callbacks. A record owns its agent handle, in-flight prompt, live tool-call presentation state, pending idle config switches, session cwd, and client capability snapshot. A separate loading-id set reserves each id before asynchronous resume so two pipelined loads cannot construct duplicate agents; distinct ids may load concurrently.
|
||||
The ACP bridge stores live sessions in `Map<SessionId, SessionRecord>`. Agent-scoped callbacks use `ownedRecord`: look up `agent.session.id` in that forward map and accept the record only when it owns the exact agent object, so a foreign same-id object cannot claim the session. A record owns its agent handle, in-flight prompt, live tool-call presentation state, pending idle config switches, session cwd, and client capability snapshot. A separate loading-id set reserves each id before asynchronous resume so two pipelined loads cannot construct duplicate agents; distinct ids may load concurrently.
|
||||
|
||||
Every `session/event` and `agent/status` callback resolves the owning record before sending or settling anything. Each session permits one in-flight prompt independently. The prompt records a log watermark, captures its own `turn/start`, and settles only on the matching `turn/end`; a late end from a cancelled prior turn cannot resolve a newer prompt. `session/cancel` addresses one record and calls only that agent's queue-aware cancel path.
|
||||
|
||||
Permission ownership uses the same reverse index. The ACP `approval/request` answerer prompts only the editor session that owns the requesting agent and delegates foreign requests. User-interaction elicitations likewise route by agent ownership. Per-session sandbox and approval config values fold only that session's events, with pending idle switches stored on that record until the next turn anchors them.
|
||||
Permission ownership uses the same exact-agent check against the forward map. The ACP `approval/request` answerer prompts only the editor session that owns the requesting agent and delegates foreign requests. User-interaction elicitations likewise route by agent ownership. Per-session sandbox and approval config values fold only that session's events, with pending idle switches stored on that record until the next turn anchors them.
|
||||
|
||||
Background bash tasks carry an opaque owner token equal to the owning session id. `bash_output` and `bash_kill` compare the caller's token with the executor's task ownership before reading or killing; a predictable task id alone grants no access. Ownership is stored with the executor task, so a tool plugin reload does not erase it.
|
||||
|
||||
130
.agents/notes/implemented/feature/2026-06-15-code-mode.md
Normal file
130
.agents/notes/implemented/feature/2026-06-15-code-mode.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Agent Note: Code Mode — the model writes TypeScript against the tool registry
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
In the registry's native presentation, the agent loop advertises every visible capability as a JSON-schema function definition. `ToolRegistry` contributes its schemas to the system-prompt assembly, the assembly's `tools` land on the wire (and in the logged request header), the model invokes one `tool-call` block per step, and the loop dispatches each call through `ctx.tools.execute()` **sequentially** (parallel tool execution is an explicit open TODO in `dsh-tools` and [docs/architecture.md](../../../../docs/architecture.md)), with **every** intermediate `tool-result` re-entering the model's context on the next request.
|
||||
|
||||
For multi-step tool work this is token-heavy and serial. The model cannot compose tools — loop over a result set, branch on an intermediate value, fan out, post-process — without a full model round-trip per call, and each round-trip drags the entire intermediate result back into context whether the model needs it or not.
|
||||
|
||||
Cloudflare's [Code Mode](https://blog.cloudflare.com/code-mode/) proposes an alternative grounded in a simple observation: LLMs are better at writing code than at emitting tool calls, because they have seen millions of lines of real code and comparatively few contrived tool-calling traces. Instead of one tool call per step, the model writes a TypeScript program against a generated API over the tools, the program executes in a sandboxed runtime, and the model curates what comes back — only what it prints or returns — instead of every intermediate result.
|
||||
|
||||
Tool presentation belongs to the registry that owns tool visibility: implementing a second presentation as an after-the-fact waterfall transform would make correctness depend on listener order and fight [reconstructable requests](../architecture/2026-07-05-reconstructable-requests.md). The execution substrate is also part of the foundation rather than a placeholder: Node `worker_threads` provides a separate isolate, an empty environment, heap caps, and termination of a hot synchronous loop, while fitting the harness's existing trust model (§Trust posture).
|
||||
|
||||
## Decision
|
||||
|
||||
Three decisions, each elaborated in its own section below:
|
||||
|
||||
1. **Code Mode is a first-class presentation mode of `ToolRegistry`** (`dsh-tools`), selected by a validated `mode` config: `'native'` (the default, contributing the visible capability schemas), `'code'` (the registry contributes only its reserved `run_code` transport plus a generated SDK `.d.ts` in the system prompt), or `'both'` (native schemas and the transport + SDK). The registry shapes its canonical contribution at the source; the cooperative prompt-assembly result remains authoritative, and the logged request header records exactly that returned presentation.
|
||||
2. **Code execution is a capability seam** — `packages/code-runtime/` contains the interface package `@deepseek-ai/dsh-code-runtime`, which owns `ctx.codeRuntime` ([capability seams](../architecture/2026-06-13-capability-seams.md); consumer = `dsh-tools`, with core-consumes-a-seam precedent in `agent-loop` → `dsh-llm`). The runtime knows nothing about tools: it is handed a program and named async bindings, runs the program, and reports `{ value, logs, error? }`. Language and substrate are backend properties, so a future Python or container backend is another implementation package, not a redesign.
|
||||
3. **The shipped implementation is `@deepseek-ai/dsh-code-runtime-worker`**: one fresh Node worker thread per run, executing the model's TypeScript after type-strip, with bindings bridged over the message port, an empty environment, configurable heap/output/time caps, and hard termination. Its trust posture is bash-equivalent by design — no unsafe-acknowledgement flags — because the harness already ships `dsh-bash-local`, which executes arbitrary model-written shell commands with strictly *more* ambient authority.
|
||||
|
||||
### The registry owns the mode
|
||||
|
||||
`ToolRegistry` gains a schemastery-validated config (`static Config`), its first: `mode: 'native' | 'code' | 'both'`, default `'native'`. A deployment flips it from `cordis.yml` (`tools: { mode: code }`) — no code edit, per the no-hardcoded-tunables convention.
|
||||
|
||||
**Wire tool list.** The registry contributes visible capabilities in `'native'`, only `run_code` in `'code'`, and both in `'both'`. The final `PromptAssembly.tools` list is logged in the request header. `run_code` is a reserved presentation transport outside registration and restriction layers; direct prompt providers and the assembly waterfall remain responsible for their own contributions.
|
||||
|
||||
**Interaction with `toolOrder`, stated up front:** a configured `systemPrompt.toolOrder` naming native capabilities rejects every assembly under `mode: 'code'`, because those names are outside that mode's wire-validation universe. This is correct behavior, not a bug: a deployment using Code Mode updates its order config or drops it.
|
||||
|
||||
**SDK prompt section.** In `'code'` and `'both'`, the lazy `tools:sdk` section in the tool-guidance order band renders TypeScript declarations plus fixed usage instructions for the scope's visible capabilities. It shares lookup and execution visibility, excludes `run_code`, and sorts tools lexicographically for byte-stable output.
|
||||
|
||||
**Assembly ownership.** `run_code` and `tools:sdk` enter the trusted `system-prompt/assemble` waterfall as normal assembly inputs. A scoped `tools:sdk` section may shadow the global default before dispatch, and a listener may remove or replace either contribution. The waterfall's returned assembly is final, so whoever changes these inputs owns preserving a viable Code Mode protocol when the deployment expects Code Mode to remain usable; no restoration pass overrides deliberate composition.
|
||||
|
||||
**Codegen.** `jsonSchemaToTs()` maps the `defineTool` JSON-Schema subset to TypeScript, carries schema descriptions into JSDoc, and degrades unsupported constructs to `unknown`. The SDK exposes tools as quoted object keys, supporting arbitrary names without aliases or collisions. Typing is advisory because the runtime strips types before execution.
|
||||
|
||||
### The run_code tool and the dispatch bridge
|
||||
|
||||
Under `'code'` and `'both'` the registry owns `run_code` as a reserved presentation transport with one required parameter, `{ code: string }`. It is represented by a normal `ToolDefinition` for dispatch but stays outside the filterable capability layers, so restrictions cannot accidentally remove Code Mode's only entry point. Calls traverse the complete tool pipeline — `tools/pre-execute` → monotonic guards → `tools/execute` around dispatch → `tools/post-execute` → immutable `tools/result` notification — exactly like native calls; a permission plugin can inspect the program text before it runs, and final-result observers see the normalized outer outcome. Its `execute(args, exec)`:
|
||||
|
||||
1. **Build bindings.** One run-scoped signal follows outer cancellation and is aborted whenever the run settles. Each visible tool binding JSON-normalizes its arguments—rejecting lossy values before dispatch—waits on the serialization queue, executes with a deterministic call id and the outer token as `parent`, defers returned contexts through the outer execution, and logs `tool/code-dispatch`. Successful text becomes a string and non-text blocks become placeholders; tool errors reject the binding promise. Every sub-call retains its own immutable execution identity and traverses the full tool pipeline.
|
||||
2. **Runs the program**: `ctx.codeRuntime.run({ program: args.code, bindings: [{ global: 'tools', functions }], signal: runController.signal })`. The runtime receives the run-scoped signal, not only the caller's outer signal, so any way the outer run settles also aborts work inside the runtime.
|
||||
3. **Settle after quiescence.** When the runtime settles, the bridge aborts outstanding work and drains the dispatch queue before returning. Success returns captured output and presentation metadata. A runtime failure becomes `CodeRunFailedError`; backend rejection uses the registry's normal error boundary. Both produce structured error results, and no sub-call can append after `run_code` settles.
|
||||
|
||||
**Sub-call contexts are deferred through the parent.** Injecting inside `run_code` would break parent call/result adjacency, so `ToolRunContext.deferContext()` collects every sub-result `additionalContexts` entry in dispatch order. The registry carries that array even when the program later throws, and the loop appends each entry only after the outer result and every sibling result in the step. An outer post-execute block discards tool-deferred entries and exposes only contexts explicitly attached by the blocking decision.
|
||||
|
||||
**Concurrency is serialized.** Each run owns a dispatch queue, so even `Promise.all` executes tool calls in submission order. Settlement abandons queued calls that have not started. Parallelism requires per-tool concurrency-safety metadata.
|
||||
|
||||
**Presentation.** `run_code`'s render intent is decided here per the [render-intent Agent Note](../architecture/2026-07-02-tool-render-intent-union.md): `presentCall` → a `generic` card, `kind: 'execute'`, title = the program text, `rawInput` = the same program text; `presentResult` → a `generic` card whose content is the captured output (from `meta`). The program is the title because ACP execute cards reliably render that field while some clients omit body and raw-input content. This is not a `terminal` card: that card's semantics are "a shell command in a working directory", which a program is not.
|
||||
|
||||
### Observability: `tool/code-dispatch`
|
||||
|
||||
Each sub-dispatch appends a log-only `tool/code-dispatch` event containing parent and child call ids, tool identity, normalized arguments, and result summary. It remains outside model history but available to persistence and UIs. Appends occur inside the open `run_code` turn. Direct executions without an agent still run but cannot log the event.
|
||||
|
||||
### The code-runtime seam
|
||||
|
||||
`packages/code-runtime/code-runtime/` — `@deepseek-ai/dsh-code-runtime`, depending only on `cordis`. An abstract `CodeRuntime extends Service` (`super(ctx, 'codeRuntime')`) plus the vocabulary:
|
||||
|
||||
- `CodeRunRequest = { program: string; bindings: CodeBindingNamespace[]; signal?: AbortSignal }`
|
||||
- `CodeBindingNamespace = { global: string; functions: Record<string, (args: unknown) => Promise<unknown>> }` — the runtime exposes each namespace as a global object of async functions inside the program; binding arguments and resolutions must be structured-cloneable (a runtime may cross a serialization boundary; ours does).
|
||||
- `CodeRunResult = { value?: unknown; logs: CodeLogEntry[]; error?: CodeRunFailure }` — program execution outcomes, including exception, timeout, abort, and worker exit, resolve as the `error` field. `run()` may reject only for caller/seam misuse (for example a duplicate binding namespace); consumers still contain a non-conforming backend rejection at their own error boundary.
|
||||
- `CodeLogEntry = { source: 'console' | 'stdout' | 'stderr'; level?: 'log' | 'info' | 'warn' | 'error' | 'debug'; text: string }`
|
||||
- `CodeRunFailure = { kind: 'exception' | 'timeout' | 'abort' | 'worker-exit'; message: string }` — orthogonal outcomes reported independently per [defensive patterns](../../../../docs/defensive-patterns.md); a timed-out run is not an exception, an abort is not a timeout.
|
||||
- Two readonly backend descriptors, informational not gating: `language` (what the program must be written in — `'typescript'` for the shipped backend; a Python backend would say so, and pair with its own SDK generator on the presentation side) and `isolation` (`'worker-thread'` for the shipped backend; `'process'`, `'container'`, … for future ones). `dsh-tools` requires `language === 'typescript'` in the MVP — its codegen emits TS — and fails the assembly loudly otherwise, the same misconfiguration idiom as `toolOrder` violations (as when `mode` is non-native with no `ctx.codeRuntime` loaded at all).
|
||||
|
||||
Requests contain every runtime input; implementations own validated timeout and cap defaults. The registry looks up the optional runtime only when Code Mode is assembled, so native mode does not depend on one. Missing or language-incompatible runtimes fail loudly. Alternate substrates or languages can replace the implementation behind the same seam, paired with the appropriate SDK generator.
|
||||
|
||||
### The worker-thread runtime
|
||||
|
||||
`@deepseek-ai/dsh-code-runtime-worker`, the second package of the `packages/code-runtime/` group. Per `run()`:
|
||||
|
||||
1. **Type-strip host-side** with Node's built-in `stripTypeScriptTypes` (`node:module`; present across the repo's whole engines range, `^22.19.0 || >=24.0.0`, and position-preserving, so runtime error line numbers match the model's source). Strip-only mode rejects non-erasable syntax (`enum`, namespaces) — that rejection returns as `error.kind: 'exception'` with Node's message, the SDK instructions say "erasable TypeScript only", and the model self-corrects like any other program error. A syntax-level failure never spawns a worker.
|
||||
2. **Spawn one fresh `Worker` per run** from the package's own bootstrap module: `env: {}` (truly empty — stronger than the scrubbed-env rule for spawned commands), `resourceLimits` from config, `stdout`/`stderr` captured into `logs` rather than inherited. No pooling and no cross-run state: a program's world dies with its worker, which keeps runs reconstructable from the log alone and makes state bleed unrepresentable.
|
||||
3. **Execute** in the bootstrap: the stripped program becomes the body of an `AsyncFunction` whose parameters are the binding globals and a capturing `console` shim, so top-level `await` and `return` work and the program's completion value is the run's `value` (structured-cloneable values cross as-is; anything else is replaced by its `util.inspect` rendering, documented).
|
||||
4. **Bridge bindings over the message port**: each binding function in the worker posts `{ id, global, name, args }` and awaits the reply; the host validates the name against the request's bindings, invokes, and replies `{ id, ok, value }` or `{ id, ok: false, message }` (a host-side binding rejection becomes a program-side rejection). The worker-side namespace objects are built null-prototype via `defineProperty`, so a binding named `__proto__`, `constructor`, or `toString` is an ordinary own property, not a prototype collision. Unknown names, duplicate ids, and post-settlement messages are rejected or ignored — the port protocol assumes a hostile peer, because the peer runs model code.
|
||||
5. **Enforce independent budgets.** `computeMs` meters worker busy time, allowing slow awaited tools without excusing a hot loop. `maxWallMs` bounds total elapsed time, including unresolved waits. Expiry, cancellation, and completion terminate the worker. Heap exits and truncation are reported explicitly; compute, wall, heap, log, and return-value caps are validated configuration.
|
||||
6. **Dispose to quiescence**: the service's own disposal terminates in-flight workers and *awaits* their exits before resolving, per [defensive patterns](../../../../docs/defensive-patterns.md).
|
||||
|
||||
### Trust posture
|
||||
|
||||
The worker runtime provides containment, not a security boundary: model code can reach Node APIs and has authority comparable to the bash tool. `worker.terminate()` stops the thread but not OS processes it spawned. Code Mode uses the same `tools/pre-execute` policy gate as bash and adds an empty environment, heap limits, a separate isolate, and hard termination of the program itself. Deployments that need a hard multi-tenant boundary need a container-class backend for both code and bash; the runtime's isolation descriptor lets them distinguish that backend.
|
||||
|
||||
### What the model sees
|
||||
|
||||
The SDK instructs the model to write an async erasable-TypeScript body, call tools through `await tools.name(args)`, catch rejected tool calls when needed, and return or log only the output that should re-enter context. Calls remain sequential even under `Promise.all`. The declaration prefix can be as large as native schemas, especially in `'both'`, but remains stable for provider caching.
|
||||
|
||||
## Consequences
|
||||
|
||||
Deployments switching to `'code'` must update any native-only `toolOrder`. Assembly listeners own the integrity of any rewritten protocol surface. Sub-dispatch remains serialized, while per-call contexts retain their source, envelope, and metadata through the outer result.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Worker runtime:** Real-worker tests cover output and value capture, failure kinds, compute and wall budgets, hostile binding traffic, empty environment, structured-clone fallback, output caps, and disposal to quiescence. A built-package test runs the worker entry under plain Node.
|
||||
- **Registry integration:** Tests cover code generation, all presentation modes, reserved-name and restriction rules, scoped visibility, authoritative assembly rewrites, `toolOrder`, runtime compatibility failures, full-pipeline sub-dispatch, parent-token correlation, serialization, cancellation and queue drain, JSON normalization, error propagation, log events, ordered context deferral across successful and failed programs, outer-block suppression, and HMR cleanup.
|
||||
- **With-key e2e:** A real model composes two bash calls in one program; another discovers nested workspace instructions through a Code Mode fs dispatch. The tests verify collapsed request headers, correlated dispatch events, resulting files, deferred context, and model behavior.
|
||||
- **Snapshot:** The `code-mode-turn`, `both-mode-turn`, and `code-mode-workspace-context` fixtures pin SDK text, header tool lists, dispatch events, deferred context, and result cards.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**An add-on consumer plugin with zero core changes.** Rejected because `agent/request` is call-config-only under [reconstructable requests](../architecture/2026-07-05-reconstructable-requests.md), while transforming an assembled tool list would have to undo `toolOrder` canonicalization without owning its config and would depend on listener order. Which tools the model is offered, and in which representation, is the registry's single concern: native schemas and the SDK are two projections of one visible store.
|
||||
|
||||
**`node:vm` as the reference runtime, with hardening deferred.** Rejected: `node:vm` is not isolation (prototype-chain escapes reach the host realm) and cannot interrupt a hot loop. A worker thread provides a separate isolate, empty environment, `resourceLimits`, and reliable `terminate()` at bash-equivalent trust, so the reference and production implementation are one package without an unsafe-acknowledgement ceremony.
|
||||
|
||||
**Result elision / summarization over native tool-calling.** Addresses only the context-bloat half of the problem: trimming old `tool-result`s is cheap to add as a logged surface replacement under reconstructable requests, but still pays one model round-trip per call and cannot express loops, branches, or joins. Complementary, not competing; it can layer under Code Mode for residual native calls.
|
||||
|
||||
**Parallel native dispatch in the loop.** The other answer to round-trip cost; still valid future work (the open TODO), still blocked on concurrency-safety metadata, and still no composition — it parallelizes calls the model already decided on in one step. Code Mode's serialized-queue decision keeps the two compatible: when the metadata lands, both native parallel dispatch and per-tool binding parallelism unlock together.
|
||||
|
||||
**Always-exclusive (Cloudflare-faithful, no mode).** Rejected for this SDK's primary consumer: a coding agent's bread-and-butter single calls (`bash`, `read`, `edit`) are already ideal as native calls, and forcing every edit through a program taxes the common case. The mode config keeps the faithful form (`'code'`) one line away without imposing it.
|
||||
|
||||
**Per-tool visibility tiers (this tool native, that tool code-only).** Deferred: it needs per-tool metadata and a presentation split that `'native' | 'code' | 'both'` does not, and its design depends on evidence about how models split usage under `'both'`.
|
||||
|
||||
**Sanitized identifier aliases in the SDK** (`my-tool` → `my_tool`, Cloudflare's approach). Rejected: quoted keys on a `declare const` make every name reachable with zero alias-collision logic; models handle `tools["my-tool"](…)` fine.
|
||||
|
||||
**A REPL-style persistent kernel** (state survives across `run_code` calls). Rejected for the MVP: cross-call state would be invisible to the session log, breaking the reconstructability guarantee that every request is a pure function of the log; fresh-per-run keeps it. A kernel-style backend remains expressible behind the seam later, with its own logging story.
|
||||
|
||||
## Risks
|
||||
|
||||
**The worker is not a hard security boundary.** Deliberate and documented (§Trust posture): posture equals the existing bash tool, containment exceeds it, gating uses the same seams. Deployments needing more need a future `isolation: 'container'` backend — tracked as the seam's designed extension, not a TODO on this design.
|
||||
|
||||
**`stripTypeScriptTypes` is marked experimental.** It is the same engine (amaro/swc) behind Node's own native `.ts` execution, exposed as an API across this repo's whole engines range. Mitigations: the runtime's unit suite pins the behaviors relied on (position preservation, erasable-only rejection message shape loosely), the call sits behind one private function, and `amaro`/`sucrase` are drop-in replacements if the API shifts. The erasable-only subset is a model-facing contract line, and the error path is a working feedback loop, not a dead end.
|
||||
|
||||
**Prompt cost of the SDK, especially under `'both'`.** The `.d.ts` can rival the native schemas it complements; `'both'` carries two representations. Prefix stability + provider caching amortize per-session cost; the mode is per-deployment; the Agent Note makes no unconditional-savings claim. Measured guidance (when to prefer which mode) is explicitly post-ship learning.
|
||||
|
||||
**Registry scope growth.** `dsh-tools` absorbs codegen, a tool, a bridge, and an event. Contained by module boundaries inside the package (`ts-types.ts`, `code-mode.ts` beside `schema.ts`/`json-schema.ts`/`presentation.ts`) and by the seam: everything substrate-shaped lives behind `ctx.codeRuntime`.
|
||||
|
||||
**Structured-clone values can exceed JSON.** Tool bindings therefore JSON-normalize arguments before dispatch, ensuring every executed call can be logged. The lower-level runtime keeps its wider port contract, while stricter consumers validate at their boundary. Non-text sub-results become placeholders.
|
||||
|
||||
**Serialized-only sub-dispatch.** `Promise.all` gains no wall-clock parallelism yet, only fewer round-trips; models may over-expect. The instructions state it; lifting it is tied to the same concurrency-safety metadata the native parallel-dispatch TODO needs.
|
||||
|
||||
**Budget metering reads the event loop, not a flag.** Busy-time polling (`eventLoopUtilization()`) is coarser than an exact CPU meter — a budget expires up to one poll interval late — and its correctness claim ("a pending dispatch cannot pause it") is load-bearing against a hostile program. Both sides are unit-tested (hot loop with a pending decoy dispatch dies at `computeMs`; idle-on-slow-binding survives to `maxWallMs`), and the poll interval is an internal constant, not config — nothing a deployment could mis-tune into a bypass.
|
||||
@@ -1,12 +1,12 @@
|
||||
# RFC: Filesystem tool schemas — model-facing read/write/edit shapes
|
||||
# Agent Note: Filesystem tool schemas — model-facing read/write/edit shapes
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
[The filesystem capability-seam RFC](../architecture/2026-06-17-filesystem-capability-seam.md) defines the filesystem capability seam (`ctx.fs`), the package split (`dsh-fs`, `dsh-fs-local`, `dsh-tool-fs`, plus the `dsh-fs-policy` policy plugin), and the observed-file/stale-version policy for read-before-write/edit checks — which the [split-fs-seam](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [event-gate](../architecture/2026-06-26-file-context-as-event-gate.md) RFCs moved off `ctx.fs` into the `dsh-fs-policy` plugin on the `fs/*` event gate. The remaining decision for the first filesystem tool delivery is the model-facing schema surface: what arguments the model sees for `read`, `write`, and `edit`.
|
||||
[The filesystem capability-seam Agent Note](../architecture/2026-06-17-filesystem-capability-seam.md) defines the filesystem capability seam (`ctx.fs`), the package split (`dsh-fs`, `dsh-fs-local`, `dsh-tool-fs`, plus the `dsh-fs-policy` policy plugin), and the observed-file/stale-version policy for read-before-write/edit checks — which the [split-fs-seam](../simplification/2026-06-26-fsspec-style-fs-seam.md) and [event-gate](../architecture/2026-06-26-file-context-as-event-gate.md) Agent Notes moved off `ctx.fs` into the `dsh-fs-policy` plugin on the `fs/*` event gate. The remaining decision for the first filesystem tool delivery is the model-facing schema surface: what arguments the model sees for `read`, `write`, and `edit`.
|
||||
|
||||
The schema should be small enough to implement in the first `dsh-tool-fs` pass, but stable enough that future local/remote/sandboxed filesystem backends do not require model-facing churn. It should also avoid importing every option from reference systems. Claude Code and OpenCode expose similar core file tools but differ in naming style and extra flags; this RFC chooses the minimal shared surface for the prototype.
|
||||
The schema should be small enough to implement in the first `dsh-tool-fs` pass, but stable enough that future local/remote/sandboxed filesystem backends do not require model-facing churn. It should also avoid importing every option from reference systems. Claude Code and OpenCode expose similar core file tools but differ in naming style and extra flags; this Agent Note chooses the minimal shared surface for the prototype.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -103,8 +103,8 @@ Schema tests pin the required/optional argument set per tool, empty-`old_string`
|
||||
|
||||
## Consequences
|
||||
|
||||
**The first schema is intentionally smaller than Claude Code's.** Dropping PDF pages, multimodal read, rich grep/list flags, and expected hash fields keeps the implementation focused, but users may ask for those quickly. They arrive as separate RFCs or focused follow-ups rather than overloads of the initial schema.
|
||||
**The first schema is intentionally smaller than Claude Code's.** Dropping PDF pages, multimodal read, rich grep/list flags, and expected hash fields keeps the implementation focused, but users may ask for those quickly. They arrive as separate Agent Notes or focused follow-ups rather than overloads of the initial schema.
|
||||
|
||||
**No explicit model-facing stale guard in v1.** The schema does not ask the model to provide an expected hash/version. That is intentional: stale checks come from backend-produced versions and the `dsh-fs-policy` plugin's observed state, not from fragile model-copied tokens. Filesystem safety failures surface through structured `FsError` codes owned by `dsh-fs`, not through model-supplied version fields.
|
||||
|
||||
**Naming becomes public surface.** Once shipped, changing `file_path` to `filePath` or `old_string` to `oldString` would churn prompts, examples, and downstream clients. This RFC chooses snake_case up front and treats it as the stable model-facing contract.
|
||||
**Naming becomes public surface.** Once shipped, changing `file_path` to `filePath` or `old_string` to `oldString` would churn prompts, examples, and downstream clients. This Agent Note chooses snake_case up front and treats it as the stable model-facing contract.
|
||||
@@ -1,12 +1,12 @@
|
||||
# RFC: Rich ACP bash rendering — the terminal card via the `_meta` convention
|
||||
# Agent Note: Rich ACP bash rendering — the terminal card via the `_meta` convention
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The ACP bridge lets each tool own its call rendering via `presentCall`/`presentResult` (see [tool-call UI presentation](../../implemented/feature/2026-06-14-acp-agent-client-protocol.md) and `packages/core/tools`). For `bash` we surface the exact command as the `tool_call` title, the model's `description` as a content text block, `kind: 'execute'`, and the completed output wrapped in a fenced ` ```console ` text block.
|
||||
The ACP bridge lets each tool own its call rendering via `presentCall`/`presentResult` (see [tool-call UI presentation](2026-06-14-acp-agent-client-protocol.md) and `packages/core/tools`). For `bash` we surface the exact command as the `tool_call` title, the model's `description` as a content text block, `kind: 'execute'`, and the completed output wrapped in a fenced ` ```console ` text block.
|
||||
|
||||
That is a correct, capability-free baseline, but not how the reference editors render a *terminal* tool at its best. An editor like Zed has a dedicated terminal tool-call card — a header showing the working directory, the command as the label, the command output rendered as a terminal, and an exit-status pill — but it only builds that card when the `tool_call` carries terminal metadata (below). With a plain text block the output appears as static markdown and there is no cwd header. (Zed also HIDES `rawInput` for `kind: 'execute'`, which is why the command IS the title — both reference adapters do the same. The human-readable description rides as a separate content block above the card; note this is a DELIBERATE divergence — claude-agent-acp DROPS the description in terminal mode and renders only the card — we keep the summary visible alongside.)
|
||||
Reference editors render terminal metadata as a dedicated card with cwd, command, live-style output, and exit status; plain text loses that structure. The command is the title because execute cards hide raw input, while the human-readable description remains a separate block above the card.
|
||||
|
||||
## Key finding: agent-executed terminals use a `_meta` convention, NOT `terminal/create`
|
||||
|
||||
@@ -43,4 +43,4 @@ Keep `dsh-bash` agent-side execution; render the terminal card via the `_meta` c
|
||||
|
||||
## Out of scope / non-goals
|
||||
|
||||
The text-block baseline stays the no-capability default. Two follow-ups are deliberately NOT built here and would each warrant their own RFC when someone takes them on: **live incremental streaming** (`_meta.terminal_output_delta` as chunks arrive, which needs an incremental-output seam on `dsh-bash`), and **command classification** (parsing a `cat`/`sed` as a `read` card with a file location, a `grep` as a `search`, etc., falling back to the terminal card — display-only, must never change what executes).
|
||||
The text-block baseline stays the no-capability default. Two follow-ups are deliberately NOT built here and would each warrant their own Agent Note when someone takes them on: **live incremental streaming** (`_meta.terminal_output_delta` as chunks arrive, which needs an incremental-output seam on `dsh-bash`), and **command classification** (parsing a `cat`/`sed` as a `read` card with a file location, a `grep` as a `search`, etc., falling back to the terminal card — display-only, must never change what executes).
|
||||
@@ -0,0 +1,126 @@
|
||||
# Agent Note: Compaction as a capability seam (abstract contract + basic backend)
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
A long-running agent conversation grows without bound. As the event log accumulates turns, the derived message history eventually approaches the model's context window — the model then truncates mid-response (`max-tokens`) or degrades. **Compaction** is the mitigation: replace a run of older history with a concise summary, keeping recent context intact.
|
||||
|
||||
The [session surface](../architecture/2026-06-18-session-surface.md) was built as the foundation for exactly this — an ordered projection over the event log with a `surfaceOp: { op: 'replace', start, end }` operation purpose-built to shadow a range of entries and insert a replacement, with `sourceEventSeqs` recording provenance so the decision replays deterministically. What remained was the plugin that *decides what to compact and produces the summary*.
|
||||
|
||||
Two forces shape the design. First, compaction policy and reusable token measurement vary independently: measurement belongs to the LLM-family [`ctx.tokenMeter` service](../architecture/2026-07-15-replay-token-meter-service.md), while summarization can be a model call, a template, or a remote service. Second, `SurfaceEventType` is closed to five event types (`user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`); only those may carry `surfaceOp`. A bespoke `compaction/*` event therefore **cannot** itself appear on the surface — the compiler and Session's always-on append/seed boundary reject `surfaceOp` on it.
|
||||
|
||||
## Decision
|
||||
|
||||
### Compaction is a capability seam, split interface / implementation
|
||||
|
||||
Per the [capability-seams Agent Note](../architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently:
|
||||
|
||||
1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
|
||||
2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that consumes `ctx.tokenMeter` and owns the tail→head retention walk, summarization via `ctx.llm.stream()`, the surface replacement, the lock, post-step pressure, and canonical context-overflow recovery. `summarize()` is its sole subclass hook; pricing and replay stay with the meter.
|
||||
3. **Model-free companion** — `@deepseek-ai/dsh-compact-tool-result-prune`: a concrete optional service that rewrites oversized current `tool/result` nodes before the backend selects a summary range. It is not a second compaction implementation and does not implement `CompactService`.
|
||||
4. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
|
||||
|
||||
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
|
||||
|
||||
The capability-seams Agent Note states the interface package "depends only on cordis" (true of `dsh-bash`, whose vocabulary is self-contained). Compaction **cannot** honor that: its verbs act on an agent-owned `Session` (`compactRegion(start, end, agent)`) and its output uses the content vocabulary (`CompactionResult.summary: ContentBlock[]`). There is no way to express the contract without naming `Session`/`SessionEvent` (from `dsh-session`) and `ContentBlock` (from `dsh-llm`).
|
||||
|
||||
This is not a coupling smell — it is the contract's domain. The "only cordis" guidance was always shorthand for "the interface depends only on what the contract genuinely names, and never on an implementation." `dsh-session` and `dsh-llm` are themselves interface/vocabulary packages, not implementations; `dsh-compact` still imports no backend. The seam's real invariant — *consumers and implementations evolve independently behind an abstract service* — holds intact.
|
||||
|
||||
### Abstract `compactIfNeeded` / `compactRegion`, algorithm in the backend
|
||||
|
||||
An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface. That recouples the contract to one strategy: a backend that wants a different retention policy or event sequence would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend and keeps the interface a statement of *what*. Token measurement is not a compaction hook at all; the singleton service lets multiple consumers share one per-session replay fold.
|
||||
|
||||
`compactIfNeeded(agent, trigger, signal)` takes an explicit `'pressure' | 'context-overflow'` trigger and cancellation. It reads only the latest durable routed request; no header means no work, while any routed provider/model target uses the singleton estimator. `compactRegion(start, end, agent, signal?)` uses `agent.session` as its single session identity and keeps an optional signal for manual callers. The default summarizer resolves its target from explicit config, the latest logged routed target, then agent options, and records the provider/model pair after any `llm/stream` routing. It replays the routed request's prefix and appends the compaction directive as a trailing user message so the provider's warm KV cache is reused — see the [summary prefix-cache Agent Note](../bug-fix/2026-07-21-compaction-summary-prefix-cache-reuse.md).
|
||||
|
||||
### Automatic pressure runs after successful durable step work
|
||||
|
||||
Successful-call pressure cannot run at pre-step because final `agent/request` routing, provider output, tool results, buffered context, and steering do not exist there. Serial `agent/post-step(agent, turn, step, signal)` fires after those facts are durable and before `step/end`. `dsh-compact-basic` measures the canonical logged request through `ctx.tokenMeter`, so the next request sees any replacement without a speculative envelope override. Once pressure qualifies, optional `ctx.toolResultPrune` rewriting runs before summary selection; compact-basic remeasures the durable surface and skips summarization if pruning restores safe pressure.
|
||||
|
||||
Canonical provider context overflow takes a separate path. The failed step closes, `agent/request-error` receives the original request error and consecutive retry count, and compact-basic prunes before forcing one useful balanced reduction. It returns retry only if `session.surface.replaceGeneration` increases, including pruning-only progress when no summary range exists; the loop then opens a new numbered step and reconstructs its request from the durable log. No replacement, a recovery failure before any replacement, cancellation, an exhausted cap, or an unrelated error preserves the original provider failure. If pruning already advanced the generation before later summary work fails, recovery retries from that durable pruned surface unless cancellation or disposal wins. The complete lifecycle decision is in the [after-call recovery Agent Note](../architecture/2026-07-10-after-call-compaction-pressure-and-overflow-recovery.md).
|
||||
|
||||
```
|
||||
assistant/message → tool/result/context/steering
|
||||
await serial agent/post-step ⟵ pressure compaction inside the successful step
|
||||
step/end
|
||||
|
||||
provider overflow → step/end
|
||||
await waterfall agent/request-error ⟵ forced compaction between attempts
|
||||
retry → next numbered step/start ⟵ derives from the replacement surface
|
||||
```
|
||||
|
||||
### Retention is turn-agnostic; tool-pairing balance is the only structural guard
|
||||
|
||||
Auto-compaction checks after **every successful** step, not once per turn. This is load-bearing for runaway-turn survival: a tool-heavy ReAct turn appends an `assistant/message` + a `tool/result` per step, so the surface grows within a turn. The post-step check can compact early closed tool pairs before continuation opens the next step, and provider-confirmed overflow remains the backstop when a request crosses the limit first.
|
||||
|
||||
`compactIfNeeded` retains the smallest tail of whole surface units whose estimated size reaches the resolved retained-token budget and compacts older nodes. A unit is a complete closed step or one no-step message. If the token cutoff lands inside a step, retention expands until the cut is tool-pairing balanced. Balance is checked on surface order, not log sequence, because replacement summaries have new sequence numbers at old surface positions. `dsh-compact` exports the before/after edge helpers; their per-session cache folds only appended surface-tail nodes while `replaceGeneration` is unchanged, does no event reads for log-only growth, and rebuilds current membership and balances after replacement. `compactRegion` rejects boundaries that split a tool call from its result. The in-flight turn receives no special retention.
|
||||
|
||||
A runaway turn thus compacts exactly like any other history: its early *closed* steps get summarized while its recent steps stay verbatim. When the only compactable content left is an un-splittable open tail step (its tool-calls have no results yet), compaction declines (`null`) and retries once that step closes.
|
||||
|
||||
**Some single-unit overflow remains out of scope.** Summary range selection cannot split an indivisible unit. The optional pruner can repair a closed tool pair when removable text-bearing tool-result content is the bulk and the pruned remainder fits. Envelope-only pressure, an oversized indivisible non-tool node such as a pasted `user/message`, and a tool unit whose non-prunable remainder is still oversized remain outside compaction; bounding those units is a separate concern.
|
||||
|
||||
### Head-anchoring: one auto checkpoint, always at the head
|
||||
|
||||
Auto-compaction always starts at the surface head, merging the prior checkpoint with newly compacted history so only one automatic checkpoint remains. `shadowedRange` is therefore positional rather than a numeric sequence interval: a newer summary sequence may occupy an older surface position. `shadowedSeqs` records the authoritative surface order. Manual mid-range compaction may leave multiple checkpoints.
|
||||
|
||||
### Approximate convergence invariant
|
||||
|
||||
`resolveConfig` supplies usable defaults: threshold ratio `0.8`, retained-tail ratio `0.16`, empty summarization provider/model overrides, `maxTokens: 8192`, `compactionRetries: 1`, `maxOverflowRetries: 1`, and `auto: true`. Optional exact provider/model policies partially override the top-level defaults; pressure scales ratios against capacity from the route-owning LLM adapter, while `retainTokens` can replace ratio retention. Retention must remain below the resulting threshold. Convergence remains dynamic because provider output caps can be spent on hidden or surfaced reasoning tokens and summary size is unpredictable. If pressure remains over threshold, `compactIfNeeded()` re-compacts the head checkpoint up to the configured retry count, but each committed summary must be smaller than what it shadows. Overflow needs no capacity metadata and bypasses threshold and retained-tail policy for one maximal balanced head reduction, leaving the newest indivisible unit. The ownership split is specified by the [routed model context and compaction policy Agent Note](../architecture/2026-07-20-routed-model-context-and-compaction-policy.md).
|
||||
|
||||
### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary
|
||||
|
||||
Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the (framed) summary and whose `sourceEventSeqs` covers the shadowed entries *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance). The surface mutation sits **inside** the lock — `compact/end` is the last event appended:
|
||||
|
||||
```
|
||||
compact/start → log-only. Acquires the lock.
|
||||
[summarize older range via the backend]
|
||||
compact/summary → log-only. Provenance: raw summary, range, shadowed seqs, token count.
|
||||
user/message → surfaceOp { op:'replace', start, end }. THE surface mutation (framed summary).
|
||||
deriveMessages() renders it as a user-role message.
|
||||
compact/end → log-only. Releases the lock (carries `error` on a recoverable failure).
|
||||
```
|
||||
|
||||
`deriveMessages()` then yields `[summary_as_user_message, ...retained_entries]`. Reusing `user/message` is honest rather than a workaround: a summary genuinely *is* user-role context.
|
||||
|
||||
### Checkpoint framing + incremental merge (backend-private)
|
||||
|
||||
The basic backend wraps the summary as established checkpoint context and tags it for incremental merging on the next cycle. The raw summary remains on `compact/summary`. Framing is backend policy; the seam promises only that one replacement user message carries the possibly framed summary.
|
||||
|
||||
### Blocking via a log-recorded lock, plus a crash/recoverable failure taxonomy
|
||||
|
||||
The `compact/start … compact/end` bracket is justified, in order of what now does the work:
|
||||
|
||||
1. **Crash-detectable orphan + provenance** (primary). Summarization is a slow model call persisted *after* `compact/start`. A crash mid-summarization leaves a `compact/start` with no matching `compact/end` — a detectable orphan. Releasing the lock last (rather than first) converts the crash window from *silent corruption* into that detectable orphan.
|
||||
2. **Prevents concurrent compaction.** `compactRegion` refuses to start if the current turn holds an unmatched `compact/start`. (The loop is single-threaded across either awaited automatic seam, so this is also a re-entry tripwire — a thrown "already in progress" signals a real bug.)
|
||||
|
||||
Two failure paths, both documented:
|
||||
|
||||
- **Crash** (the loop dies mid-summarization): a dangling `compact/start`, no closer. Because `compact/*` are **log-only**, the orphan is **inert** — no summary replacement lands. The derived surface remains the durable surface present at `compact/start`: full history when pruning made no replacement, or the already-pruned history when it did. Generic turn-repair (`interruptedTurnClosers`) closes the turn with a synthetic `turn/end`; the orphan sits *before* that `turn/end`, so the turn-scoped in-progress check never sees it and a crash cannot wedge future compaction.
|
||||
- **Recoverable** (summarization throws but the loop survives): the backend appends `compact/end` with its **`error`** field set and lands no summary replacement. Post-step pressure warns and continues from the latest durable surface — full history if no replacement preceded the attempt, or the pruned surface if pruning already landed. Overflow recovery delegates only before any replacement; generation progress from earlier pruning authorizes a retry from that durable surface unless cancellation or disposal wins.
|
||||
|
||||
`compact/end` keeps its `error?` field (mirroring `tool/result`'s self-contained error — one event tells success from failure without correlating a sibling). There is no separate `compact/error` event.
|
||||
|
||||
**Core session repair stays compaction-agnostic — deliberately.** `interruptedTurnClosers` is never taught about `compact/*`. Teaching it would force every future `xxx/start … xxx/end` plugin pair to patch a core module — exactly the coupling the capability-seam architecture exists to avoid. Because the log-only orphan is inert, no special repair is needed: generic turn-repair plus the inertness of an un-landed surface mutation is sufficient.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **The full algorithm as concrete interface methods** — rejected because it recouples the contract to one retention strategy. Both core methods are abstract; reusable measurement is a separate LLM-family service and `summarize()` is basic's sole hook.
|
||||
- **Compaction on `agent/request` or provisional `agent/pre-step` inputs** — rejected because neither proves the final durable request and both couple generic lifecycle to compaction-specific envelope data. Post-step replay plus canonical overflow recovery covers both successful and rejected calls.
|
||||
- **A separate `compact/error` event** — rejected: `compact/end` keeps an `error?` field, mirroring `tool/result`'s self-contained error — one event tells success from failure without correlating a sibling.
|
||||
- **Teaching core turn-repair about `compact/*`** — rejected: the log-only orphan is inert, and a core module patched for every future `xxx/start … xxx/end` plugin pair is exactly the coupling the capability-seam architecture exists to avoid.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Packages**: `packages/compact/compact` supplies the interface, `compact-basic` supplies the backend, and `compact-tool-result-prune` supplies optional deterministic rewriting. `packages/llm/token-meter` owns replay-aware measurement independently. The consumer tier is deferred.
|
||||
- **Automatic seams**: `agent/post-step` (`@mode serial`) handles successful-call pressure and `agent/request-error` (`@mode waterfall`) handles final request failures after the failed step closes. Generic `agent/pre-step` remains a four-argument checkpoint with no compaction-only prompt/prefix payload.
|
||||
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
|
||||
- **`dsh-compact`** owns `toolPairingBalancedBefore(session, seq)` and `toolPairingBalancedAfter(session, seq)`, the cached surface-edge checks that `compactRegion` and `compactIfNeeded` use to avoid splitting a tool-call/result pair. The cache validates current membership by seq and answers both edges from one per-cut balance sequence; stale or missing seqs and orphan results reject.
|
||||
- **`dsh-session`** validates positional replacement, complete provenance, and content-only single-node `tool/result` rewrites through its one surface manager. Its invariant companion treats fresh appended tool results as executions that require an open step and pending call; validated replacements remain turn-enclosed rewrites.
|
||||
- **Wiring**: `examples/tui-agent/cordis.yml` loads zero-config `dsh-token-meter`, `dsh-compact-tool-result-prune`, then `dsh-compact-basic`; service-wide defaults make the composition usable without repeated numeric policy.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit:** Real Loader and invariant plugins cover whole-unit retention, pruning configuration and replay, rich-block ordering, metadata preservation, convergence, both `compact/end` outcomes, open-tail refusal, pruning-only and summarized overflow recovery, generation proof, caps, and original-error preservation.
|
||||
- **Loop:** Tests pin post-step after durable tool results and before `step/end`, actual `agent/request` routing, closed failed steps, fresh retry numbering, and complete thrown/in-band overflow → compaction → reconstructed retry composition.
|
||||
- **With-key e2e:** A real model and bash session with lowered limits triggers compaction, records a complete `compact/start…end` pair, shrinks the surface, and finishes the task.
|
||||
- **Snapshot gap:** Runaway-turn compaction cannot yet replay because the summarization call records no `assistant/chunk` events or `sessionId`; interleaved summarization-call replay remains follow-up work.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Agent Note: Subagent capability seam
|
||||
|
||||
Status: implemented
|
||||
|
||||
> The full seam is shipped: the `dsh-subagent` interface and `dsh-tool-subagent` consumer; the two in-process backends (`dsh-subagent-spawn`, `dsh-subagent-fork`); the nested-agent snapshot infrastructure ([per-session snapshot replay](../testing/2026-06-22-subagent-snapshot-replay.md)); and the out-of-process `dsh-subagent-acp` backend ([its Agent Note](2026-06-22-acp-subagent-backend.md)).
|
||||
|
||||
## Problem
|
||||
|
||||
The harness has a long-deferred seam for **subagents** — an agent delegating work to another agent. The intent was sketched in the `Agent`/`AgentLoop` interfaces ([packages/core/agent/src/types.ts](../../../../packages/core/agent/src/types.ts), [packages/core/agent-loop/src/index.ts](../../../../packages/core/agent-loop/src/index.ts)): a creation option referencing a parent agent (fork = seed the child session with the parent's event log; spawn = fresh session), with the child returned as an `Agent` handle so steering and event subscription work uniformly. This Agent Note realizes that seam; the banner above lists what shipped.
|
||||
|
||||
The distinctive requirement — the one that shapes the whole design — is that **multiple subagent implementations must coexist at runtime**. A parent may want a cheap in-process child for a scoped subtask AND an isolated out-of-process child (over ACP) in the same session. The transports we foresee:
|
||||
|
||||
- **in-process** — a child concrete `Agent` on the same `Context` (the cheapest, and nearly free given the existing agent factory);
|
||||
- **ACP** — act as an ACP *client* driving another agent process (which can be another instance of ourselves);
|
||||
- later: **A2A**, the **Codex app-server**, and the **Claude Code Agent SDK** — each the same out-of-process "start a child, prompt it, stream updates, cancel" shape as the ACP backend.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Why not the bash seam shape
|
||||
|
||||
The bash seam ([capability seams](../architecture/2026-06-13-capability-seams.md)) registers exactly one `BashExecutor` per context; loading a second throws. That is correct for bash (one machine, one way to run a command) but wrong here: coexistence is the requirement. So the subagent service is a **named-provider registry** — each implementation registers under a unique name and a caller picks one by name — mirroring the **LLM adapter registry** (`LlmService.registerAdapter`), not the single-service bash executor. The seam is still three-package (interface / implementation / consumer); only the "one vs. many implementations" axis differs.
|
||||
|
||||
## Decision
|
||||
|
||||
### The three-package seam
|
||||
|
||||
A new package group `packages/subagent/`:
|
||||
|
||||
| Package | Role |
|
||||
|---|---|
|
||||
| `@deepseek-ai/dsh-subagent` | interface: `SubagentService` (`ctx.subagents`), `SubagentProvider`, `SubagentRun`, the request/result/capability vocabulary, the `subagent/*` events |
|
||||
| `@deepseek-ai/dsh-subagent-spawn` | implementation: a fresh in-process child via `ctx.agents.create` |
|
||||
| `@deepseek-ai/dsh-subagent-fork` | implementation: an in-process child seeded with a snapshot of the parent's log |
|
||||
| `@deepseek-ai/dsh-subagent-acp` | implementation: an ACP client driving a configured child process |
|
||||
| `@deepseek-ai/dsh-tool-subagent` | consumer: the model-facing `subagent` tool over `ctx.subagents` |
|
||||
|
||||
### The primitive: async `start → SubagentRun`
|
||||
|
||||
A provider exposes `start(request) → Promise<SubagentRun>`. Fulfillment publishes a ready child and transfers its run handle to the caller. One signal covers cancellation before and after readiness; `dispose()` cancels remaining work and awaits quiescence. A rejected start cleans partial resources and emits no lifecycle event. `start` is transport-neutral; `spawn` names only the fresh in-process backend.
|
||||
|
||||
### Two kinds of optional capability, discovered two ways
|
||||
|
||||
- **Start-time features** (`outputSchema`, `depthLimit`, `toolFilter`, `persona`) ride on a static `provider.capabilities` descriptor. The service checks every requested one BEFORE delegating and **rejects loud** (`SubagentError('UNSUPPORTED_CAPABILITY')`) if the provider lacks it — never accepted-then-ignored. They must be checked before a run exists, which is why they cannot be runtime methods.
|
||||
- **Runtime features** (steering via `sendMessage`, follow-up via `resume`) are **optional methods** on `SubagentRun`. The method's presence IS the capability, and TypeScript narrowing is the discovery mechanism: a consumer cannot call an absent method without narrowing first, so there is no silent-degradation path and no separate flags object to keep in sync.
|
||||
|
||||
### Fork vs. fresh are separate backends, not a flag
|
||||
|
||||
Fresh and forked children are separate providers, not a request flag. `dsh-subagent-spawn` starts an isolated child; `dsh-subagent-fork` seeds a balanced prefix containing only completed parent turns. The in-flight turn is excluded because its subagent call has no result yet and cannot form valid replay history.
|
||||
|
||||
### Child isolation and the parent log
|
||||
|
||||
Each subagent runs in its **own `Session`** (own id, `parentSession` lineage), persisted independently. The parent's log records only the spawn `tool/call` and its `tool/result` (the child's final output) — the child's internal steps and tool calls stay in the child's own session, never injected into the parent log. This is the only design that is identical across transports: an ACP child's internal events physically cannot be injected into our parent log, so making in-process behave the same keeps the seam transport-agnostic.
|
||||
|
||||
### Synchronous collect (first cut)
|
||||
|
||||
`dsh-tool-subagent` passes its execution signal to `start()`, awaits the child result, and disposes the run in `finally`. Non-completed outcomes become error results rather than successful partial output. This foreground consumer does not use the run's optional steering method.
|
||||
|
||||
### Provider selection is config, not model-facing
|
||||
|
||||
`dsh-tool-subagent` binds to exactly one provider name (`Config.provider`); the model sees only `{ description, prompt }`. To expose more than one transport, load the tool plugin more than once, each bound to a different provider and a distinct `toolName` (the tool registry rejects a duplicate name). The *service* holds the multi-provider registry; the *tool* picks one — no provider/type parameter in the schema this cut.
|
||||
|
||||
## Testing
|
||||
|
||||
Registry and tool tests replace only the nondeterministic child boundary with a package-local scripted provider while exercising the real `SubagentService`, lifecycle, task integration, and model-facing tool. Provider and consumer export shapes retain their Loader regression coverage for the failure described in [postmortem 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md). Registry tests cover reload safety, duplicate names, and start-time capability rejection; nested-agent scenarios replay keylessly through [per-session snapshot replay](../testing/2026-06-22-subagent-snapshot-replay.md); in-process backends also have real-loop unit tests and a with-key e2e.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Recursion.** Without a bound, an in-process child can see the delegation tool and recurse. The in-process backends implement the optional absolute depth limit and scoped live-global `toolFilter`; ACP advertises both capabilities off and rejects such a request. The [subagent composition-controls Agent Note](2026-07-12-subagent-persona-tool-filter-and-depth.md) owns their exact semantics and security limits.
|
||||
- **Blocking the parent turn.** Foreground collection holds the parent's step open for the child's full duration. Background delegation uses the shared `ctx.tasks` runtime and generic `task_*` tools, the same collection mechanism as background bash; the subagent seam itself remains task-agnostic.
|
||||
- **Live progress.** This cut surfaces only lifecycle + final result; a per-chunk child→parent update stream is deferred with the background redesign.
|
||||
- **ACP client surface.** Proxying `fs`/`terminal` from the ACP child back to the parent (a shared-workspace mode) is future work; the first cut advertises neither, so the child self-serves in its own process.
|
||||
@@ -1,10 +1,10 @@
|
||||
# RFC: ACP subagent backend (out-of-process delegation)
|
||||
# Agent Note: ACP subagent backend (out-of-process delegation)
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The subagent seam ([the seam RFC](2026-06-21-subagent-capability-seam.md)) was built so multiple backends coexist by name on `ctx.subagents`. The in-process backends (`-spawn`/`-fork`) run a child as a second `Agent` on the SAME cordis context — cheap, but the child shares the parent's process, model client, and tools. The seam's whole point was to also support an OUT-OF-PROCESS child reached over a protocol, proving the abstraction generalizes across a process boundary. This RFC adds the first such backend: an Agent Client Protocol (ACP) client.
|
||||
The subagent seam ([the seam Agent Note](2026-06-21-subagent-capability-seam.md)) was built so multiple backends coexist by name on `ctx.subagents`. The in-process backends (`-spawn`/`-fork`) run a child as a second `Agent` on the SAME cordis context — cheap, but the child shares the parent's process, model client, and tools. The seam's whole point was to also support an OUT-OF-PROCESS child reached over a protocol, proving the abstraction generalizes across a process boundary. This Agent Note adds the first such backend: an Agent Client Protocol (ACP) client.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -16,11 +16,15 @@ Each `start` spawns a new child, runs exactly one ACP session (`initialize` →
|
||||
|
||||
### Minimal client stub
|
||||
|
||||
The client advertises NO optional capabilities (no `fs`, no `terminal`): the child self-serves file/terminal access in its own process. `session/update` notifications are consumed — the backend accumulates `agent_message_chunk` text as the result output and ignores the rest (thoughts, tool-call cards) in this cut, which surfaces only the child's final answer. `session/request_permission` is auto-answered by a configured policy (`reject` declines every prompt, `allow` approves via the first allow-shaped option) — the first cut surfaces no prompt to a human. Proxying `fs`/`terminal` back to the parent (a shared-workspace mode) remains future work, as the seam RFC noted.
|
||||
The client advertises NO optional capabilities (no `fs`, no `terminal`): the child self-serves file/terminal access in its own process. `session/update` notifications are consumed — the backend accumulates `agent_message_chunk` text as the result output and ignores the rest (thoughts, tool-call cards) in this cut, which surfaces only the child's final answer. `session/request_permission` is auto-answered by a configured policy (`reject` declines every prompt, `allow` approves via the first allow-shaped option) — the first cut surfaces no prompt to a human. Proxying `fs`/`terminal` back to the parent (a shared-workspace mode) remains future work, as the seam Agent Note noted.
|
||||
|
||||
### No start-time capabilities
|
||||
|
||||
The provider's `capabilities` are all `false`. An out-of-process child cannot honor the parent's `maxDepth` (it has no access to `parent.options.subagentDepth`) or `toolFilter` (it owns its own tool registry), and the first cut does not implement `outputSchema`. The service rejects a request needing any of them before `start` runs. The backend injects only `subagents` (not `ctx.agents`) and ignores `request.parent`.
|
||||
The provider's `capabilities` are all `false`. An out-of-process child cannot honor the parent's `maxDepth` (it has no access to `parent.options.subagentDepth`) or `toolFilter` (it owns its own tool registry), and the first cut does not implement `outputSchema`. The service rejects a request needing any of them before `start` runs. The backend injects only `subagents` (not `ctx.agents`); the ONE thing it reads off `request.parent` is the session header's cwd (see the workspace resolution below) — no conversation context, depth, or tool state crosses the process boundary.
|
||||
|
||||
### Workspace cwd resolution
|
||||
|
||||
The child's working directory is an explicit resolution, never the harness process cwd: the deployment `cwd` override when configured (made absolute against the launch directory and validated at load), else the parent session header's cwd (validated at start), and a loud rejection before anything spawns when neither exists. One ACP server process serves sessions from many workspaces, so `process.cwd()` cannot stand in for a session's workspace — the old implicit fallback ran children in the server's launch directory. A candidate must be an absolute path naming a directory the harness can ENTER (`X_OK` — `statSync().isDirectory()` alone accepts a mode-600 directory that spawn would fail with EACCES), and the same resolved path becomes both the subprocess cwd and the ACP `session/new` workspace.
|
||||
|
||||
### StopReason mapping
|
||||
|
||||
@@ -32,17 +36,16 @@ The child is a separate process, so it inherits an environment. Credential-shape
|
||||
|
||||
## Testing
|
||||
|
||||
Designed at every tier the backend touches, per the root AGENTS.md rule that a new capability shape names its coverage at every tier at plan time:
|
||||
|
||||
- **Keyless unit/integration** (`subagent-acp.spec.ts`): spawns a scripted mock ACP server subprocess (`tests/mock-acp-server.ts`) and drives it through the real backend over real ACP stdio. Coverage includes the prompt round-trip and output accumulation; every StopReason mapping; cancellation through the required request signal and through disposal; already-aborted and cancel-races-ahead-of-newSession starts; a torn pipe after cancellation settling `aborted`; permission auto-answer under both policies; non-message updates; nonexistent-command startup failure with process reaping; provider HMR; and the namespace export shape.
|
||||
- **With-key e2e** (`subagent-acp.e2e.ts`): the harness drives ITSELF — the backend spawns the real `acp-agent` example process and a real model in that child answers a prompt (PONG) and does real file work (writes `proof.txt`, verified on disk). Self-skips without `DEEPSEEK_API_KEY`. This is the "talk to our own process" smoke and the out-of-process analogue of the in-process spawn e2e.
|
||||
- **Snapshot**: deferred as `TODO(acp-subagent-replay)`. An ACP child is a distinct replay shape — each child is its own PROCESS with its own single-agent replay (booted under `DSH_SNAPSHOT=replay` with its own sessions-root + fixture), unlike the in-process per-session keying that [the per-session replay RFC](../testing/2026-06-22-subagent-snapshot-replay.md) added. The keyless mock-server tests give deterministic coverage of the backend in the meantime; the snapshot follow-up would record the parent driving a real-but-replayed ACP child.
|
||||
- **Keyless unit/integration:** A scripted ACP subprocess exercises real stdio for prompt/output flow, every stop-reason mapping, signal and disposal cancellation (including pre-abort, pre-session race, and torn-pipe cases), both permission policies, ignored non-message updates, missing-command cleanup, provider reload, and namespace exports.
|
||||
- **Keyless Loader composition:** A test-only cordis.yml boots the stdio app through the real Loader with the backend's `cwd` omitted; a scripted model delegates once and the scripted child proves it ran in — and was announced — the parent session's workspace (the cwd-inheritance branch end to end).
|
||||
- **With-key e2e:** The backend spawns the real ACP example; its model answers `PONG`, writes `proof.txt`, and the parent verifies the file.
|
||||
- **Snapshot gap:** Each ACP child is a separate process with its own replay session, unlike in-process per-session replay. Deterministic mock-server coverage exists, while `TODO(acp-subagent-replay)` tracks parent replay against a replaying child.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Why not the 0.28.x SDK bump?
|
||||
### Why stay on SDK 0.25.1?
|
||||
|
||||
The plan proposed bumping `@agentclientprotocol/sdk` 0.25.1 → 0.28.x for the new fluent `acp.client()` / `ActiveSession.nextUpdate()` API. Validating that against the code (the AGENTS.md "RFC is a proposal, not golden truth" discipline) reversed the decision: the backend only needs `ClientSideConnection` + `ndJsonStream` + `PROTOCOL_VERSION` + the `Client`/`Agent`/`StopReason` types, **all present and non-deprecated in 0.25.1**. The fluent API and `unstable_forkSession` that motivated the bump are never used here, so the "cleaner client code" benefit did not materialize. Worse, 0.28.x **deprecates both** `ClientSideConnection` AND `AgentSideConnection` (it wants all callers on the fluent builders), which turns the `no-deprecated` lint red across the entire existing ACP layer — 33 usages including the server bridge this backend has no business rewriting. That cross-cutting connection-API migration is its own change, not baggage for "add an ACP subagent backend". So the bump was reverted and the backend is written against 0.25.1 (the plan's own fallback clause: "if the bump proves disruptive, fall back to `ClientSideConnection` (0.25.1), which is sufficient"). Migrating the whole ACP layer to the fluent API on a later 0.28.x bump is a worthwhile standalone follow-up.
|
||||
The backend needs only `ClientSideConnection`, `ndJsonStream`, `PROTOCOL_VERSION`, and the client protocol types, all supported in 0.25.1. The 0.28 fluent API would require migrating both client and server connection classes across the ACP layer without improving this backend, so that upgrade remains a separate change.
|
||||
|
||||
### Why not a persistent child process?
|
||||
|
||||
@@ -54,4 +57,4 @@ Every run pays a fresh subprocess (spawn + `initialize` + `newSession`). The par
|
||||
|
||||
## Future providers
|
||||
|
||||
The same out-of-process spawn/prompt/stream/cancel shape generalizes to other transports named in the seam RFC — A2A, the Codex app-server, and the Claude Code Agent SDK — each a sibling provider registered by name. The ACP backend is the proof that the seam supports the boundary; those are mechanically similar.
|
||||
The same out-of-process spawn/prompt/stream/cancel shape generalizes to other transports named in the seam Agent Note — A2A, the Codex app-server, and the Claude Code Agent SDK — each a sibling provider registered by name. The ACP backend is the proof that the seam supports the boundary; those are mechanically similar.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Agent Note: Workspace context instruction files
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
Repository guidance such as `AGENTS.md` belongs in a coding session's effective context so project conventions, build commands, and review rules arrive without repeated user pasting. The stdio and ACP products need the same behavior, isolated by session cwd: a global system-prompt section leaks one workspace's files into another live ACP session.
|
||||
|
||||
Neighboring products establish useful conventions but differ in details. Codex treats `AGENTS.md` as native, Claude Code uses `CLAUDE.md` and familiar system-reminder-style user context, and opencode supports both names with one winner per directory plus lazy nested discovery. The harness needs cross-tool compatibility without loading duplicate or contradictory files from the same scope.
|
||||
|
||||
The lifecycle has two distinct classes of content. The initial applicable chain is stable enough to live in the request prefix and benefit from provider prefix caching. Nested files, edits, candidate switches, and removals happen after the session starts and belong in durable append-only history rather than the frozen prefix.
|
||||
|
||||
## Decision
|
||||
|
||||
The implementation lives in `packages/context/workspace-context` as `@deepseek-ai/dsh-workspace-context`. It is a request-context extension, not a core service or a filesystem backend. `@deepseek-ai/dsh-agent-core` mounts it for both product front doors and forwards its config. The plugin consumes `agent/session-prefix`, `tools/post-execute`, and the optional `ctx.fs` capability.
|
||||
|
||||
The plugin does not statically inject `fs`. Providerless product trees therefore boot normally and the plugin no-ops until a filesystem provider exists. All production reads go through that provider. Candidate probes call `lstat` before `resolve`, so a repository-owned final-component symlink is rejected rather than followed outside the workspace. The session-prefix signal and dynamic tool execution signal propagate through resolution, metadata probes, and streaming reads, so cancellation does not wait for an unrelated filesystem scan. Once `lstat` identifies a regular-file winner, a provider exception or disagreement during resolve/stat is classified as unavailable: it is neither interpreted as a deletion nor allowed to fall through to a lower-priority candidate.
|
||||
|
||||
### File Names And Precedence
|
||||
|
||||
The default per-directory candidate list is `['AGENTS.md', 'CLAUDE.md']`. The list is configurable as `instructionFileCandidates`, and `AGENTS.md` is an ordinary first candidate rather than a hidden priority. In one directory, only the first existing regular-file candidate loads. With defaults, `AGENTS.md` is native and `CLAUDE.md` is a compatibility fallback.
|
||||
|
||||
Candidate entries are same-directory file names. Empty entries, `.`/`..`, and entries containing `/` or `\` are ignored. Lowercase names, local variants, and other same-directory names can be opted into explicitly; rule directories and import semantics are outside this contract.
|
||||
|
||||
The user-global file is fixed at `$DSH_HOME/AGENTS.md` and is not affected by `instructionFileCandidates`. `$DSH_HOME` defaults to `~/.dsh`, matching the harness-level home role of `~/.codex` or `~/.claude` rather than introducing a plugin-specific home. Tilde expansion and the default live in `dsh-paths` so future harness features share the same convention.
|
||||
|
||||
### Baseline Prefix
|
||||
|
||||
On the first request of an agent-loop instance, the plugin contributes one user-role message through `agent/session-prefix`. It loads the user-global file first, then finds the project root by walking upward from `agent.session.header.cwd` to a configured root marker (default `.git`), then loads one candidate from each directory from the root to the cwd. A `.git` file and a `.git` directory are both valid markers, covering linked worktrees and submodules. Without a marker, the cwd itself is the root.
|
||||
|
||||
The plugin prepends its contribution before `await next()` returns, so session-prefix contributions appear in plugin registration order. In the product spine workspace instructions are registered before a skills catalog and therefore appear first. The loop deep-freezes the composed prefix, logs it in `EpochHeader.messagePrefix`, and reuses it verbatim for that instance. It is request state, not `Session.deriveMessages()` history.
|
||||
|
||||
A resumed agent creates a new loop instance and recomposes the baseline from current files, with the new prefix anchored by the resume request header. This permits current baseline content on resume without mutating a prefix already used by an earlier instance.
|
||||
|
||||
The baseline is a user-role `<system-reminder>` with `Instructions from: <path>` sections and explicit authority and precedence language. This familiar model-facing frame avoids a harness-specific XML vocabulary. Project paths are root-relative and the user-global path is `~/.dsh/AGENTS.md` for the default home or `$DSH_HOME/AGENTS.md` for a configured home. A literal `</system-reminder>` inside file content is escaped. The package README owns the exact current [prompt shape](../../../../packages/context/workspace-context/README.md#prompt-shape).
|
||||
|
||||
### Dynamic Discovery And Refresh
|
||||
|
||||
After a successful first-party `read`, `write`, or `edit` call, the `tools/post-execute` listener reconciles the touched descendant chain and every scope already known to the session. A newly reached scope is returned through `additionalContexts` for the next request using an `Additional instructions from: <path>` system-reminder. Under Code Mode, `run_code` defers sub-dispatch contexts onto its outer result, so the same update is appended only after the parent result rather than being injected mid-call.
|
||||
|
||||
A content edit appends `Updated instructions from: <path>`, states that the new content replaces the previous content, and includes the complete current file. If precedence changes from one candidate to another, the message also names the previous path and says it no longer applies. If no candidate remains, the plugin appends `Instructions removed: <path>` and states that the previously loaded instructions no longer apply.
|
||||
|
||||
Dynamic messages carry their complete system-reminder framing in `content`, and every `context/message` reaches the model verbatim as a user-role message (there is no core wrapper to opt out of). `context/message.meta` carries opaque JSON state that is persisted but never rendered to the model.
|
||||
|
||||
Shell commands are not discovery triggers. Local bash calls start fresh shells, and inferring reached paths from arbitrary command strings would require shell semantics the prompt plugin does not own.
|
||||
|
||||
### Duplicate Suppression And Change Detection
|
||||
|
||||
Every dynamic workspace context event stores versioned metadata with `{ action, scope, path, previousPath?, digest? }`, where `digest` is SHA-1 over the loaded content. The model-facing prompt has no HTML comments, hidden markers, or headings that are parsed back into state.
|
||||
|
||||
At reconciliation time the plugin scans plugin-owned `context/message` events and derives the latest state for each visible scope. A short per-session pending map begins only after the immutable top-level `tools/result` proves an `additionalContexts` entry survived every post-execute listener, then covers the interval before the loop appends that context to the log. Each entry records the open `{ turn, step }`: an equal durable `context/message` at or after its sequence boundary confirms and removes it, while a matching `step/end` arriving first means the loop discarded its context buffer, so the plugin removes both the pending entry and its version-cache fast path. A nested Code Mode result stages its changes under the parent's opaque execution token so repeated sub-dispatches in one run do not duplicate them; the parent result rolls that provisional state back and commits only contexts retained by outer policy.
|
||||
|
||||
An unchanged path and digest is suppressed. A logged removal is a tombstone, so a reappearing candidate becomes a new `set`. Resume works from persisted metadata. If compaction removes an instruction event from the visible surface, that state no longer suppresses a later load, matching the fact that the model can no longer see it. Only changes actually included under the byte budget enter metadata or pending state, so an omitted file remains eligible on a later touch.
|
||||
|
||||
The frozen baseline keeps an in-memory path/digest map for comparison. A later successful filesystem touch appends baseline edits or removals as dynamic messages; it never rewrites the prefix. During resumed prefix composition the plugin also reconciles visible dynamic scopes, so nested changes made while the agent was offline can append an update before the first resumed request.
|
||||
|
||||
There is intentionally no watcher. Detection occurs at the next successful structured filesystem touch or resumed prefix composition. A provider failure produces no removal; absence is only accepted when all configured candidates in that scope were probed successfully.
|
||||
|
||||
### Byte Budget And Bounded Reads
|
||||
|
||||
`maxBytes` is required and applies separately to a rendered baseline or one dynamic reconciliation batch; there is no implicit or unbounded render budget. Non-positive and non-finite values disable loading. When content exceeds the budget, broader files are omitted before the most-specific file is truncated. A visible `Workspace instruction budget ...` notice names omitted and truncated paths and byte counts, and output never exceeds the configured bytes.
|
||||
|
||||
`maxSourceBytes` is a positive per-file cap with a 1 MiB default. The loader checks reported size before reading and still consumes content through `streamText()` with a running UTF-8 byte count, so missing/stale metadata cannot force an unbounded allocation. An oversized winning candidate is unavailable rather than a reason to fall through to another same-directory name. The plugin deliberately keeps no process-wide cache and never retains instruction prose. It keeps only `{ path, version, digest }` per effective scope in a `WeakMap<Session, Map<scope, state>>`: a matching provider `FsVersion` plus matching effective prompt state skips the read, while a changed version triggers a bounded read and SHA-1 confirmation. SHA-1 remains the cross-provider content identity persisted in visible structured metadata; provider versions are only an in-memory invalidation fast path. Cache transitions for model-visible changes commit only when the corresponding context survives the complete tool-result policy chain, and are invalidated if that accepted context is later dropped with its aborted step before reaching the log.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Use a global `ctx.systemPrompt.section()`.** Rejected because one Cordis context can host sessions with different cwd values, while repository-owned text is lower-authority context rather than top-authority provider system content.
|
||||
|
||||
**Inject the baseline on every `agent/pre-step`.** Rejected because repeated history injection wastes tokens, complicates duplicate state, and prevents a structurally stable provider prefix. Prefix composition gives a frozen, logged, per-instance baseline while dynamic append-only messages handle changes.
|
||||
|
||||
**Load both `AGENTS.md` and `CLAUDE.md` in one directory.** Rejected because repositories in transition commonly duplicate guidance across both files. Ordered candidates make precedence explicit and configurable.
|
||||
|
||||
**Parse rendered headings or hidden comments to recover loaded state.** Rejected because instruction prose can contain the same text, causing silent false positives. Persisted JSON metadata provides an unambiguous state channel that is invisible to the model.
|
||||
|
||||
**Summarize files with a model.** Rejected because instruction files are already curated summaries; another model call is nondeterministic and can erase edge-case requirements. Deterministic full text with byte budgeting is simpler.
|
||||
|
||||
## Consequences
|
||||
|
||||
Workspace guidance is isolated per session and shared by both product front doors and every tool presentation mode. Initial instructions benefit from stable prefix caching, while nested and changed content remains durable and replayable. The generic session/agent context contract carries JSON metadata propagated through prompt-submit and post-tool `additionalContexts` arrays without flattening entries.
|
||||
|
||||
Repository text remains untrusted input. Lower-authority user-role framing, explicit precedence language, delimiter escaping, and symlink rejection reduce risk but do not eliminate prompt injection. Permission and sandbox layers treat workspace files as data rather than authority.
|
||||
|
||||
The system is event-driven rather than watch-driven. Edits are not visible at the exact filesystem mutation instant unless that mutation goes through a structured tool; externally changed files are noticed on the next successful structured touch or resume. This keeps the design deterministic and provider-neutral.
|
||||
|
||||
## Deferred
|
||||
|
||||
Bash-derived path reporting, recursive startup scans, file watchers, lowercase defaults, `.claude/CLAUDE.md`, `.claude/rules/*.md`, import directives, ACP `additionalDirectories`, trust acknowledgements, and model-generated summaries are deferred. Same-directory private variants can be configured today; directory rule systems and imports need their own precedence and trust designs.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: Ask-user question capability
|
||||
# Agent Note: Ask-user question capability
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -20,9 +20,9 @@ Providers return `{ answers: [{ id, selected, custom? }] }`. `selected` is alway
|
||||
|
||||
## UI mappings
|
||||
|
||||
`dsh-stdio-agent`'s in-package readline module renders each question, shows each option's `description` on the next line, supports comma/space-separated numeric choices for `multi_select`, accepts free-form custom answers, and rejects pending questions on abort, provider disposal, or stdin EOF. A batched request is asked in order and resolved as one answer object. The stdio provider serializes simultaneous requests with an internal queue so only one prompt owns stdin at a time.
|
||||
`dsh-tui` renders each question as a keyboard overlay, shows option descriptions, supports single- and multi-select choices plus free-form custom answers, and rejects pending questions on abort, provider disposal, or terminal shutdown. Batched and simultaneous requests are queued so one overlay owns keyboard focus at a time.
|
||||
|
||||
`dsh-acp` provides the same seam for ACP sessions. It routes an ask request from the calling `Agent` through the bridge's `agent→sessionId` reverse map and calls ACP `unstable_createElicitation` with a session-scoped form for each question. Single-select options become a `choice` string enum; `multi_select` options become a `choice` array enum; optionless questions use a required `custom` text field. If the client returns both `choice` and non-empty `custom`, the custom answer wins. ACP `decline`/`cancel`, a missing answer, a missing session, and a client without elicitation support all become structured `UserInteractionError`s.
|
||||
`dsh-acp` provides the same seam for ACP sessions. It resolves the calling `Agent` through `ownedRecord`, requiring the forward session-map record at `agent.session.id` to own that exact agent object, and calls ACP `unstable_createElicitation` with a session-scoped form for each question. Single-select options become a `choice` string enum; `multi_select` options become a `choice` array enum; optionless questions use a required `custom` text field. If the client returns both `choice` and non-empty `custom`, the custom answer wins. ACP `decline`/`cancel`, a missing answer, a missing session, and a client without elicitation support all become structured `UserInteractionError`s.
|
||||
|
||||
The ACP mapping deliberately uses elicitation, not `session/request_permission`. `request_permission` is still reserved for the separate permission gate: it is a yes/no-or-policy authorization protocol around tool execution. `ask_user_question` is a general information-gathering tool with optional free-form answers, so ACP form elicitation is the closer protocol fit. The bridge's session routing is shared with the future permission gate, but the user intent is different.
|
||||
|
||||
@@ -42,8 +42,8 @@ ACP elicitation is currently marked unstable in the SDK. The fallback is still s
|
||||
|
||||
The feature gives the model a powerful pause primitive, so prompt guidance matters. The tool description tells the model to ask concise questions and use options when possible. Product policy can later wrap `tools/execute` to restrict when the tool is allowed, but the loop should not special-case it.
|
||||
|
||||
`dsh-user-interaction` and `dsh-tool-ask-user` both live in `packages/ui` because they form one product-facing human-interaction capability. `agent-core` does not load either the tool or a provider. `stdio-agent` opts into the seam, its readline provider, and the model-facing tool. `acp-agent` keeps only the `userInteraction` seam/provider by default: ACP elicitation support is still client-dependent, so an ACP leaf must opt into the model-facing tool deliberately once its client can complete elicitation requests.
|
||||
`dsh-user-interaction` and `dsh-tool-ask-user` both live in `packages/ui` because they form one product-facing human-interaction capability. `agent-core` does not load either the tool or a provider. `dsh-tui-demo` opts into the seam, TUI provider, and model-facing tool. `acp-agent` keeps only the `userInteraction` seam/provider by default: ACP elicitation support is still client-dependent, so an ACP leaf must opt into the model-facing tool deliberately once its client can complete elicitation requests.
|
||||
|
||||
## Testing
|
||||
|
||||
Unit coverage pins provider registration/disposal, duplicate-provider rejection, abort-before-provider, empty-question rejection, structured tool errors through `ctx.tools.execute()`, batched answers, multi-select answers, custom answers, and the model schema including the removal of `value`, `recommended`, `allow_custom`, and `desc`. `dsh-stdio-agent` tests cover option descriptions, queued requests, EOF/abort cleanup, optionless free-form input, invalid option reprompts, duplicate multi-select numbers, and batched question flows. ACP bridge tests drive a real in-memory ACP connection with the real `ask_user_question` tool and verify selected-option, custom-overrides-choice, multi-select, and optionless free-form elicitation paths continue the agent loop.
|
||||
Unit coverage pins provider registration/disposal, duplicate-provider rejection, abort-before-provider, empty-question rejection, structured tool errors through `ctx.tools.execute()`, batched answers, multi-select answers, custom answers, and the model schema including the removal of `value`, `recommended`, `allow_custom`, and `desc`. TUI tests cover option descriptions, queued requests, shutdown/abort cleanup, optionless free-form input, invalid choices, duplicate multi-select selections, and batched question flows. ACP bridge tests drive a real in-memory ACP connection with the real `ask_user_question` tool and verify selected-option, custom-overrides-choice, multi-select, and optionless free-form elicitation paths continue the agent loop.
|
||||
@@ -1,4 +1,4 @@
|
||||
# RFC: The `todo_write` tool — model task list as event-sourced session state
|
||||
# Agent Note: The `todo_write` tool — model task list as event-sourced session state
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -20,7 +20,7 @@ The list is appended as a `todo/write` event carrying the full `{ todos }` snaps
|
||||
|
||||
### NOT a surface event
|
||||
|
||||
`todo/write` is deliberately excluded from `SurfaceEventType`. The surface is the projection that produces the LLM message history (`deriveMessages()`); a todo write produces no conversation message. So it carries no `surfaceOp`, never joins the surface linked list, and never reaches `deriveMessages()` — it is durable, replayable *UI* state that travels alongside the conversation without being part of it. (The dev-mode invariants still require it to sit inside an open turn, which it always does: it is appended mid-step during a tool call.)
|
||||
`todo/write` is deliberately excluded from `SurfaceEventType`. The surface is the projection that produces the LLM message history (`deriveMessages()`); a todo write produces no conversation message. So it carries no `surfaceOp`, never joins the ordered surface, and never reaches `deriveMessages()` — it is durable, replayable *UI* state that travels alongside the conversation without being part of it. (The dev-mode invariants still require it to sit inside an open turn, which it always does: it is appended mid-step during a tool call.)
|
||||
|
||||
### Priority synthesized only at the ACP boundary
|
||||
|
||||
@@ -32,7 +32,7 @@ claude-code V1's item is `{ content, status, activeForm }`; later (V2) it grew i
|
||||
|
||||
### Single owner — no swarm machinery (YAGNI)
|
||||
|
||||
The list belongs to the ONE agent session that called the tool (`exec.agent.session`); a non-agent caller is rejected. There is deliberately no shared/multi-owner scope, no capability seam (interface/impl/consumer), no scope resolver, and no delta protocol. The harness does have subagents, and a shared cross-agent list is conceivable — but building that now means designing for a form the product does not yet have. The whole-list-replace + single-owner shape is what claude-code V1, opencode, and codex all ship; if a shared list is ever needed, the on-log representation would change to per-item deltas (so concurrent writers can't clobber each other) and a scope resolver would choose the target log. That is a future RFC, not speculative scaffolding today.
|
||||
Each list belongs to the calling agent session, and non-agent calls are rejected. There is no shared scope, resolver, or delta protocol. Cross-agent lists would require per-item log deltas and explicit scope selection, so they remain a separate future design.
|
||||
|
||||
### Validation: the cheap middle
|
||||
|
||||
@@ -49,7 +49,7 @@ Four tiers, designed up front:
|
||||
- **Real-Loader path** — the plugin run through `Loader.unwrapExports`, asserting the namespace export shape survives (it HAS `inject`, so a stray default would crash at load — postmortem/0001).
|
||||
- **Full-loop integration** — a scripted mock model calls `todo_write` through the real agent loop; the `todo/write` event lands and a second call replaces it.
|
||||
- **`session/load` replay** — a persisted `todo/write` re-emits the `plan` update when a fresh ACP bridge loads the session.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot golden gains the `plan` notification and the log event.
|
||||
- **With-key e2e + snapshot** — a real prompt induces a `todo_write`; the snapshot expected output gains the `plan` notification and the log event.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
68
.agents/notes/implemented/feature/2026-06-30-hook-bridges.md
Normal file
68
.agents/notes/implemented/feature/2026-06-30-hook-bridges.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Agent Note: dsh-hooks-claude + dsh-hooks-codex — the Claude Code / Codex hook bridges
|
||||
|
||||
Status: implemented
|
||||
|
||||
## Problem
|
||||
|
||||
The harness's extension surface is its typed interception seams ([the interception-seams Agent Note](2026-06-30-interception-seams.md)): a "native hook" is just an ordinary cordis plugin subscribing to `agent/session-start`, `agent/prompt-submit`, `tools/pre-execute`, `tools/post-execute`, `agent/turn-continuation`, `subagent/start`, `subagent/end`. But users arrive with **existing** Claude Code (CC) and Codex hook configs — a `hooks.json` (or a settings file's `hooks` key) full of shell-command hooks — and want those to run unmodified. This Agent Note introduces the two **bridge plugins** that translate that external shell-hook protocol onto the typed seams, built on the shared wire-protocol library ([the hook-protocol-lib Agent Note](2026-06-30-hook-protocol-lib.md)).
|
||||
|
||||
The framing that shapes the whole design: **a bridge is a compatibility adapter, not a power tool.** Anything a bridge does (block a tool, inject context, force continuation, observe a subagent) a native cordis plugin does more powerfully — typed returns, full `ctx`, no serialization boundary. The bridge's reason to exist is to run the explicitly supported subset of external CC/Codex command hooks. That keeps each bridge thin: parse the config, pick a matcher mode, build the per-event payload, call `runHook` + `mergeHookOutputs` from the shared lib, and map the neutral outcome onto a seam Decision. The package READMEs own the exact current unsupported-event and partial-field inventory against the official protocols.
|
||||
|
||||
## Decision
|
||||
|
||||
Two independent plugins in the `packages/hooks/` group, each a function/namespace plugin (`name`/`inject`/`Config`/`apply`, NO default export — see [postmortem 0001](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)) injecting only `bash`:
|
||||
|
||||
- **`dsh-hooks-claude`** — the CC dialect. Seven of Claude Code's current hook points: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop`, `SubagentStart`, and `SubagentStop`. Owns CC-shaped per-event stdin payloads (a base of `session_id`/`transcript_path`/`cwd`/`hook_event_name` plus per-event fields), `CLAUDE_PROJECT_DIR` plus `${CLAUDE_PLUGIN_ROOT}`/`${CLAUDE_PROJECT_DIR}` substitution, and the literal-or-regex matcher mode. `transcript_path` is the persistence locator result or `''`; stdin carries a **trailing newline**.
|
||||
- **`dsh-hooks-codex`** — five of Codex's current hook points: `PreToolUse`, `PostToolUse`, `SessionStart`, `UserPromptSubmit`, and `Stop`. It uses an always-regex matcher, Codex-shaped snake_case payloads with `turn_id`/`model`/`permission_mode` extras written WITHOUT a trailing newline, no Codex plugin-env injection or config-time placeholder substitution, and no pre-tool approval or rewrite path. `transcript_path` is the same locator result or `null`; tool payloads carry the real `tool_name` in the reduced `tool_input: { command }` shape.
|
||||
|
||||
### Outcome → Decision mapping
|
||||
|
||||
Each bridge maps the neutral `MergedHookOutcome` from the shared lib onto the seam's typed Decision:
|
||||
|
||||
| Seam | CC | Codex |
|
||||
|---|---|---|
|
||||
| `agent/session-start` (emit) | additionalContext → `agent.inject()` | plain-stdout output → additionalContext → `agent.inject()` |
|
||||
| `agent/prompt-submit` | `deny`→`block`; context-only→delegate+fold | `block`→`block`; context-only→delegate+fold |
|
||||
| `tools/pre-execute` | `deny`→`deny`; `ask`→`ask` | `block`→`deny` (no allow/ask) |
|
||||
| `tools/post-execute` | `deny`→`block`+feedback; context-only→delegate+fold | same |
|
||||
| `agent/turn-continuation` | blocking Stop → `continue` (reason = next-step steering) | same |
|
||||
| `subagent/start` (emit) | additionalContext → inject into a live in-process child; a remote child has no local injection target | unsupported by this bridge |
|
||||
| `subagent/end` (emit) | observe-only | unsupported by this bridge |
|
||||
|
||||
The CC bridge's `ask` result is a real permission path, not a terminal bridge decision: `dsh-tools` resolves it through the optional [approval seam](2026-07-06-approval-seam.md). A composed ACP answerer prompts the owning editor session and `allowed-once` proceeds; without an ApprovalService or answerer, the call fails closed to `deny`.
|
||||
|
||||
### Context source is always the plugin (the mislabel guard)
|
||||
|
||||
`agent.inject()` defaults a missing `MessageSource` to `{ kind: 'user' }`, so every bridge `inject()` and `HookContext` passes `{ kind: 'plugin', plugin: 'hooks-claude' | 'hooks-codex' }`. Unit coverage pins the resulting `context/message.source` as the plugin rather than the user.
|
||||
|
||||
### Adding context is not a veto — delegate, then prepend
|
||||
|
||||
A hook that only attaches `additionalContext` (no block/deny) is NOT a decision the bridge should return on its own: returning `allow`/`accept` from a waterfall listener WITHOUT calling `next()` short-circuits every later `agent/prompt-submit` / `tools/post-execute` listener, so a policy/sandbox plugin registered after the bridge would never see the prompt. Each bridge therefore delegates via `next()` before adding its context to the downstream decision. Both seams carry ordered `additionalContexts` arrays, so the bridge prepends its separately sourced entry while preserving every downstream source, envelope, and metadata field; a downstream prompt block still drops all context because the prompt never reaches the model, while post-tool block semantics may explicitly retain contexts. Code Mode ferries the same array through the outer `run_code` result. Only a real `deny`/`block` from the hook itself short-circuits. Tests assert a later listener can still block a prompt a context-only hook allowed and that retained prompt and post-tool contexts remain separate.
|
||||
|
||||
### CLAUDE_PROJECT_DIR defaults to the session workspace
|
||||
|
||||
Claude Code always exports `CLAUDE_PROJECT_DIR`, and common unmodified hooks reference `$CLAUDE_PROJECT_DIR` for project-relative paths. An explicit `config.projectDir` wins; when it is omitted (the default ACP wiring configures only `configPath`), the bridge defaults the env var per-run to the agent's session workspace — the same `session.header.cwd` the hook already runs in — rather than leaving it empty. So a stock project-relative hook works in the default setup.
|
||||
|
||||
### Containment
|
||||
|
||||
The config is parsed ONCE at load; a read/parse failure logs and registers nothing rather than crashing boot (a typo'd path must not take the agent down). Only shell-form `type: 'command'` hooks run for CC; `http`, `mcp_tool`, `prompt`, and `agent` handlers are parsed-and-skipped. Codex runs only synchronous command handlers and skips `async: true` or non-command entries. The emit-listener paths (`session-start`, `subagent/start`) run detached, with their `inject` contained in a `.catch` that logs (a throwing inject must not break session boot or the loop).
|
||||
|
||||
### Where hooks run, and where their config comes from
|
||||
|
||||
Hooks run in the agent's session workspace, so relative paths target the user's project. `configPath` is resolved once against the process launch cwd and applies to every session. Per-session project-local discovery remains deferred under `TODO(per-session-hook-config)`.
|
||||
|
||||
## Deferred compatibility gaps
|
||||
|
||||
- **Tool-input rewrite.** A CC/Codex `updatedInput` is logged + warned, not honored — input rewrite is a deferred consistency-design problem ([the pre-tool-input-rewrite Agent Note](../../proposed/feature/2026-06-30-pre-tool-input-rewrite.md)), because the pre-execution args are read by `tool/call` audit + `assistant/message` history + ACP/tool-bash presentation, so an honest rewrite is a design unit, not a field.
|
||||
- **Stop loop-guard** (`TODO(stop-loop-guard)`). Claude Code supplies `stop_hook_active` and overrides a hook after eight consecutive blocks; Codex supplies `stop_hook_active` but documents no equivalent cap. Both bridges always report `false`, so a Stop hook that unconditionally blocks force-continues every step — a hook author must self-limit until state tracking lands.
|
||||
- **Hook `continue:false` (hard halt).** A hook can ask to halt the whole run (CC/Codex `continue:false`); the shared merge folds it into `MergedHookOutcome.stop`/`stopReason`, but no bridge acts on it (`TODO(hook-continue-false)`) — the interception seams have no "hard-halt the agent" primitive yet (a Decision blocks/steers a single point, not the run). Deferred with the loop-guard work; the halt request is recorded in the `hook/result` log, and the hook keeps its per-point effect (decision/context) meanwhile.
|
||||
- **Config discovery.** The path is explicit in `cordis.yml` and process-level (see above); the full multi-layer CC/Codex precedence walk, per-session project-local discovery, and the trust/hash model are not reimplemented (`TODO(per-session-hook-config)`).
|
||||
- **Session-start / subagent-start context is best-effort (`TODO(session-start-gating)`).** Both hooks run detached from startup, so their context is injected when ready but may miss the first request or a short-lived child. Guaranteeing first-request delivery requires an awaited startup seam.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Concurrent per-point hook execution.** The reference engines run a point's matched hooks concurrently and fold the results. These bridges run them **serially** (`await` per hook inside the match loop) and fold with the same most-restrictive merge. Serial is deliberate: it keeps each hook's `hook/invoked`/`hook/result` pair adjacent and in a deterministic order in the session log, and the fold is order-independent for the decision (`deny > ask > allow`) so the outcome matches. The cost is latency (hook *N* waits for hook *N−1*) and that per-hook timeouts are not overlapped — acceptable for the hook counts real configs use; revisit if a config ever fans out enough for the wall-clock to matter.
|
||||
|
||||
## Consequences
|
||||
|
||||
Matcher semantics, exit-code handling, and merge precedence live in `dsh-hook-protocol`; each bridge only parses config, builds dialect payloads, and maps outcomes. Per-file coverage includes config branches plus end-to-end mappings through a real loop, `dsh-bash-local`, and shell scripts, while a real-Loader smoke guards the package export shape. Native plugins bypass the wire protocol and return typed decisions directly.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user